Saturday, November 16, 2013

Javascript JSON Trailing Comma and IE

Seems like Google Chrome and Firefox are both pretty tolerant when it comes to trailing commas in JSON. However IE can be really pissed off by those extra characters.

Two symptoms I typically see:

  • Syntax Error: IE will simply report a syntax error like this: SCRIPT1028: Expected identifier, string or number
JSON string that can cause above error message:

var x = {
    a: 1,
    b: 2,};

  • Undefined object: when the trailing comma is in an array declaration, IE will add a undefined object at the end of the array (Firefox, and Chrome does not do this). For example, for the code below, my test results are: IE -> 4, Firefox -> 3, Google Chrome -> 3

var x = [1, 2, 3,];
alert(x.length);

       This usually will cause unexpected error in the code down below, which may loop through the array and try to access the undefined last element.

It is the second kind of problem that causes the most headache. Because in the first case, IE will pinpoint line number so that we can fix the problem immediately. In the second case, you are likely to get an obscure error message from a code that is trying to access the undefined last object.

Make Coldfusion 1000 Times Faster

Here is a well kept secret to make Coldfusion runs 1000 times faster: make your initial implementation 1000 time SLOWER.

Kidding aside, here are some performance tips:

  • Clean up debug code, especially CFDUMP. I have a production server that runs a query for 14 minutes, turns out a CFDUMP for debugging purpose is left behind. After it's removed, the query only took less than 5 seconds to finish;
  • JSON serialization is not as fast as you wished. I have a query that took 1 second to get result from SQL server, but then it took about 10 seconds to serialize the result to JSON string;
  • Similarly WDDX, SOAP could take significant amount of time to serialize large object;
  • Use AMF binary communication if possible (at least if you use Flex, or Coldfusion to Coldfusion communication, do give AMF some serious consideration). Moving from SOAP webservice call to AMF have make our response time order of magnitude faster;
  • Other usual suspects that are applicable to many other programming languages: loop, nested loop, multiply vs. divide, ...
  • Profiling: I was not able to find a good profiling tool for Coldfusion, so end up roll my own hand coded profiler. It's a little cumbersome, requires manually insert check points to collect performance statistics. Never the less, it helps in quickly identify bottlenecks;
  • Caching: turn on caching in Coldfusion is very easy. So, whenever it make sense, do turn on query caching, page caching, ... This often can be a 10 second job that improves performance by 100 times;
  • Coldfusion Admin: trusted cache, disable debugging
  • Always use fully qualified variable names to save scope lookup time;
  • Application.cfc: double check, triple check this file, because any performance issue here will affect ALL the pages of your website;
  • use cfqueryparam instead of text in cfquery;
  • Fine tune JVM parameters
  • Hardware: CPU, RAM, network, and Disk (again measure and identify bottleneck, before purchase any expensive upgrades)
  • Scale out: use load-balancer to distribute work load
  • Last but not least: make sure your SQL queries are all in good shape. Often times, Coldfusion server itself is not the bottleneck, your SQL server is;

Tuesday, October 22, 2013

ExtJS Image Component with Link

Trying to create an image in ExtJS that links to another page? I have tried direct html, button with image src, ... They all have layout problems. The image component seem to work the best, but there is no direct way to put a link in it. Finally, I found a way using the autoEl configuration. It works beautifully.
 See it in action on jsfiddle: http://jsfiddle.net/hnPbm/
See the sample code for more details:

Saturday, October 12, 2013

Always Maximize Browser Window (Selenium Web Driver + JUnit + ExtJS)

Selenium WebDriver test will fail if you ask it to click a button that is out of sight. Since WebDriver is designed to simulates human interaction, this behavior is correct. But for most of us, scrolling the object into view is a trivial task that is not the subject of testing. So to reduce a lot of trivial errors like this, it's usually preferable to start browser window in maximum size.

Here is the Java code to maximize browser window:
  WebDriver driver;
  ... 
  driver.manage().window().maximize();

Examine ExtJS Grid Data (Selenium Web Driver + JUnit + ExtJS)

This post continue the same idea I used in previous posts: use Javascript to examine data, query component and use WebDriver to simulate user interaction, and JUnit to driver assertions. Below are code snippets that will get data of the currently selected row in a data grid, and do some assertion against this data:

Remeber to Reset IE Zoom (Selenium Web Driver + JUnit + ExtJS)

If you are unlucky as me, you may be doing research of this error message right now:

org.openqa.selenium.remote.SessionNotFoundException: Unexpected error launching Internet Explorer. Browser zoom level was set to 125%. It should be set to 100% (WARNING: The server did not provide any stacktrace information)

Everything was working perfectly, and all tests passed with green color. Then all of a sudden all test cases failed with the error above. It's very a clear error message, and you don't need any research to fix the problem. Just open IE and visit the website under test, then set zoom level to 100%. Then all tests should pass again (of course, only if they were passing before).

However, I do not want to ensure zoom level before every test runs, so I found the following code snippet to ask Selenium WebDriver to ignore IE zoom level:
Things were quiet for a while until I was hit by a strange error message about object not clickable. Of course, when I put it in context, you knew I should check for zoom level immediately. However, in real life, it threw me a big loop, and I spent a good hour researching how to click ExtJS component or related catches that I may not know before. Until, I accidentally saw the zoom level is not 100%. Now after I reset zoom level back to 100%, problem is gone. I imagine WebDriver has some coordinate conversion issue when IE's zoom level is 100%.

So what is the conclusion? Two bullet points for IE testing:

  • Do NOT use the code snippet above to ignore zoom level
  • Always ensure zoom level is 100%

Thursday, October 10, 2013

Capture Runtime JavaScript Errors (Selenium Web Driver + JUnit + ExtJS)

In my previous post Capture ExtJS Ajax Error (Selenium Web Driver + JUnit + ExtJS), I showed how to capture Ajax errors. In this post, I'm going to show how to use similar approach to capture general Javascript errors.

First, need to inject the following Javascript code to the page under testing:
The Java code to inject the JS code above and to check runtime script error:

Wednesday, October 02, 2013

sendKeys to combobox, textbox or datefield (Selenium Web Driver + JUnit + ExtJS)

In my previous post Locate ExtJS Component (Selenium Web Driver + JUnit + ExtJS), I introduced some code snippets to find an ExtJS component using ComponentQuery. However, the elements located using that approach generally do not accept user input (except mouse click seems to work for button). Turns out ExtJS render these components using a lot of HTML markups wrapping around the real input element. The inputEl is usually the HTML element that accepts user input. So to send keys to them, a slight change is needed from the code I published in the previous post. We will need to drill down 1 level to retrieve HTML id of inputEl, and get its DOM element:

Tuesday, October 01, 2013

Capture ExtJS Ajax Error (Selenium Web Driver + JUnit + ExtJS)

Here is sample code to check ExtJS Ajax error. It's highly recommended to check this error during teardown(@After), because WebDriver won't automatically capture these problems.

First, need to inject the following JS code to the page under testing:
 Java code to inject the code and to check ExtJS Ajax error:

Monday, September 30, 2013

Examine ExtJS Store Data (Selenium Web Driver + JUnit + ExtJS)

Part of the goal of unit test is to check result against expected values using various assertions. An essential data structure in ExtJS web application is store. So, no test case is complete without some assertion against the stores. For example, we might want to verify number of records, or value of certain column, ... Here are some code samples to access store data, get number of rows, or retrieve value of a column:

Saturday, September 28, 2013

Wait for AJAX Complete (Selenium Web Driver + JUnit + ExtJS)

With AJAX, Seleinum testing can have a lot of tricky timing issue. Tests can fail due to slightly delayed AJAX response. Implicit wait solved the problem to some extent, but would be nicer to have a way to wait for all AJAX calls to be finished.

Here is a solution for ExtJS AJAX requests:
Core concept is this JS code: Ext.Ajax.requests && !_.isEmpty(Ext.Ajax.requests). It works for ExtJS 4 only, might change for future or older version of ExtJS due to how AJAX calls are managed.

Thursday, September 26, 2013

Locate ExtJS Component (Selenium Web Driver + JUnit + ExtJS)

Selenium WebDriver offers many ways to find an element. You can find it by class, name, partial link text, xpath, css selector, ... However, when it comes to ExtJS component, none of them works very well, because ExtJS page elements tend to be generated dynamically, their xpath/id/... are all moving target. Trying to figure out a repeatable way to locate any of them can be very time consuming and the result is fragile.

Fortunately, as all ExtJS developers should know, Ext.ComponentQuery provides a powerful and very reliable way to find your component. So, the idea is to use Ext ComponentQuery to find the component of interest, then pass it along to WebDriver. Along this line, below is my approach to locate components:
  • PageObject owns all the top level components, and specify a ComponentQuery that can uniquely identify them;
  • All views or components of interest have their hierarchy represented in the test code, and at each level, they should have a ComponentQuery that can unique identify each of them within the container;
  • When it's time to get the element, we should traverse up from element to page, rebuild the fully qualified component query

Ok, some sample code might explain the idea better:

Wednesday, September 25, 2013

Take Screenshot on Failure (Selenium Web Driver + JUnit + ExtJS)

First a little background: this is the first of a series of lessons I learned using Selenium Web Driver to test ExtJS web application.

It's usually desirable to always take a screenshot when test failed. The concept of JUnit rules makes this an easy task. First, implement a rule as shown below:

Then in your test classes, you just need to declare this rule, then all test methods in it will automatically take screenshot on failure.
The examples here are just for illustration purpose. In real world application, you might make some improvement like:

  • Make sure the file name is legal for the OS (for example, the following characters are not allowed in Windows file name: \/:?*<>|\
  • Pictures might be further organized into folders by test cases?
  • The rule might be declared in a base class so that all test cases have the same behavior without extra boiler plate code

Wednesday, August 28, 2013

Link shared folder in SVN

Found this article, and followed it's instruction. Very straight forward.
TortoiseSVN and Subversion Cookbook, Part 4: Sharing Common Code
https://www.simple-talk.com/dotnet/.net-framework/tortoisesvn-and-subversion-cookbook-part-4-sharing-common-code/

What I did is just these steps (assuming both folder1 and folder2 want to have a subfolder: sharedlib):
  1. Open repo-browser
  2. Drag and drop shared folder into a root folder (folder name sharedlib, level same as folder 1 and folder 2)
  3. Right click on folder 1,Show Properties > New > External > New, local path: sharedlib, URL: ../sharedlib
  4. Do the same thing for folder 2
Now both folder 1 and folder 2 have sharedlib.

OpenCV in Visual Studio 2012

It's surprisingly easy, just took about 15 minutes to see some example code running.

Steps to create the first OpenCV project using Visual Studio 2012
  1. Download: opencv.org > OpenCV for Windows
  2. Run the downloaded executable and decompress to d:\opencv (to help the examples below)
  3. Open Visual Studio 2012, create new project  Visual C++ > Win32 > Win32 Console Application
  4. Add "Include Directories": Project Properties > Configuration Properties > C/C++ > General > Additional Include Directories, add: D:\opencv\build\include
  5. Change to static link (optional, but if this is not done, then the library below need to be changed slightly): Project Properties > Configuration Properties > C/C++ > Code Generation > Runtime Library, select Multi-threaded (Debug)
  6. Add "Library Directories": Project Properties > Configuration Properties > Linker > General > Additional Library Directories, add:D:\opencv\build\x86\vc11\staticlib (or  D:\opencv\build\x86\vc11\lib if you don't want static link)
  7. Add lib files: Project Properties > Configuration Properties > Linker > Input >Additional Dependencies, add the libraries listed below. (Note: 1. comctl32 is a Windows library; 2. replace "246" to version number you have; 3. files ending with "d" is debug version, for release configuration, remove it)
IlmImfd.lib
libjasperd.lib
libjpegd.lib
libpngd.lib
libtiffd.lib
opencv_calib3d246d.lib
opencv_contrib246d.lib
opencv_core246d.lib
opencv_features2d246d.lib
opencv_flann246d.lib
opencv_gpu246d.lib
opencv_haartraining_engined.lib
opencv_highgui246d.lib
opencv_imgproc246d.lib
opencv_legacy246d.lib
opencv_ml246d.lib
opencv_nonfree246d.lib
opencv_objdetect246d.lib
opencv_ocl246d.lib
opencv_photo246d.lib
opencv_stitching246d.lib
opencv_superres246d.lib
opencv_ts246d.lib
opencv_video246d.lib
opencv_videostab246d.lib
zlibd.lib
comctl32.lib


Now, for a quick sanity check, run one of the sample code from opencv.org (the example I used is "Hough Circle Transform" http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/hough_circle/hough_circle.html
 
Environment:
  • Windows 7
  • Visual Studio 2012
  • OpenCV 2.4.6

Monday, August 26, 2013

Selenium WebDriver

Two lessons learned

  • IE driver need to start with "ignoreZoomSettings" true
  • JavascriptExecutor return for JS object

IE driver need to start with "ignoreZoomSettings" true

Got this error when using WebDriver for IE: "
org.openqa.selenium.remote.SessionNotFoundException". Turns out I was using IE to visit the same site I'm testing, and I adjusted zooming to 120% there. Fix for this problem is easy, simply change IE web driver creation to this:


DesiredCapabilities caps = DesiredCapabilities.internetExplorer();
caps.setCapability("ignoreZoomSetting", true);
driver = new InternetExplorerDriver(caps);

JavascriptExecutor return for JS object

What can be returned from JavaScriptExecutor.executeScript? According to the documentation, these are the types that it handles: Boolean, Long, String, List, or WebElement or null. I did a little experiment, and pleasantly surprise by the fact that array of JSON can be handled too. The return will be ArrayList of Maps. Calling obj.toString (Java side) will actually give me expected JSON string too.
Figure: Return Object as Shown in Eclipse Variables View

Thursday, August 22, 2013

Selenium WebDriver sendKey is very slow in IE

sendKey for IE is very slow. Turns out there is an easy fix. Simply download 32 bit IE driver (guess it's my bad used 64 bit driver before).

Environment:
Windows 7 Pro 64 bit
IE 10
Selenium 2

Saturday, August 17, 2013

Make Custom Fields in Coldfusion Solr Collections not Searchable

How

Here is how I did it, might not be the best way, but it works:

  • Update Solr schema
    • Change custom fields definition to: type: string, indexed: false
    • Comment out all the copyField definitions for the custom fields
  • Purge all Solr collections and reindex all of them
Details of the XML Changes
Changes of custom field definitions
Before:
<field name="custom1"   type="text"   indexed="true" stored="true" required="false" />
<field name="custom2"   type="text"   indexed="true" stored="true" required="false" />
<field name="custom3"   type="text"   indexed="true" stored="true" required="false" />
<field name="custom4"   type="text"   indexed="true" stored="true" required="false" />
After:
<field name="custom1"   type="string"   indexed="false" stored="true" required="false" />
<field name="custom2"   type="string"   indexed="false" stored="true" required="false" />
<field name="custom3"   type="string"   indexed="false" stored="true" required="false" />
<field name="custom4"   type="string"   indexed="false" stored="true" required="false" />
Changes for the copyField definitions
Before:
<copyField source="contents_*"  dest="contents" />
<copyField source="custom*"  dest="contents" />
<copyField source="title"  dest="contents" />
<copyField source="contents_*"  dest="contentsExact" />
<copyField source="custom*"  dest="contentsExact" />

After
<copyField source="contents_*"  dest="contents" />
<!--
<copyField source="custom*"  dest="contents" />
-->
<copyField source="title"  dest="contents" />
<copyField source="contents_*"  dest="contentsExact" />
<!-- 
<copyField source="custom*"  dest="contentsExact" />
-->
Environment
Coldfusion 10

Why


Why do I need to make custom fields not searchable?

Migrating Coldfusion collections from Verity to Solr is never straightforward. See my other post: Solr in Coldfusion 9 for some lessons I learn during migration.

Fast forward to last week (a couple of months after the production servers are migrated), another surprise surfaced. Some search will result in random text that does not appear to be relevant. After some investigation, I finally found out it's due to unexpected behavior of custom fields.

Custom field in Verity was not seem to be searchable. After migration to Solr, the custom fields are indexed and became searchable. It may be a good thing for people who want it to be searchable. But, in my case they were used for storage only, contains database ID and other contents that should not be used for search purpose.

Parting Words

It took me nearly half a day to finally figure out how to do it. Hopefully it can save someone a few hours of frustration. I started by naively changing only custom fields to index: false, and turns out they are still searchable. Google search did not turn up anything helpful, except a few posts asking the exact question I was asking. Finally after exploring the Solr admin pages, and looking at schema of the collections, I found out that they are also copied into searchable fields, and type: text will also trigger analyzer on it.

Tuesday, July 30, 2013

ExtJS multiselect problem when shift click to select entries

Environment

IE  10; ExtJS 4.1

Problem

I am using ExtJS "multiselect" control. When trying to select multiple rows using Shift+Click, I'm getting a messy screen (Figure 1). The data behind the scene is correct, but the look is disturbing. Ideally, it should look like Figure 2. 
Figure 1. Screenshot showing the problem

Figure 2. Screenshot showing the expected behavior

Solution

Solution to this problem is relatively trivial, but not obvious. Just add the following attribute to the multiselect control, you will get behavior like Figure 2. I found the solution by looking up example hosted on Sencha web site, and found the only difference is in this "ddReorder" attribute. Once it's added, the selection behavior is as expected. Drag drop reorder is harmless for what I am working on.
ddReorder: true 

Friday, July 19, 2013

Adobe Flex RSL Error in IE

This is another case which I know the cause and the solution for a problem. But I don't understand why it would cause a problem.

Environment:
  - IE10, Win7, Flash Player 11.8 debugger
  - Google Chrome  28.0 with Flash Player 11.8

Symptom:
  Flex application stopped  working in IE. Always getting this error message: "RSL Error 1 of 2". RSL is short for Runtime Shared Library. Using ServiceCapture, I do see the main SWF loaded, and the first RSL loaded, but no the rest.
  The same Flex application works perfectly in Google Chrome.

Cause:
  In the the IE Security setting, "Enable Protected Mode" is unchecked.


Solution:
  Go to IE > Tools > Internet Options > Security, make sure "Enable Protected Mode" is checked. Then restart IE. Voila, it all works again. In retrospect, I might turned it off when I was trying to set up Selenium test framework.

Question:
  It's still beyond me why "Protected Mode" seems to make IE 10 more compatible. Just a few days back I had another problem with Office Web application due to the exact same cause. An extra IE windows will always popup when I open an Office 2013 application in Excel. The problem is gone after I enabled protected mode.

----
Flex: a RIA web application development framework based on Adobe Flash technology

Tuesday, July 09, 2013

Upgrading Coldfusion 9.0.0 to 9.0.1 with HotFix 2

Instructions from Adobe are not clear or simply wrong sometimes.
Here is actual steps I took:
Environment: Windows XP, CF 9.0.0 Server Installation, IIS

  • Update Coldfusion from 9.0.0 to 9.0.1
  • Apply Cumulative Hot Fix 2 | Coldfusion 9.0.1 (http://helpx.adobe.com/coldfusion/kb/cumulative-hot-fix-2-coldfusion-1.html)
    • Download and extract files:
    • Go to your {ColdFusion-Home}/lib directory and make a backup of lib folder (I just copy and paste the folder in the same place)
    • Apply update JAR file
      • In the ColdFusion Administrator, select System Information page by clicking the "i" icon in the upper-right corner. 
      • In the Update File text box, browse and select chf9010002.jar located under CF901/lib/updates directory. 
      • Click Submit Changes. 
    • Stop ColdFusion instance. 
    • Update CFIDE
      • Go to {CFIDE-HOME} and make a backup of CFIDE folder (I just copy and paste the folder in the same place)
      • Go to extracted CFIDE folder (typically CFIDE-901\CFIDE), copy all files into current CFIDE folder and overwrite existing files
    • Update WEB-INF
      • Go to {ColdFusion-Home}/wwwroot/WEB-INF directory and make a backup of WEB-INF folder (I just copy and paste the folder in the same place)
      • Go to extracted WEB-INF folder (typically CF901\CF901\WEB-INF\WEB-INF), copy all file into the current WEB-INF folder and overwrite existing files
    • Update lib folder (it's already backed up at the begining)
      • Go to extract lib folder (typically CF901\CF901\lib), copy all files (exclude the "updates" folder) into the current lib folder and overwrite existing files
    • Start Coldfusion instance
To Uninstall: 
  • Stop Coldfusion
  • Delete CFIDE, WEB-INF and lib folders, rename the copied folders back to their original name
  • Start Coldfusion

Tuesday, April 30, 2013

Breaking change in Coldfusion 10 vs Coldfusion 9

Here is one breaking change:
  • DeserializeJSON of null value
Test Code to show the change:
result = deserializeJSON('{"test": null}');
writedump(result);
writeoutput(result.test);

In CF9, result.test is string value: "null".
In CF10, the above code will have error, because result.test is "undefined".

Debugging DotNet component on Coldfusion Server

Here is a recipe to setup Visual Studio debugging of remote Coldfusion server's .NET application.

Environment: Visual Studio 2012, Coldfusion 9
Setup:
  1. Coldfusion Server: 
    1. install "Remote Tools for Visual Studio 2012 Update 2"
    2. Start "Remote Debugger Configuration Wizard" and configure accordingly
    3. Start "Remote Debugger" As Administrator
  2. Visual Studio computer:
    1. Click Debug > Attach to Process
    2. In "Qualifier" type the Coldfusion server's computer name
    3. Check "Show processes from all users"
    4. Click "Refresh"
    5. Select "JNBDotNetSide.exe"
    6. Click "Attach"

Done!

To see it in action, set a break point in your code, load a Coldfusion page that invokes your .NET application, Visual Studio debugger should break accordingly.

Wednesday, April 03, 2013

test embedding Skydrive Spreadsheet

Above is an example to embed skydrive document to blog. Only catch? Use the "JavaScript" instead of the "Embed Code" version. It works beautifully, impressive.

Wednesday, March 20, 2013

ExtJS Store Shared Between Multiple ComboBox Controls

Environment: ExtJS 4.1.3, auto loaded store, shared by multiple comboboxes

Problem:
Combobox with auto-complete behaves strangely. Sometimes, the list is obviously filtered down by something else.

Cause:
The same data store instance is shared by multiple combobox on the page. "doQuery" function in the combobox will try to reserve the store's filter that is not created by the combobox itself (see ComboBox.js > clearFilter, and doQuery implementation). This design allows the auto-complete filter of different dropdown pickers to interfere with each other.

Solution:
1. create custom control, then override "doQuery" function to call store.clearFilter()
Pros:
- Clean, reusable design
Cons
- Validation still has cross contamination of conflicting filtering
- In cases you want to preserve an existing filter, this solution need more tweaks or configuration input to make it working

2. Declare store of the combobox using the following style: store: {type: 'myStoreAlias'}, instead of store: 'myStoreName'. This way it will ensure each combobox has their own instance of the store, thus not interfering each other's filtering.
Pros:
- No other hidden surprises lurking around
Cons:
- Pretty much have to abandon the store: 'myStoreName' completely, because in a module design, you just don't know which combobox may live in the same page with another one and using the same store
- Network and memory overhead to maintain multiple copies of the same data

Wednesday, March 13, 2013

Coldfusion Error: "Value can not be converted to requested type."

Symptom: All of a sudden the test cases that worked before, stopped working. Error message is: "Value can not be converted to requested type.". Wasted hours trying to diff and check typos with no avail. Finally, surrender, and Googled the following article published in 2006:

ColdFusion Query Error: Value Can Not Be Converted To Requested Type

http://www.bennadel.com/blog/194-ColdFusion-Query-Error-Value-Can-Not-Be-Converted-To-Requested-Type.htm

So, it is an old problem that never go away. Everything clicks now. Database schema was just updated with a few new columns. "SELECT *" combined with CFQUERYPARAM seems to have some caching issue and everything just messed up from there.

Solution? remove SELECT *, change to use explicit column names. The problem simply goes away for good.

Friday, March 01, 2013

HTML + JavaScript Charting Solutions

  • Ext JS, Example (good for intranet application)
  • d3.js (extremely popular at this moment)
  • Dojo Charting
  • YUI Chart
  • Google Chart (Big down side, at least at this time, is that Google will receive every single data point you want to plot. Not a good thing for any data that requires privacy!)
  • Flotr2
  • Flash charts? lots of good choices there, but not accessible on mobile clients, and in danger of being completely out in a few years?

Tuesday, February 19, 2013

Support JSONP in Coldfusion

Let say you already have a Coldfusion page that serves JSON data to your AJAX web application, now you want to support JSONP from cross domain client call. Here is the code snippet that will do it:

Solr in Coldfusion 9


Some tips for using Coldfusion 9 Solr search:
  • Lesson Number 1: Make sure you review ALL the files under your collections \conf folder, many of them need to be commented out or changed from default values. Here is an incomplete list of what I changed:
    • Commented out words in protwords.txt
    • Deleted content in spellings.txt
    • Updated synonyms.txt, deleted the test words, and added new synonyms based on the vocabulary specific to my application's domain
    • schema.xml:
      • Updated default query operator from AND to OR
    • solrconfig.xml
      • commented out "solr", "rocks" and other similar configurations
    • Enable term highlighting (see http://help.adobe.com/en_US/ColdFusion/9.0/Developing/WSe9cbe5cf462523a0-5bf1c839123792503fa-8000.html \), but it's still not quite there. Still looking for a complte solution for this one.
  • In Coldfusion Admin, change Solr buffer limit from default value 40 to 80 (http://bloggeraroundthecorner.blogspot.com/2009/08/tuning-coldfusion-solr-part-1.html)
  • Ensure "Coldfusion 9 Solr Service" is started and set to "Automatic" startup type
  • To access Solr collection from remote Coldfusion servers, 
    • For the remote CF servers: change Solr server name in Coldfsuion Admin, DATA & Service > Solr Server > Solr Host Name
    • For the Solr Host: Allow remote server to have access to its Solr instance. Configure Jetty to listen on all incoming IP address. Coldfusion's default configuration is allow only 127.0.0.1, thus disabling remote CF server's access to it's hosted Solr instance. To change this, just open jetty.xml in {coldfusion home}\solr\ect, and comment out the host line which restricts to listen on 127.0.0.1 only (interestingly, I found this solution from this link: http://helpx.adobe.com/coldfusion/kb/coldfusion-9-limit-access-solr.html, which claims Coldfusion's default configuration is to listen on all ip addressed
  • Web access to Solr admin: http://{Your CF Server Name}:8983/solr/
  • Last and a big one, when you try to update multiple indexes, you will see error like this:
Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers4_try_again_later

Stack Trace:
...
org.apache.solr.common.SolrException: Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers4_try_again_later

Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers4_try_again_later

request: http://localhost:8983/solr/.../update?commit=true&waitFlush=false&waitSearcher=false&wt=javabin&version=1
    at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:424)
    at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:243)
    at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
    at coldfusion.tagext.search.SolrUtils.commitToServer(SolrUtils.java:1024)
    at coldfusion.tagext.search.SolrUtils.addDocument(SolrUtils.java:664)
    at coldfusion.tagext.search.IndexTag.doQueryUpdate(IndexTag.java:929)
    at coldfusion.tagext.search.IndexTag.doStartTag(IndexTag.java:254)
    at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2722)


This error will be consistently showing up in production environment with even very light traffic, and became a big show stopper for the usage of Solr collection provided by Coldfusion (does not seem to be a Solr problem, instead a problem of Coldfusion integration problem). 

Root cause seems to be that Coldfusion commits every index update, and causing excessive Solr searcher warming. Combining with misconfigured default Solr configuration, this problem starts to appear for a collection of only a few hundred entries.

In Coldfusion 10, this problems seems like easier to solve by using the newly introduced "autoCommit" attribute, which should always be "no" (i.e. never use the default value). Then I guess configuring the "autoCommit" in solr config file will solve the above problem in Coldfusion 10.

However, production environment that is already in CF 9 is stuck. My solution for it is a combination attack:
    • Manually throttle cfindex update by adding a sleep after every update
    • Wrap try catch block around the cfindex update, then add a longer sleep time in the catch block, and add retry logic to try update again after the sleep
    • Database flag of index entries, this flag will be set only after index is updated successfully. A scheduled CF task will scan this flag to identify missed updates (due to the above error), and try to update index again
The above approach so far allows me to update thousands of indexes without any problem.