Code coverage is an essential part of testing, it reflects the completeness of the testing done. Code coverage is instrumental in providing information about how much of the total code is actually tested by the test suite. However code coverage does not guaranty the quality of testing but it reflects the area covered in testing.
Another important point is white box testing and code coverage(functional) are two separate activities. while the former determine the probable inputs and output for a function, the later determine if all lines and branching conditions of the function are executed, in short all paths have been covered or not. There are quiet a bunch of tools available for Java applications. But for ColdFusion applications, till now there is no way to know the code coverage.
This post provides an experimental tool, Rancho, to get code coverage for any ColdFusion application. Rancho can be used as a ColdFusion Builder extension or as an application. Download Rancho as an extension to CFB.
Rancho is the code coverage tool written in CFML for code coverage of CFML application. There are different metrics a code coverage tool can provide(line coverage,functional coverage,decision coverage) but as of now Rancho only provides the line coverage.
Another important point is white box testing and code coverage(functional) are two separate activities. while the former determine the probable inputs and output for a function, the later determine if all lines and branching conditions of the function are executed, in short all paths have been covered or not. There are quiet a bunch of tools available for Java applications. But for ColdFusion applications, till now there is no way to know the code coverage.
This post provides an experimental tool, Rancho, to get code coverage for any ColdFusion application. Rancho can be used as a ColdFusion Builder extension or as an application. Download Rancho as an extension to CFB.
Rancho is the code coverage tool written in CFML for code coverage of CFML application. There are different metrics a code coverage tool can provide(line coverage,functional coverage,decision coverage) but as of now Rancho only provides the line coverage.
How to get code coverage?
There are 3 steps to get the code coverage.
- Instrument your CFML application using Rancho.
- Run your test suite.
- Generate a report of code coverage using Rancho.
Using Rancho as an extension to CFB
Download the Rancho and add it as an extension to CFB. View this article to add an extension to CFB.
Note: Since this project is hosted on Github and it provides project to download as a zip. This zip can not directly be imported to CFB as extension because extension files are one level down. So either rezip at the level of ide_config.xml OR directly download from RIAForge
You can also run Rancho as an application. Download it from here.
STEP 1 - Instrument: Go to the Application folder in CFB navigator you need to instrument. Do a right click and select Rancho -> Instrument
 |
| Figure 1: Rancho Instrumentation |
This will first take backup of the existing folder and start instrumenting all files in that folder.
 |
| Figure 2: Instrumentation Done |
Note: Instrumentation is a one time activity. Do not instrument more than once.
STEP 2 - Running test suite: Once instrumentation is done you can proceed to running your testcases for the application instrumented.
STEP 3 - Getting Code Coverage Data: Once you are done with your testing or otherwise you can extract a report of code coverage. From Figure 1 click on Rancho -> Code Coverage or you can directly access http://localhost:8500/Rancho/handlers/coveragegrid.cfm
 |
| Figure 3: Code Coverage preview |
This will show you the preview of your code coverage, indicating total coverage of application. Also it lists code coverage for individual files. You can further drill down to individual files to see executed and un-executed lines.
 |
| Figure 4: Code Coverage details |
This shows individual CF statements executed(green) and un-executed CF statements(red).
Configuring Rancho
While instrumenting Rancho adds a trace before each CF statement. Now there could be some statements that can not be traced like or .
To handle this 2 files doNottrace.txt and putTraceAfter.txt are placed at Rancho\handlers\config. Add any tag/end tag which falls in above category. As of now only some tags/end tags have been added. As you find more such tags please update them to the repository.
Limitations
- Supports only tags not script, so any CFML code inside
will be uncovered. - Multiple level of folders is not supported yet. Instrument and generate report for single folder level at a time.
- Instrumentation of single file is not supported. Only folders can be instrumented.
- Only line coverage is provided.
Improvements
Rancho is an experimental code coverage tool so there are some improvements required like:
- Support for .
- Multi folder support
- UI improvements
- Fine tuning tags in doNottrace.txt and putTraceAfter.txt
- Enhance Rancho to extend Function and Decision coverage
I have created a github project(I am new to it). Please contribute to it.
PS: You can also run Rancho as an application. Download it from here. Place Rancho in webroot of CF and run index.cfm. Also make sure the folder you are instrumenting has write permissions.
PS: You can also run Rancho as an application. Download it from here. Place Rancho in webroot of CF and run index.cfm. Also make sure the folder you are instrumenting has write permissions.
