ITK Release 4/Performance Experiments/Reducing CTest Output

Testing is critical to high quality software and the ITK developers are expected to produce unit tests for each class. However, some tests produce large amounts of output which may be useful to the developer, but places a burden on the cdash database. Furthermore, some hypothesize that test output size may affect the performance of cdash.

This experiment looks at the size of output produced by ITKv4 and looks for ways to reduce a test's output.

Objective

Reduce the total test output of ITKv4 without affecting code coverage or value of the tests.

Approach

This experiment uses the DMAIC methodology of the Six Sigma management process to "Define", "Measure", "Analyze", "Improve" and "Control" test output in ITKv4. The basic methodology (from Wikipedia) consists of the following five steps:

• Define process goals that are consistent with customer demands and ITKv4's strategy.
• Measure key aspects of the current process and collect relevant data.
• Analyze the data to verify cause-and-effect relationships. Determine what the relationships are, and attempt to ensure that all factors have been considered.
• Improve or optimize the process.
• Control to ensure that any deviations from target are corrected before they result in defects. Set up pilot runs to establish software quality, move on to production, set up control mechanisms and continuously monitor the process.