VTK/SoftwareQuality/NightlyExpectedBuildTestFailures: Difference between revisions

From KitwarePublic
< VTK
Jump to navigationJump to search
(Created page with "The VTK [http://open.cdash.org/index.php?project=VTK&filtercount=1&showfilters=1&field1=groupname/string&compare1=61&value1=Nightly%20Expected Nightly Expected] builds show the s...")
 
 
(28 intermediate revisions by 3 users not shown)
Line 14: Line 14:


==Measure==
==Measure==
As of February xx there were xx defects on the xx VTK Nightly Expected builds:
Recent summaries:
 
[[VTK/SoftwareQuality/NightlyBuildSummary-2013-02-19|2013-02-19]]
 
[[VTK/SoftwareQuality/NightlyBuildSummary-2013-02-20|2013-02-20]]
 
[[VTK/SoftwareQuality/NightlyBuildSummary-2013-02-21|2013-02-21]]
 
[[VTK/SoftwareQuality/NightlyBuildSummary-2013-02-23|2013-02-23]]
 
[[VTK/SoftwareQuality/NightlyBuildSummary-2013-02-26|2013-02-26]]
 
[[VTK/SoftwareQuality/NightlyBuildSummary-2013-02-27|2013-02-27]]
 
[[VTK/SoftwareQuality/NightlyBuildSummary-2013-03-02|2013-03-02]]


==Analyze==
==Analyze==
{| class="wikitable sortable" border="1" cellpadding="5" cellspacing="0"
|-
| Test
| Failing
| Defect
| Assigned To
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersParallelStatisticsCxx-MPI-TestRandomPMomentStatisticsMPI vtkFiltersParallelStatisticsCxx-MPI-TestRandomPMomentStatisticsMPI]
| 1
| Memory Leak
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumePython-volTM3DCropRegions vtkRenderingVolumePython-volTM3DCropRegions]
| 1
| Failing since [http://open.cdash.org/index.php?project=VTK&date=20130204 Feb 4]
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersGeometryCxx-TestImageDataToUniformGrid vtkFiltersGeometryCxx-TestImageDataToUniformGrid]
| 1
| AIX only
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionWidgetsCxx-TestContourWidget2 vtkInteractionWidgetsCxx-TestContourWidget2]
| 1
| [http://open.cdash.org/testDetails.php?test=178390042&build=2823952 Regression defect, small]
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestGlyph3DMapperOrientationArray vtkRenderingOpenGLCxx-TestGlyph3DMapperOrientationArray]
| 1
| [http://open.cdash.org/testDetails.php?test=178422932&build=2824162 Regression defect] AIX only
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkParallelMPIPython-TestPProbe vtkParallelMPIPython-TestPProbe]
| 1
| Python syntax
| Unassigned [http://review.source.kitware.com/#/t/2111/ Existing gerrit topic]
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumePython-TestFixedPointRayCasterNearestCropped vtkRenderingVolumePython-TestFixedPointRayCasterNearestCropped]
| 1
| [http://open.cdash.org/testDetails.php?test=178384129&build=2823951 Regression defect]
AIX exit
| Bill
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingParallelCxx-MPI-TestClientServerRendering vtkRenderingParallelCxx-MPI-TestClientServerRendering]
| 1
| [http://open.cdash.org/testDetails.php?test=178354074&build=2823558 Failed to connect]
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=JavaDeleteRegression JavaDeleteRegression]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInfovisLayoutCxx-TestCirclePackLayoutStrategy vtkInfovisLayoutCxx-TestCirclePackLayoutStrategy]
| 1
| [http://open.cdash.org/testDetails.php?test=178377163&build=2823849 Regression defect, major]
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=JavaDeleteAndGC JavaDeleteAndGC]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionStylePython-TestStyleTrackballCamera vtkInteractionStylePython-TestStyleTrackballCamera]
| 1
| [http://open.cdash.org/testDetails.php?test=178422784&build=2824162 Regression defect] AIX and OSMesa only
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestTextureRGBADepthPeeling vtkRenderingOpenGLCxx-TestTextureRGBADepthPeeling]
| 1
| [http://open.cdash.org/testDetails.php?test=178390201&build=2823952 Regression defect] Intermittent
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionWidgetsCxx-TestPickingManagerWidgets vtkInteractionWidgetsCxx-TestPickingManagerWidgets]
| 1
| [http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionWidgetsCxx-TestPickingManagerWidgets Regression defect] AIX and OSMesa only
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumePython-TestFixedPointRayCasterLinearCropped vtkRenderingVolumePython-TestFixedPointRayCasterLinearCropped]
| 1
| Threading issues
| Bill
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestTextActor3DAlphaBlending vtkRenderingOpenGLCxx-TestTextActor3DAlphaBlending]
| 1
| DASH23 X server issue.
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumeCxx-TestHAVSVolumeMapper vtkRenderingVolumeCxx-TestHAVSVolumeMapper]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestGlyph3DMapperMasking vtkRenderingOpenGLCxx-TestGlyph3DMapperMasking]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionWidgetsCxx-TestImplicitPlaneWidget2b vtkInteractionWidgetsCxx-TestImplicitPlaneWidget2b]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkIONetCDFCxx-SLACMultipleModes vtkIONetCDFCxx-SLACMultipleModes]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersProgrammablePython-progGlyphsBySource vtkFiltersProgrammablePython-progGlyphsBySource]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestTexturedBackground vtkRenderingOpenGLCxx-TestTexturedBackground]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=JavaManualGC JavaManualGC]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionStylePython-TestStyleJoystickCamera vtkInteractionStylePython-TestStyleJoystickCamera]
| 1
| [http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionStylePython-TestStyleJoystickCamera Regression defect] AIX and OSMesa only
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkChartsCorePython-TestLinePlotColors vtkChartsCorePython-TestLinePlotColors]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumeTcl-volTM3DCompressedCropRegions vtkRenderingVolumeTcl-volTM3DCompressedCropRegions]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumePython-volTM3DCompressedCropRegions vtkRenderingVolumePython-volTM3DCompressedCropRegions]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkIOGeometryCxx-TestWindBladeReader vtkIOGeometryCxx-TestWindBladeReader]
| 1
| Bigendian defect
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestOpacity vtkRenderingOpenGLCxx-TestOpacity]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersGeneralCxx-TestGradientAndVorticity vtkFiltersGeneralCxx-TestGradientAndVorticity]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumePython-TestFixedPointRayCasterNearest vtkRenderingVolumePython-TestFixedPointRayCasterNearest]
| 1
| Threading
| Bill
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkImagingCorePython-TestCheckerboard vtkImagingCorePython-TestCheckerboard]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkIOExportCxx-TestGL2PSExporterRaster-VerifyRasterizedPNG vtkIOExportCxx-TestGL2PSExporterRaster-VerifyRasterizedPNG]
| 1
| CubeAxesActor error, used wrong buffer in GL2PS rasterization
| David L. -- Baselines updated, [http://review.source.kitware.com/#/t/2242/ Gerrit] patch pushed
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersGeometryCxx-TestStructuredGridConnectivity vtkFiltersGeometryCxx-TestStructuredGridConnectivity]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionWidgetsPython-TestInteractorEventRecorder vtkInteractionWidgetsPython-TestInteractorEventRecorder]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumePython-TestFixedPointRayCasterLinear vtkRenderingVolumePython-TestFixedPointRayCasterLinear]
| 1
| Threading
|
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionStylePython-TestStyleTrackballActor vtkInteractionStylePython-TestStyleTrackballActor]
| 1
| [http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionStylePython-TestStyleTrackballActor Regression defect] AIX and OSMesa only
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionWidgetsCxx-TestTextWidget vtkInteractionWidgetsCxx-TestTextWidget]
| 1
| DASH23 X server issue
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=JavaGCRegression JavaGCRegression]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingParallelCxx-MPI-TestParallelRendering vtkRenderingParallelCxx-MPI-TestParallelRendering]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersGeneralCxx-TestIntersectionPolyDataFilter2 vtkFiltersGeneralCxx-TestIntersectionPolyDataFilter2]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionStylePython-TestStyleJoystickActor vtkInteractionStylePython-TestStyleJoystickActor]
| 1
| [http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkInteractionStylePython-TestStyleJoystickActor Regression defect] AIX and OSMesa only
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=JavaRegression JavaRegression]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkChartsCoreCxx-TestScatterPlotMatrix vtkChartsCoreCxx-TestScatterPlotMatrix]
| 1
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DGeometryMaterial vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DGeometryMaterial]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingCorePython-TestOpacity2 vtkRenderingCorePython-TestOpacity2]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumeCxx-TestProjectedHexahedra vtkRenderingVolumeCxx-TestProjectedHexahedra]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersSelectionCxx-TestLinearSelector3D vtkFiltersSelectionCxx-TestLinearSelector3D]
| 2
| incorrect cardinality
| Dave Demarle
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHybridTcl-3DMorph vtkFiltersHybridTcl-3DMorph]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DAxisCut vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DAxisCut]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DUnstructured vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DUnstructured]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DClip vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DClip]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumePython-volTM2DRotateClip vtkRenderingVolumePython-volTM2DRotateClip]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernarySphereMaterial vtkFiltersHyperTreeCxx-TestHyperTreeGridTernarySphereMaterial]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestGaussianBlurPass vtkRenderingOpenGLCxx-TestGaussianBlurPass]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DGeometry vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DGeometry]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DContourMaterial vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DContourMaterial]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DUnstructuredMaterial vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DUnstructuredMaterial]
| 2
| Unknown
|
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersGeneralCxx-TestDensifyPolyData vtkFiltersGeneralCxx-TestDensifyPolyData]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingCoreTcl-TestOpacity2 vtkRenderingCoreTcl-TestOpacity2]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkIOMPIParallelCxx-TestPWindBladeReader vtkIOMPIParallelCxx-TestPWindBladeReader]
| 2
| Bigendian defect
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkGUISupportQtCxx-TestQtDebugLeaksView vtkGUISupportQtCxx-TestQtDebugLeaksView]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DCut vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DCut]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestTranslucentLUTTextureDepthPeeling vtkRenderingOpenGLCxx-TestTranslucentLUTTextureDepthPeeling]
| 2
| Disabled for ATI's
| Sean
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DAxisCutMaterial vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DAxisCutMaterial]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHybridPython-3DMorph vtkFiltersHybridPython-3DMorph]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingCoreTcl-CamBlur vtkRenderingCoreTcl-CamBlur]
| 2
| Started failing after tcl changes
| Bill
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingVolumeTcl-TestFixedPointRayCasterLinearCropped vtkRenderingVolumeTcl-TestFixedPointRayCasterLinearCropped]
| 2
| Threading
| Bill
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeelingPass vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeelingPass]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersFlowPathsCxx-TestBSPTree vtkFiltersFlowPathsCxx-TestBSPTree]
| 2
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeeling vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeeling]
| 3
| Unknown
| Unassigned
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkChartsCoreCxx-TestLinePlot3D vtkChartsCoreCxx-TestLinePlot3D]
| 3
| [http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkChartsCoreCxx-TestLinePlot3D Regression defects]
| Marcus [http://review.source.kitware.com/#/t/2230/ Gerrit topic]
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG]
| 3
| [http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG Regression defects] that differ in various machines
volume rendering
| Dave L. [http://review.source.kitware.com/#/t/2242/ Gerrit]
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DContour vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DContour]
| 4
| Unknown
| Berk
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-27&name=vtkCommonCoreTcl-TestSetGet vtkCommonCoreTcl-TestSetGet]
| 5
| Timeout
| Unassigned
|}


==Improve==
==Improve==
The following Gerrit reviews addressed the defects.
The following Gerrit reviews addressed the defects.
 
{| class="wikitable sortable" border="1" cellpadding="5" cellspacing="0"
|-
| Test
| Gerrit patch
| Summary
|-
|
[http://open.cdash.org/testSummary.php?project=11&date=2013-02-23&name=vtkCommonCoreTcl-TestSetGet vtkCommonCoreTcl-TestSetGet]
| [http://review.source.kitware.com/#/t/2197/ ENH: Reduce the output of TestSetGet.tcl]
| TestSetGet.tcl produces over 6 meg of output. This patch reduces that to 50k.
|}
==Control==
==Control==


Once reduced to 0, developer diligence is needed to keep the defects to 0.
Once reduced to 0, developer diligence is needed to keep the defects to 0.

Latest revision as of 16:37, 2 March 2013

The VTK Nightly Expected builds show the status of VTK on VTK supported platforms at a fixed time each day. Although the Gerrit Review Process and Continuous Builds attempt to catch build defects early, it is the Nightly Expected builds that show the health of VTK on a broader collection of systems. Recurring nightly failures make it difficult for a developer to see if her changes have affected the spectrum of platforms.

This experiment uses the DMAIC methodology of the Six Sigma management process to "Define", "Measure", "Analyze", "Improve" and "Control" to resolve these issues.

The basic methodology (from Wikipedia) consists of the following five steps:

  • Define process goals that are consistent with customer demands and VTK's strategy.
  • Measure key aspects of the current process and collect relevant data.
  • Analyze the data to verify cause-and-effect relationships. Determine what the relationships are, and attempt to ensure that all factors have been considered.
  • Improve or optimize the process.
  • Control to ensure that any deviations from target are corrected before they result in defects. Set up pilot runs to establish software quality, move on to production, set up control mechanisms and continuously monitor the process.

Define

Keep the number of VTK Nightly Expected recurring test failures(defects) to 0. When the defects are above 0, developers find it difficult to notice that their changes introduce new defects.

Measure

Recent summaries:

2013-02-19

2013-02-20

2013-02-21

2013-02-23

2013-02-26

2013-02-27

2013-03-02

Analyze

Test Failing Defect Assigned To

vtkFiltersParallelStatisticsCxx-MPI-TestRandomPMomentStatisticsMPI

1 Memory Leak Unassigned

vtkRenderingVolumePython-volTM3DCropRegions

1 Failing since Feb 4 Unassigned

vtkFiltersGeometryCxx-TestImageDataToUniformGrid

1 AIX only Unassigned

vtkInteractionWidgetsCxx-TestContourWidget2

1 Regression defect, small Unassigned

vtkRenderingOpenGLCxx-TestGlyph3DMapperOrientationArray

1 Regression defect AIX only Unassigned

vtkParallelMPIPython-TestPProbe

1 Python syntax Unassigned Existing gerrit topic

vtkRenderingVolumePython-TestFixedPointRayCasterNearestCropped

1 Regression defect

AIX exit

Bill

vtkRenderingParallelCxx-MPI-TestClientServerRendering

1 Failed to connect Unassigned

JavaDeleteRegression

1 Unknown Unassigned

vtkInfovisLayoutCxx-TestCirclePackLayoutStrategy

1 Regression defect, major Unassigned

JavaDeleteAndGC

1 Unknown Unassigned

vtkInteractionStylePython-TestStyleTrackballCamera

1 Regression defect AIX and OSMesa only Unassigned

vtkRenderingOpenGLCxx-TestTextureRGBADepthPeeling

1 Regression defect Intermittent Unassigned

vtkInteractionWidgetsCxx-TestPickingManagerWidgets

1 Regression defect AIX and OSMesa only Unassigned

vtkRenderingVolumePython-TestFixedPointRayCasterLinearCropped

1 Threading issues Bill

vtkRenderingOpenGLCxx-TestTextActor3DAlphaBlending

1 DASH23 X server issue. Unassigned

vtkRenderingVolumeCxx-TestHAVSVolumeMapper

1 Unknown Unassigned

vtkRenderingOpenGLCxx-TestGlyph3DMapperMasking

1 Unknown Unassigned

vtkInteractionWidgetsCxx-TestImplicitPlaneWidget2b

1 Unknown Unassigned

vtkIONetCDFCxx-SLACMultipleModes

1 Unknown Unassigned

vtkFiltersProgrammablePython-progGlyphsBySource

1 Unknown Unassigned

vtkRenderingOpenGLCxx-TestTexturedBackground

1 Unknown Unassigned

JavaManualGC

1 Unknown Unassigned

vtkInteractionStylePython-TestStyleJoystickCamera

1 Regression defect AIX and OSMesa only Unassigned

vtkChartsCorePython-TestLinePlotColors

1 Unknown Unassigned

vtkRenderingVolumeTcl-volTM3DCompressedCropRegions

1 Unknown Unassigned

vtkRenderingVolumePython-volTM3DCompressedCropRegions

1 Unknown Unassigned

vtkIOGeometryCxx-TestWindBladeReader

1 Bigendian defect Unassigned

vtkRenderingOpenGLCxx-TestOpacity

1 Unknown Unassigned

vtkFiltersGeneralCxx-TestGradientAndVorticity

1 Unknown Unassigned

vtkRenderingVolumePython-TestFixedPointRayCasterNearest

1 Threading Bill

vtkImagingCorePython-TestCheckerboard

1 Unknown Unassigned

vtkIOExportCxx-TestGL2PSExporterRaster-VerifyRasterizedPNG

1 CubeAxesActor error, used wrong buffer in GL2PS rasterization David L. -- Baselines updated, Gerrit patch pushed

vtkFiltersGeometryCxx-TestStructuredGridConnectivity

1 Unknown Unassigned

vtkInteractionWidgetsPython-TestInteractorEventRecorder

1 Unknown Unassigned

vtkRenderingVolumePython-TestFixedPointRayCasterLinear

1 Threading

vtkInteractionStylePython-TestStyleTrackballActor

1 Regression defect AIX and OSMesa only Unassigned

vtkInteractionWidgetsCxx-TestTextWidget

1 DASH23 X server issue Unassigned

JavaGCRegression

1 Unknown Unassigned

vtkRenderingParallelCxx-MPI-TestParallelRendering

1 Unknown Unassigned

vtkFiltersGeneralCxx-TestIntersectionPolyDataFilter2

1 Unknown Unassigned

vtkInteractionStylePython-TestStyleJoystickActor

1 Regression defect AIX and OSMesa only Unassigned

JavaRegression

1 Unknown Unassigned

vtkChartsCoreCxx-TestScatterPlotMatrix

1 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DGeometryMaterial

2 Unknown Unassigned

vtkRenderingCorePython-TestOpacity2

2 Unknown Unassigned

vtkRenderingVolumeCxx-TestProjectedHexahedra

2 Unknown Unassigned

vtkFiltersSelectionCxx-TestLinearSelector3D

2 incorrect cardinality Dave Demarle

vtkFiltersHybridTcl-3DMorph

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DAxisCut

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DUnstructured

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DClip

2 Unknown Unassigned

vtkRenderingVolumePython-volTM2DRotateClip

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernarySphereMaterial

2 Unknown Unassigned

vtkRenderingOpenGLCxx-TestGaussianBlurPass

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DGeometry

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DContourMaterial

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DUnstructuredMaterial

2 Unknown

vtkFiltersGeneralCxx-TestDensifyPolyData

2 Unknown Unassigned

vtkRenderingCoreTcl-TestOpacity2

2 Unknown Unassigned

vtkIOMPIParallelCxx-TestPWindBladeReader

2 Bigendian defect Unassigned

vtkGUISupportQtCxx-TestQtDebugLeaksView

2 Unknown Unassigned

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DCut

2 Unknown Unassigned

vtkRenderingOpenGLCxx-TestTranslucentLUTTextureDepthPeeling

2 Disabled for ATI's Sean

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DAxisCutMaterial

2 Unknown Unassigned

vtkFiltersHybridPython-3DMorph

2 Unknown Unassigned

vtkRenderingCoreTcl-CamBlur

2 Started failing after tcl changes Bill

vtkRenderingVolumeTcl-TestFixedPointRayCasterLinearCropped

2 Threading Bill

vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeelingPass

2 Unknown Unassigned

vtkFiltersFlowPathsCxx-TestBSPTree

2 Unknown Unassigned

vtkRenderingOpenGLCxx-TestTranslucentLUTDepthPeeling

3 Unknown Unassigned

vtkChartsCoreCxx-TestLinePlot3D

3 Regression defects Marcus Gerrit topic

vtkIOExportCxx-TestGL2PSExporterVolumeRaster-VerifyRasterizedPNG

3 Regression defects that differ in various machines

volume rendering

Dave L. Gerrit

vtkFiltersHyperTreeCxx-TestHyperTreeGridTernary3DContour

4 Unknown Berk

vtkCommonCoreTcl-TestSetGet

5 Timeout Unassigned

Improve

The following Gerrit reviews addressed the defects.

Test Gerrit patch Summary

vtkCommonCoreTcl-TestSetGet

ENH: Reduce the output of TestSetGet.tcl TestSetGet.tcl produces over 6 meg of output. This patch reduces that to 50k.

Control

Once reduced to 0, developer diligence is needed to keep the defects to 0.

Retrieved from "https://public.kitware.com/Wiki/index.php?title=VTK/SoftwareQuality/NightlyExpectedBuildTestFailures&oldid=51656"