Clever way of identifying the problem! 100000 is possibly catching only the low hanging fruit. There may be even more culprits.<br><br>I concur with the other community members that these fixes are not show-stoppers for 4.2, and we can hammer them all out for an nice 4.3.<br>
<br>A 4.1.1 release would be nice as would a 3.20.2 release, but it is hard to find the resources for them... I believe the Debian gentleman and ladies are packaging 4.1 for wheezy, and they might appreciate an easy-to-apply patch, though.<br>
<br>Thanks,<br>Matt<br><br>On Tue, Jul 3, 2012 at 2:51 PM, Bradley Lowekamp <<a href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a>> wrote:<br>> Ok, <br>><br>> If you haven't notice I tend to iterate on things until I think they are<br>
> right. So yea, release early and often, is a philosophy I should keep in<br>> mind.<br>><br>> I believe that these performance issue were introduced with changes to<br>> ITKv4. I hope that ITKv4 doesn't give the impression of being slower than<br>
> v3.<br>><br>> Should we at least prepare this as a patch for a 4.1.1 release?<br>><br>> Brad<br>><br>> On Jul 3, 2012, at 10:39 AM, Bill Lorensen wrote:<br>><br>> I agree with Hans that we should not delay the release.<br>
><br>> I also agree with Brad that we need to address this particular performance<br>> issue. We should make this a high priority for the next release.<br>><br>> Bill<br>><br>> On Tue, Jul 3, 2012 at 10:35 AM, Johnson, Hans J <<a href="mailto:hans-johnson@uiowa.edu">hans-johnson@uiowa.edu</a>><br>
> wrote:<br>>><br>>> Brad,<br>>><br>>> My opinion is that this has been an issue for many years, and that the<br>>> cost benefit is not worth trying to squeeze this into this release cycle. <br>
>> The results are (or at least should be) the same both before and after this<br>>> change.<br>>><br>>> The backlog of new features and other performance improvements are<br>>> beginning to back up on the gerrit dashboard, so I would definitely vote for<br>
>> cutting ITKv4.2 as soon as possible so that all other projects can start<br>>> moving forward again.<br>>><br>>> My guess is that the main culprits of those failing tests are related to<br>>> following 15 lines of code. <br>
>><br>>> ==============================<br>>> johnsonhj@neuron$ git grep "\->GetInput()\->GetPixel"<br>>><br>>> Modules/Filtering/DistanceMap/include/itkSignedMaurerDistanceMapImageFilter.hxx: <br>
>> if ( this->GetInput()->GetPixel(idx) != this->m_BackgroundValue )<br>>> Modules/Filtering/ImageGrid/include/itkCyclicShiftImageFilter.hxx: <br>>> outIt.Set( static_cast< OutputImagePixelType >( this->GetInput()->GetPixel(<br>
>> index ) ) );<br>>><br>>> Modules/Filtering/MathematicalMorphology/include/itkGrayscaleConnectedClosingImageFilter.hxx: <br>>> seedValue = this->GetInput()->GetPixel(m_Seed);<br>>><br>
>> Modules/Filtering/MathematicalMorphology/include/itkGrayscaleConnectedOpeningImageFilter.hxx: <br>>> seedValue = this->GetInput()->GetPixel(m_Seed);<br>>><br>>> Modules/Numerics/Statistics/include/itkScalarImageToRunLengthMatrixFilter.hxx: <br>
>> pixelIntensity = this->GetInput()->GetPixel( index );<br>>> Modules/Numerics/Statistics/test/itkSubsampleTest.cxx: <br>>> ArrayPixelImageType::PixelType pixel = filter->GetInput()->GetPixel(index);<br>
>><br>>> Modules/Segmentation/Voronoi/include/itkVoronoiPartitioningImageFilter.hxx: <br>>> getp = (double)( this->GetInput()->GetPixel(Plist[i]) );<br>>><br>>> Modules/Segmentation/Voronoi/include/itkVoronoiSegmentationImageFilter.hxx: <br>
>> getp = (double)( this->GetInput()->GetPixel(Plist[i]) );<br>>> ~/Dashboard/src/ITK (master)<br>>><br>>> johnsonhj@neuron$ <br>>> ~/Dashboard/src/ITK (master)<br>>> johnsonhj@neuron$ git grep "\->GetInput()\->Transform"<br>
>> Modules/Core/Mesh/include/itkBinaryMask3DMeshSource.hxx: <br>>> this->GetInput()->TransformContinuousIndexToPhysicalPoint(indTemp,new_p);<br>>><br>>> Modules/Nonunit/Review/include/itkScalarChanAndVeseDenseLevelSetImageFilter.hxx: <br>
>> this->GetInput()->TransformPhysicalPointToIndex(origin, start);<br>>><br>>> Modules/Nonunit/Review/include/itkScalarChanAndVeseSparseLevelSetImageFilter.hxx: <br>>> this->GetInput()->TransformPhysicalPointToIndex(origin, start);<br>
>><br>>> Modules/Nonunit/Review/include/itkStochasticFractalDimensionImageFilter.hxx: <br>>> this->GetInput()->TransformIndexToPhysicalPoint(It.GetIndex(i), point1);<br>>><br>>> Modules/Nonunit/Review/include/itkStochasticFractalDimensionImageFilter.hxx: <br>
>> this->GetInput()->TransformIndexToPhysicalPoint(It.GetIndex(j), point2);<br>>><br>>> Modules/Numerics/Statistics/include/itkScalarImageToRunLengthMatrixFilter.hxx: <br>>> this->GetInput()->TransformIndexToPhysicalPoint(<br>
>><br>>> Modules/Numerics/Statistics/include/itkScalarImageToRunLengthMatrixFilter.hxx: <br>>> this->GetInput()->TransformIndexToPhysicalPoint( lastGoodIndex, point );<br>>> ==============================<br>
>><br>>> Hans<br>>> -- <br>>> Hans J. Johnson, Ph.D.<br>>> <a href="mailto:hans-johnson@uiowa.edu">hans-johnson@uiowa.edu</a><br>>> Assistant Professor of Psychiatry<br>>> University of Iowa Carver College of Medicine<br>
>> W278 GH, 200 Hawkins Drive<br>>> Iowa City, Iowa 52242<br>>> Phone: 319-353-8587<br>>><br>>> From: Bradley Lowekamp <<a href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a>><br>
>> Date: Tuesday, July 3, 2012 9:26 AM<br>>> To: ITK <<a href="mailto:insight-developers@itk.org">insight-developers@itk.org</a>><br>>> Cc: "<a href="mailto:Bill@public.kitware.com">Bill@public.kitware.com</a>" <<a href="mailto:Bill@public.kitware.com">Bill@public.kitware.com</a>>, Hans Johnson<br>
>> <<a href="mailto:hans.j.johnson@gmail.com">hans.j.johnson@gmail.com</a>>, Luis Ibanez <<a href="mailto:luis.ibanez@kitware.com">luis.ibanez@kitware.com</a>><br>>> Subject: [Insight-developers] Performance Impact of using GetInput<br>
>><br>>> Hello,<br>>><br>>> A user yesterday, was reporting that going from ITK 3.20 to ITK 4.1, the<br>>> SignedMaurerDistanceMapImageFilter was running more that 2x-3x the time.<br>>> With a little bit of poking around and sampling the run time, I was able to<br>
>> develop the following patch:<br>>><br>>> <a href="http://review.source.kitware.com/#/c/6367/">http://review.source.kitware.com/#/c/6367/</a><br>>><br>>> I find that difference to be quite significant difference, and is on the<br>
>> level of a bug.<br>>><br>>> The lead me to wonder how wide spread is this incorrect usage. So I added<br>>> an atomic counter to the GetInput, and GetOutput methods, and when they<br>>> exceed a threshold, an exception is throw. This is to detect when these<br>
>> methods may be used in an inner loop.<br>>><br>>> <a href="http://review.source.kitware.com/#/c/6369/">http://review.source.kitware.com/#/c/6369/</a><br>>><br>>><br>>> I get the following test failure (where previously there was none):<br>
>><br>>><br>>> 97% tests passed, 71 tests failed out of 2382<br>>><br>>> The following tests FAILED:<br>>> 160 - itkN4BiasFieldCorrectionImageFilterTest1 (Failed)<br>>> 161 - itkN4BiasFieldCorrectionImageFilterTest2 (Failed)<br>
>> 311 - itkMultiThreaderEnvTest88 (Failed)<br>>> 313 - itkMultiThreaderEnvTest123 (Failed)<br>>> 398 - itkFFTConvolutionImageFilterTest4x4Mean (Failed)<br>>> 399 - itkFFTConvolutionImageFilterTest4x5Mean (Failed)<br>
>> 400 - itkFFTConvolutionImageFilterTest5x5Mean (Failed)<br>>> 401 - itkFFTConvolutionImageFilterTest4x4MeanValidRegion (Failed)<br>>> 402 - itkFFTConvolutionImageFilterTest4x5MeanValidRegion (Failed)<br>
>> 403 - itkFFTConvolutionImageFilterTest5x5MeanValidRegion (Failed)<br>>> 420 - itkRichardsonLucyDeconvolutionImageFilterGaussianKernelTest (Failed)<br>>> 421 - itkRichardsonLucyDeconvolutionImageFilterIrregularKernelTest<br>
>> (Failed)<br>>> 422 - itkLandweberDeconvolutionImageFilterGaussianKernelTest (Failed)<br>>> 423 - itkLandweberDeconvolutionImageFilterIrregularKernelTest (Failed)<br>>> 425 - itkProjectedLandweberDeconvolutionImageFilterGaussianKernelTest<br>
>> (Failed)<br>>> 426 - itkProjectedLandweberDeconvolutionImageFilterIrregularKernelTest<br>>> (Failed)<br>>> 427 - itkInverseDeconvolutionImageFilterGaussianKernelTest (Failed)<br>>> 428 - itkInverseDeconvolutionImageFilterIrregularKernelTest (Failed)<br>
>> 429 - itkTikhonovDeconvolutionImageFilterGaussianKernelTest (Failed)<br>>> 430 - itkTikhonovDeconvolutionImageFilterIrregularKernelTest (Failed)<br>>> 431 - itkWienerDeconvolutionImageFilterGaussianKernelTest (Failed)<br>
>> 432 - itkWienerDeconvolutionImageFilterIrregularKernelTest (Failed)<br>>> 433 - itkParametricBlindLeastSquaresDeconvolutionImageFilterTest (Failed)<br>>> 436 - itkDeformableSimplexMesh3DBalloonForceFilterTest (Failed)<br>
>> 440 - itkPatchBasedDenoisingImageFilterTest0 (Failed)<br>>> 441 - itkPatchBasedDenoisingImageFilterTestGaussian (Failed)<br>>> 442 - itkPatchBasedDenoisingImageFilterTestRician (Failed)<br>>> 443 - itkPatchBasedDenoisingImageFilterTestPoisson (Failed)<br>
>> 521 - itkDisplacementFieldToBSplineImageFilterTest (Failed)<br>>> 524 - itkContourMeanDistanceImageFilterTest (Failed)<br>>> 525 - itkContourDirectedMeanDistanceImageFilterTest (Failed)<br>>> 530 - itkHausdorffDistanceImageFilterTest (Failed)<br>
>> 532 - itkSignedMaurerDistanceMapImageFilterTest1 (Failed)<br>>> 533 - itkSignedMaurerDistanceMapImageFilterTest2 (Failed)<br>>> 656 - itkFastMarchingImageFilterTest_torus_multipleSeeds_NoTopo (Failed)<br>
>> 657 - itkFastMarchingImageFilterTest_torus_multipleSeeds_StrictTopo<br>>> (Failed)<br>>> 658 - itkFastMarchingImageFilterTest_torus_multipleSeeds_NoHandlesTopo<br>>> (Failed)<br>>> 659 - itkFastMarchingImageFilterTest_wm_multipleSeeds_NoTopo (Failed)<br>
>> 660 - itkFastMarchingImageFilterTest_wm_multipleSeeds_StrictTopo (Failed)<br>>> 661 - itkFastMarchingImageFilterTest_wm_multipleSeeds_NoHandlesTopo<br>>> (Failed)<br>>> 1072 - itkBSplineControlPointImageFilterTest2 (Failed)<br>
>> 1079 - itkCyclicShiftImageFilterTest0 (Failed)<br>>> 1080 - itkCyclicShiftImageFilterTest1 (Failed)<br>>> 1081 - itkCyclicShiftImageFilterTest2 (Failed)<br>>> 1082 - itkCyclicShiftImageFilterTest3 (Failed)<br>
>> 1083 - itkCyclicShiftImageFilterTest4 (Failed)<br>>> 1084 - itkCyclicShiftImageFilterTest5 (Failed)<br>>> 1085 - itkCyclicShiftImageFilterTest6 (Failed)<br>>> 1195 - itkModulusImageFilterTest (Failed)<br>
>> 1377 - itkExtensionVelocitiesImageFilterTest (Failed)<br>>> 1378 - itkCannySegmentationLevelSetImageFilterTest (Failed)<br>>> 1412 - itkTwoLevelSetsv4DenseImage2DTest (Failed)<br>>> 1471 - itkSimplexMeshVolumeCalculatorTest (Failed)<br>
>> 1659 - itkBinaryMask3DQuadEdgeMeshSourceTest (Failed)<br>>> 1747 - itkPointSetToPointSetRegistrationTest (Failed)<br>>> 1774 - itkDiffeomorphicDemonsRegistrationFilterTest01 (Failed)<br>>> 1775 - itkDiffeomorphicDemonsRegistrationFilterTest02 (Failed)<br>
>> 1776 - itkDiffeomorphicDemonsRegistrationFilterTest03 (Failed)<br>>> 1777 - itkDiffeomorphicDemonsRegistrationFilterTest04 (Failed)<br>>> 1778 - itkDiffeomorphicDemonsRegistrationFilterTest05 (Failed)<br>
>> 1779 - itkDiffeomorphicDemonsRegistrationFilterTest06 (Failed)<br>>> 1780 - itkDiffeomorphicDemonsRegistrationFilterTest07 (Failed)<br>>> 1781 - itkDiffeomorphicDemonsRegistrationFilterTest08 (Failed)<br>
>> 1782 - itkDiffeomorphicDemonsRegistrationFilterTest09 (Failed)<br>>> 1783 - itkDiffeomorphicDemonsRegistrationFilterTest10 (Failed)<br>>> 1784 - itkDiffeomorphicDemonsRegistrationFilterTest11 (Failed)<br>
>> 1802 - itkFastSymmetricForcesDemonsRegistrationFilterTest (Failed)<br>>> 2166 - itkVoronoiSegmentationImageFilterTest (Failed)<br>>><br>>><br>>> How big of a deal if most of the filters here are running 2x+ slower then<br>
>> what they should be? Is it big enough to delay the Release and do another RC<br>>> with the fixes?<br>>><br>>> I have also been looking at the methods used in GetInput, specifically the<br>>> methods used to create the std::string... It seems to be if we change the<br>
>> return value to a const std::string &, then we could keep a static internal<br>>> table of the common value and return reference to the static table to even,<br>>> references to what is in the std::map, the would reduce the need for mallocs<br>
>> for std::string.<br>>><br>>> Thoughts on what to do?<br>>><br>>> Brad<br>>><br>>> ========================================================<br>>> Bradley Lowekamp <br>>> Medical Science and Computing for<br>
>> Office of High Performance Computing and Communications<br>>> National Library of Medicine <br>>> <a href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a><br>>><br>>><br>>><br>
>><br>>><br>>> ________________________________<br>>> Notice: This UI Health Care e-mail (including attachments) is covered by<br>>> the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is<br>
>> confidential and may be legally privileged. If you are not the intended<br>>> recipient, you are hereby notified that any retention, dissemination,<br>>> distribution, or copying of this communication is strictly prohibited. <br>
>> Please reply to the sender that you have received the message in error, then<br>>> delete it. Thank you.<br>>> ________________________________<br>>><br>>> _______________________________________________<br>
>> Powered by <a href="http://www.kitware.com">www.kitware.com</a><br>>><br>>> Visit other Kitware open-source projects at<br>>> <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br>
>><br>>> Kitware offers ITK Training Courses, for more information visit:<br>>> <a href="http://kitware.com/products/protraining.php">http://kitware.com/products/protraining.php</a><br>>><br>>> Please keep messages on-topic and check the ITK FAQ at:<br>
>> <a href="http://www.itk.org/Wiki/ITK_FAQ">http://www.itk.org/Wiki/ITK_FAQ</a><br>>><br>>> Follow this link to subscribe/unsubscribe:<br>>> <a href="http://www.itk.org/mailman/listinfo/insight-developers">http://www.itk.org/mailman/listinfo/insight-developers</a><br>
>><br>><br>><br>><br>> --<br>> Unpaid intern in BillsBasement at noware dot com<br>><br>><br>> ========================================================<br>><br>> Bradley Lowekamp <br>><br>
> Medical Science and Computing for<br>><br>> Office of High Performance Computing and Communications<br>><br>> National Library of Medicine <br>><br>> <a href="mailto:blowekamp@mail.nih.gov">blowekamp@mail.nih.gov</a><br>
><br>><br>><br>><br>><br>> _______________________________________________<br>> Powered by <a href="http://www.kitware.com">www.kitware.com</a><br>><br>> Visit other Kitware open-source projects at<br>
> <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br>><br>> Kitware offers ITK Training Courses, for more information visit:<br>> <a href="http://kitware.com/products/protraining.php">http://kitware.com/products/protraining.php</a><br>
><br>> Please keep messages on-topic and check the ITK FAQ at:<br>> <a href="http://www.itk.org/Wiki/ITK_FAQ">http://www.itk.org/Wiki/ITK_FAQ</a><br>><br>> Follow this link to subscribe/unsubscribe:<br>> <a href="http://www.itk.org/mailman/listinfo/insight-developers">http://www.itk.org/mailman/listinfo/insight-developers</a><br>
><br><br>