Interesting re Otsu. Theres a bug in the stdev calc - update will arrive soon. Thanks for pointing out the results didnt make sense.<div><br></div><div>For my specific needs, its the slowdown in the metrics that trouble me the most. However, there may be improvements in result quality that would justify a slowdown. In general, i dont think the results are numerically identical (have not tested this specific case).<br>
<div><br></div><div>Rupert<br clear="all">--------------------------------------------------------------<br>Rupert Brooks<br><a href="mailto:rupert.brooks@gmail.com">rupert.brooks@gmail.com</a><br><br>
<br><br><div class="gmail_quote">On Wed, Jul 25, 2012 at 12:02 PM, Bradley Lowekamp <span dir="ltr"><<a href="mailto:blowekamp@mail.nih.gov" target="_blank">blowekamp@mail.nih.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Rupert,<div><br></div><div><br></div><div>I took out the OtsuThreshold filter and the resulting timings for ITKv3 and ITKv4 were comparable for your Filter Chain. More than half the time was spend in this one filter.</div>
<div><br></div><div>Also the computed stddev results do not appear to be correct. However looking at the code it looks right.</div><div><br></div><div>Brad</div><div><div class="h5"><div><br><div><div>On Jul 25, 2012, at 11:53 AM, Rupert Brooks wrote:</div>
<br><blockquote type="cite">Brad,<div><br></div><div>Agreed, VS2005 is not my favorite either, which is why im doing tests also with a recent gcc. If the performance degradation was limited to VS2005, that would not be a big deal in my view. I'm stuck with it at the moment, but I can hardly expect you folks to do your best work for it.</div>
<div><br></div><div>If and when i give other compilers a shot, i will keep you posted.</div><div><br></div><div>Rupert<br clear="all">--------------------------------------------------------------<br>Rupert Brooks<br><a href="mailto:rupert.brooks@gmail.com" target="_blank">rupert.brooks@gmail.com</a><br>
<br>
<br><br><div class="gmail_quote">On Wed, Jul 25, 2012 at 10:52 AM, Bradley Lowekamp <span dir="ltr"><<a href="mailto:blowekamp@mail.nih.gov" target="_blank">blowekamp@mail.nih.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Hello Rupert,<div><br></div><div>I see you are still using Visual Studio 8 2005, I am curious how VS10 would compare. I know that there are a couple special case optimization added to ITKv4 which VS8 is not able to take advantage off. </div>
<div><br></div><div>As others have reported we are begining to address some of these issues.</div><div><br></div><div>Thanks for sharing!</div><div>Brad</div><div><br><div><div><div><div>On Jul 25, 2012, at 10:12 AM, Rupert Brooks wrote:</div>
<br></div></div><blockquote type="cite"><div><div><div>Hi,</div><div><br></div><div>I was commenting the other day that ITK4 seems to have lower performance than ITK3. I put together a little benchmark to test this objectively. Results for a couple of systems are below. The benchmark is neither particularly sophisticated nor exact, but i have tried to make it typical of what i use ITK for. I have not tried it with Hans' patch as that got abandoned. Will follow up separately on that.</div>
<div><br></div><div>In the numbers below, you will see that performance seems to have degraded across the board. One exception is the 2D image test. In this test i was lazy, and i left the image axis aligned with unit spacing. I am suspecting a special case optimization in ITK4. At the moment, this blocks my ability to upgrade my projects to ITK4, as they are performance critical. Any hints and tips to get those milliseconds back will be much appreciated.</div>
<div><br></div><div>The code is at <a href="https://github.com/rupertbrooks/itkbench" target="_blank">https://github.com/rupertbrooks/itkbench</a>, i'd be curious if others get the same results, and of course, criticism and improvements are welcome. </div>
<div><br></div><div>Note that the output about what processor / cores is on the system is bogus. The windows system is a core i7 with 4 cores, the linux one is a Core2 Quad. I suspect a bug in itksys::SystemInformation, but i will follow up on that separately also. </div>
<div><br></div><div>In all itk builds, ITK_USE_REVIEW is on. In the itk3 builds, USE_OPTIMZIZED_REGISTRATION, and the new statistics framework are turned on.</div><div><br></div><div>Cheers,</div><div>Rupert</div><div><br>
</div><div>Data follows.....</div>
<div><br></div><div>itk3 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo </div><div><div>System: CAMD5C5PMN1</div><div>Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache </div><div> Serial #: </div>
<div> Cache: -1</div><div> Clock: 2800</div><div> Cores: 4 cpus x 1 Cores = 4</div><div>OSName: Windows</div><div> Release: XP Professional</div><div> Version: Service Pack 3 (Build 2600)</div><div> Platform: x86</div>
<div> Operating System is 32 bit</div><div>ITK Version: 3.20.0</div><div>Virtual Memory: Total: 2047 Available: 2021</div><div>Physical Memory: Total:3581 Available: 1457</div><div> Probe Name: Count Min Mean Stdev Max Total </div>
<div> FilterChain_1_threads 10 0.0345688 0.0361805 0.0381523 0.0375099 0.361805</div><div> FilterChain_2_threads 10 0.0257301 0.0265144 0.0279574 0.0282478 0.265144</div>
<div> FilterChain_3_threads 10 0.022007 0.0232452 0.0245125 0.0240822 0.232452</div><div> FilterChain_4_threads 10 0.0199661 0.0221756 0.0235205 0.0288773 0.221756</div>
<div> Image2D 10 0.144909 0.14665 0.154586 0.148136 1.4665</div><div> Image3D 10 0.19125 0.193378 0.203846 0.196495 1.93378</div>
<div> MeanSquares_1_threads 10 0.612728 0.616159 0.649503 0.627789 6.16159</div><div> MeanSquares_2_threads 10 0.359703 0.368051 0.388106 0.392696 3.68051</div>
<div> MeanSquares_3_threads 10 0.268024 0.280672 0.296142 0.312557 2.80672</div><div> MeanSquares_4_threads 10 0.199249 0.213209 0.224918 0.2243 2.13209</div>
</div><div><br></div><div><div><br></div><div>itk4 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo</div><br>
</div><div><div><div>System: CAMD5C5PMN1</div><div>Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache </div><div> Serial #: </div><div> Cache: -1</div><div> Clock: 2800</div><div> Cores: 4 cpus x 1 Cores = 4</div>
<div>OSName: Windows</div><div> Release: XP Professional</div><div> Version: Service Pack 3 (Build 2600)</div><div> Platform: x86</div><div> Operating System is 32 bit</div><div>ITK Version: 4.3.0</div><div>Virtual Memory: Total: 2047 Available: 2008</div>
<div>Physical Memory: Total:3581 Available: 1574</div><div> Probe Name: Count Min Mean Stdev Max Total </div><div> FilterChain_1_threads 10 0.0797005 0.0808155 0.0851898 0.0820313 0.808155</div>
<div> FilterChain_2_threads 10 0.0670509 0.0921925 0.0991914 0.109177 0.921925</div><div> FilterChain_3_threads 10 0.0648499 0.113566 0.120975 0.125893 1.13566</div>
<div> FilterChain_4_threads 10 0.0536919 0.104662 0.114558 0.132114 1.04662</div><div> Image2D 10 0.0953979 0.0965282 0.101753 0.0978088 0.965282</div>
<div> Image3D 10 0.196468 0.198288 0.209017 0.200562 1.98288</div><div> MeanSquares_1_threads 10 0.982071 0.986089 1.03944 0.996445 9.86089</div>
<div> MeanSquares_2_threads 10 0.686028 0.728621 0.768821 0.799675 7.28621</div><div> MeanSquares_3_threads 10 0.534885 0.554387 0.5846 0.578133 5.54387</div>
<div> MeanSquares_4_threads 10 0.441551 0.463463 0.488903 0.502502 4.63463</div></div><div><br></div><div><br></div><div>itk3 Linux 64 bit gcc 4.6.1 Build Release</div><div>System: morrigan</div>
<div>Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz</div><div> Serial #:</div><div> Cache: 2048</div>
<div> Clock: 2003</div><div> Cores: 4 cpus x 4 Cores = 16</div><div>OSName: Linux</div><div> Release: 3.0.0-21-generic</div><div> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012</div><div> Platform: x86_64</div>
<div> Operating System is 64 bit</div><div>ITK Version: 3.20.1</div><div>Virtual Memory: Total: 20795 Available: 20795</div><div>Physical Memory: Total:8001 Available: 3416</div><div> Probe Name: Count Min Mean Stdev</div>
<div> Max Total</div><div> FilterChain_1_threads 10 0.0309269 0.031097 0.0327796 0.031528 0.31097</div><div> FilterChain_2_threads 10 0.021672 0.0248219 0.0261976 0.026264 0.248219</div>
<div> FilterChain_3_threads 10 0.0213931 0.0217796 0.0229634 0.0228851 0.217796</div><div> FilterChain_4_threads 10 0.0198419 0.0207421 0.0218725 0.0216701 0.207421</div>
<div> Image2D 10 0.0263629 0.0264159 0.0278451 0.0267591 0.264159</div><div> Image3D 10 0.0263779 0.0263906 0.0278181 0.026463 0.263906</div>
<div> MeanSquares_1_threads 10 0.463909 0.465901 0.491113 0.473644 4.65901</div><div> MeanSquares_2_threads 10 0.349138 0.40529 0.429438 0.487785 4.0529</div>
<div> MeanSquares_3_threads 10 0.331908 0.357685 0.377834 0.400941 3.57685</div><div> MeanSquares_4_threads 10 0.299428 0.336914 0.356878 0.395144 3.36914</div>
<div><br></div><div>itk4 Linux 64-bit gcc 4.6.1 Build Release</div></div><div>System: morrigan</div><div>Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz</div><div> Serial #:</div><div> Cache: 2048</div><div>
Clock: 2003</div><div> Cores: 4 cpus x 4 Cores = 16</div><div>OSName: Linux</div><div> Release: 3.0.0-21-generic</div><div> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012</div><div> Platform: x86_64</div>
<div> Operating System is 64 bit</div><div>ITK Version: 4.3.0</div><div>Virtual Memory: Total: 20795 Available: 20795</div><div>Physical Memory: Total:8001 Available: 3400</div><div> Probe Name: Count Min Mean Stdev</div>
<div> Max Total</div><div> FilterChain_1_threads 10 0.0663331 0.0668247 0.0704416 0.0681288 0.668247</div><div> FilterChain_2_threads 10 0.0353 0.0393441 0.042048 0.0534279 0.393441</div>
<div> FilterChain_3_threads 10 0.025275 0.0265869 0.0282408 0.0364931 0.265869</div><div> FilterChain_4_threads 10 0.0208249 0.0222288 0.0235333 0.028336 0.222288</div>
<div> Image2D 10 0.026361 0.026421 0.0278503 0.0266371 0.26421</div><div> Image3D 10 0.0263991 0.0266509 0.0280954 0.0274949 0.266509</div>
<div> MeanSquares_1_threads 10 0.69005 0.69459 0.732169 0.701676 6.9459</div><div> MeanSquares_2_threads 10 0.590396 0.699826 0.739994 0.766702 6.99826</div>
<div> MeanSquares_3_threads 10 0.527146 0.538844 0.568047 0.548791 5.38844</div><div> MeanSquares_4_threads 10 0.417008 0.454278 0.480139 0.533593 4.54278</div>
<div>--------------------------------------------------------------</div>Rupert Brooks<br><a href="mailto:rupert.brooks@gmail.com" target="_blank">rupert.brooks@gmail.com</a><br><br></div></div><div>
_______________________________________________<br>Powered by <a href="http://www.kitware.com/" target="_blank">www.kitware.com</a><br><br>Visit other Kitware open-source projects at<br><a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>Kitware offers ITK Training Courses, for more information visit:<br><a href="http://kitware.com/products/protraining.php" target="_blank">http://kitware.com/products/protraining.php</a><br><br>Please keep messages on-topic and check the ITK FAQ at:<br>
<a href="http://www.itk.org/Wiki/ITK_FAQ" target="_blank">http://www.itk.org/Wiki/ITK_FAQ</a><br><br>Follow this link to subscribe/unsubscribe:<br><a href="http://www.itk.org/mailman/listinfo/insight-developers" target="_blank">http://www.itk.org/mailman/listinfo/insight-developers</a><br>
</div></blockquote></div><br><div>
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">========================================================</font></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Bradley Lowekamp<span> </span><span> </span></font></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Medical Science and Computing for</font></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Office of High Performance Computing and Communications</font></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">National Library of Medicine<span> </span></font></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica"><a href="mailto:blowekamp@mail.nih.gov" target="_blank">blowekamp@mail.nih.gov</a></font></div><br></span></div></span></span><br>
</div>
<br></div></div></blockquote></div><br></div>
</blockquote></div><br><div>
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><p style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">========================================================</font></p><p style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Bradley Lowekamp<span> </span><span> </span></font></p><p style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Medical Science and Computing for</font></p>
<p style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">Office of High Performance Computing and Communications</font></p>
<p style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica">National Library of Medicine<span> </span></font></p><p style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<font face="Helvetica" size="3" style="font:normal normal normal 12px/normal Helvetica"><a href="mailto:blowekamp@mail.nih.gov" target="_blank">blowekamp@mail.nih.gov</a></font></p><br></span></div></span></span><br>
</div>
<br></div></div></div></div></blockquote></div><br></div></div>