Matt,<div><br></div><div>I just upgraded master to current (which i think includes these changes), rebuilt and reran on both systems. I cannot see a statisically significant difference. To truly check we would have to run multiple times and compare, but its safe to say if theres a difference, its subtle. However, i did not particularly choose my filter chain to have the GetInput/Output problem. I threw together something more or less similar to something i already did, so this cant be considered conclusive.<div>
<br></div><div>Rupert</div><div><br></div><div>New results:</div><div>itk4, Visual 2005</div><div><div>System: CAMD5C5PMN1</div><div>Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache</div><div> Serial #:</div>
<div> Cache: -1</div><div> Clock: 2800</div><div> Cores: 4 cpus x 1 Cores = 4</div><div>OSName: Windows</div><div> Release: XP Professional</div><div> Version: Service Pack 3 (Build 2600)</div><div> Platform: x86</div>
<div> Operating System is 32 bit</div><div>ITK Version: 4.3.0</div><div>Virtual Memory: Total: 2047 Available: 2008</div><div>Physical Memory: Total:3581 Available: 1370</div><div> Probe Name: Count Min Mean Stdev Max Total</div>
<div> FilterChain_1_threads 10 0.0799255 0.0812134 0.0856113 0.0824928 0.812134</div><div> FilterChain_2_threads 10 0.0585442 0.101192 0.107718 0.1078 1.01192</div>
<div> FilterChain_3_threads 10 0.0670929 0.107661 0.115547 0.124683 1.07661</div><div> FilterChain_4_threads 10 0.0634041 0.109252 0.117602 0.125126 1.09252</div>
<div> Image2D 10 0.0952263 0.0958263 0.101011 0.0965271 0.958263</div><div> Image3D 10 0.195629 0.197874 0.208581 0.200047 1.97874</div>
<div> MeanSquares_1_threads 10 0.979141 0.98356 1.03678 0.997391 9.8356</div><div> MeanSquares_2_threads 10 0.685131 0.712635 0.751585 0.749096 7.12635</div>
<div> MeanSquares_3_threads 10 0.525139 0.549089 0.579045 0.57336 5.49089</div><div> MeanSquares_4_threads 10 0.439487 0.452317 0.476833 0.458878 4.52317</div>
</div><div><br></div><div>Linux, gcc 4.6.1<br><div>./itkbench</div><div><div>System: morrigan</div><div>Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz</div><div> Serial #:</div><div> Cache: 2048</div><div> Clock: 2003</div>
<div> Cores: 4 cpus x 4 Cores = 16</div><div>OSName: Linux</div><div> Release: 3.0.0-21-generic</div><div> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012</div><div> Platform: x86_64</div><div> Operating System is 64 bit</div>
<div>ITK Version: 4.3.0</div><div>Virtual Memory: Total: 20795 Available: 20695</div><div>Physical Memory: Total:8001 Available: 3538</div><div> Probe Name: Count Min Mean Stdev Max Total</div>
<div> FilterChain_1_threads 10 0.0684249 0.0686138 0.0723256 0.0689371 0.686138</div><div> FilterChain_2_threads 10 0.0367901 0.048942 0.0519886 0.0535438 0.48942</div>
<div> FilterChain_3_threads 10 0.0256481 0.0262128 0.0276426 0.0280211 0.262128</div><div> FilterChain_4_threads 10 0.02005 0.0246643 0.0269426 0.0390608 0.246643</div>
<div> Image2D 10 0.026371 0.0265399 0.0279799 0.0279481 0.265399</div><div> Image3D 10 0.0263801 0.0264313 0.0278611 0.0266159 0.264313</div>
<div> MeanSquares_1_threads 10 0.683572 0.686879 0.724048 0.696691 6.86879</div><div> MeanSquares_2_threads 10 0.613772 0.6994 0.739107 0.766093 6.994</div>
<div> MeanSquares_3_threads 10 0.53469 0.565863 0.596594 0.579526 5.65863</div><div> MeanSquares_4_threads 10 0.429782 0.463741 0.489788 0.50595 4.63741</div>
--------------------------------------------------------------<br>Rupert Brooks<br><a href="mailto:rupert.brooks@gmail.com">rupert.brooks@gmail.com</a><br><br>
<br><br><div class="gmail_quote">On Wed, Jul 25, 2012 at 10:47 AM, Matt McCormick <span dir="ltr"><<a href="mailto:matt.mccormick@kitware.com" target="_blank">matt.mccormick@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Rupert,<br>
<br>
Very interesting. I am curious if the recent GetInput/GetOutput patch<br>
helps with the quantities.<br>
<br>
Thanks,<br>
Matt<br>
<div><div class="h5"><br>
On Wed, Jul 25, 2012 at 2:12 PM, Rupert Brooks <<a href="mailto:rupert.brooks@gmail.com">rupert.brooks@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> I was commenting the other day that ITK4 seems to have lower performance<br>
> than ITK3. I put together a little benchmark to test this objectively.<br>
> Results for a couple of systems are below. The benchmark is neither<br>
> particularly sophisticated nor exact, but i have tried to make it typical of<br>
> what i use ITK for. I have not tried it with Hans' patch as that got<br>
> abandoned. Will follow up separately on that.<br>
><br>
> In the numbers below, you will see that performance seems to have degraded<br>
> across the board. One exception is the 2D image test. In this test i was<br>
> lazy, and i left the image axis aligned with unit spacing. I am suspecting<br>
> a special case optimization in ITK4. At the moment, this blocks my ability<br>
> to upgrade my projects to ITK4, as they are performance critical. Any hints<br>
> and tips to get those milliseconds back will be much appreciated.<br>
><br>
> The code is at <a href="https://github.com/rupertbrooks/itkbench" target="_blank">https://github.com/rupertbrooks/itkbench</a>, i'd be curious if<br>
> others get the same results, and of course, criticism and improvements are<br>
> welcome.<br>
><br>
> Note that the output about what processor / cores is on the system is bogus.<br>
> The windows system is a core i7 with 4 cores, the linux one is a Core2 Quad.<br>
> I suspect a bug in itksys::SystemInformation, but i will follow up on that<br>
> separately also.<br>
><br>
> In all itk builds, ITK_USE_REVIEW is on. In the itk3 builds,<br>
> USE_OPTIMZIZED_REGISTRATION, and the new statistics framework are turned on.<br>
><br>
> Cheers,<br>
> Rupert<br>
><br>
> Data follows.....<br>
><br>
> itk3 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo<br>
> System: CAMD5C5PMN1<br>
> Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache<br>
> Serial #:<br>
> Cache: -1<br>
> Clock: 2800<br>
> Cores: 4 cpus x 1 Cores = 4<br>
> OSName: Windows<br>
> Release: XP Professional<br>
> Version: Service Pack 3 (Build 2600)<br>
> Platform: x86<br>
> Operating System is 32 bit<br>
> ITK Version: 3.20.0<br>
> Virtual Memory: Total: 2047 Available: 2021<br>
> Physical Memory: Total:3581 Available: 1457<br>
> Probe Name: Count Min Mean<br>
> Stdev Max Total<br>
> FilterChain_1_threads 10 0.0345688 0.0361805<br>
> 0.0381523 0.0375099 0.361805<br>
> FilterChain_2_threads 10 0.0257301 0.0265144<br>
> 0.0279574 0.0282478 0.265144<br>
> FilterChain_3_threads 10 0.022007 0.0232452<br>
> 0.0245125 0.0240822 0.232452<br>
> FilterChain_4_threads 10 0.0199661 0.0221756<br>
> 0.0235205 0.0288773 0.221756<br>
> Image2D 10 0.144909 0.14665<br>
> 0.154586 0.148136 1.4665<br>
> Image3D 10 0.19125 0.193378<br>
> 0.203846 0.196495 1.93378<br>
> MeanSquares_1_threads 10 0.612728 0.616159<br>
> 0.649503 0.627789 6.16159<br>
> MeanSquares_2_threads 10 0.359703 0.368051<br>
> 0.388106 0.392696 3.68051<br>
> MeanSquares_3_threads 10 0.268024 0.280672<br>
> 0.296142 0.312557 2.80672<br>
> MeanSquares_4_threads 10 0.199249 0.213209<br>
> 0.224918 0.2243 2.13209<br>
><br>
><br>
> itk4 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo<br>
><br>
> System: CAMD5C5PMN1<br>
> Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache<br>
> Serial #:<br>
> Cache: -1<br>
> Clock: 2800<br>
> Cores: 4 cpus x 1 Cores = 4<br>
> OSName: Windows<br>
> Release: XP Professional<br>
> Version: Service Pack 3 (Build 2600)<br>
> Platform: x86<br>
> Operating System is 32 bit<br>
> ITK Version: 4.3.0<br>
> Virtual Memory: Total: 2047 Available: 2008<br>
> Physical Memory: Total:3581 Available: 1574<br>
> Probe Name: Count Min Mean<br>
> Stdev Max Total<br>
> FilterChain_1_threads 10 0.0797005 0.0808155<br>
> 0.0851898 0.0820313 0.808155<br>
> FilterChain_2_threads 10 0.0670509 0.0921925<br>
> 0.0991914 0.109177 0.921925<br>
> FilterChain_3_threads 10 0.0648499 0.113566<br>
> 0.120975 0.125893 1.13566<br>
> FilterChain_4_threads 10 0.0536919 0.104662<br>
> 0.114558 0.132114 1.04662<br>
> Image2D 10 0.0953979 0.0965282<br>
> 0.101753 0.0978088 0.965282<br>
> Image3D 10 0.196468 0.198288<br>
> 0.209017 0.200562 1.98288<br>
> MeanSquares_1_threads 10 0.982071 0.986089<br>
> 1.03944 0.996445 9.86089<br>
> MeanSquares_2_threads 10 0.686028 0.728621<br>
> 0.768821 0.799675 7.28621<br>
> MeanSquares_3_threads 10 0.534885 0.554387<br>
> 0.5846 0.578133 5.54387<br>
> MeanSquares_4_threads 10 0.441551 0.463463<br>
> 0.488903 0.502502 4.63463<br>
><br>
><br>
> itk3 Linux 64 bit gcc 4.6.1 Build Release<br>
> System: morrigan<br>
> Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz<br>
> Serial #:<br>
> Cache: 2048<br>
> Clock: 2003<br>
> Cores: 4 cpus x 4 Cores = 16<br>
> OSName: Linux<br>
> Release: 3.0.0-21-generic<br>
> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012<br>
> Platform: x86_64<br>
> Operating System is 64 bit<br>
> ITK Version: 3.20.1<br>
> Virtual Memory: Total: 20795 Available: 20795<br>
> Physical Memory: Total:8001 Available: 3416<br>
> Probe Name: Count Min Mean<br>
> Stdev<br>
> Max Total<br>
> FilterChain_1_threads 10 0.0309269 0.031097<br>
> 0.0327796 0.031528 0.31097<br>
> FilterChain_2_threads 10 0.021672 0.0248219<br>
> 0.0261976 0.026264 0.248219<br>
> FilterChain_3_threads 10 0.0213931 0.0217796<br>
> 0.0229634 0.0228851 0.217796<br>
> FilterChain_4_threads 10 0.0198419 0.0207421<br>
> 0.0218725 0.0216701 0.207421<br>
> Image2D 10 0.0263629 0.0264159<br>
> 0.0278451 0.0267591 0.264159<br>
> Image3D 10 0.0263779 0.0263906<br>
> 0.0278181 0.026463 0.263906<br>
> MeanSquares_1_threads 10 0.463909 0.465901<br>
> 0.491113 0.473644 4.65901<br>
> MeanSquares_2_threads 10 0.349138 0.40529<br>
> 0.429438 0.487785 4.0529<br>
> MeanSquares_3_threads 10 0.331908 0.357685<br>
> 0.377834 0.400941 3.57685<br>
> MeanSquares_4_threads 10 0.299428 0.336914<br>
> 0.356878 0.395144 3.36914<br>
><br>
> itk4 Linux 64-bit gcc 4.6.1 Build Release<br>
> System: morrigan<br>
> Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz<br>
> Serial #:<br>
> Cache: 2048<br>
> Clock: 2003<br>
> Cores: 4 cpus x 4 Cores = 16<br>
> OSName: Linux<br>
> Release: 3.0.0-21-generic<br>
> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012<br>
> Platform: x86_64<br>
> Operating System is 64 bit<br>
> ITK Version: 4.3.0<br>
> Virtual Memory: Total: 20795 Available: 20795<br>
> Physical Memory: Total:8001 Available: 3400<br>
> Probe Name: Count Min Mean<br>
> Stdev<br>
> Max Total<br>
> FilterChain_1_threads 10 0.0663331 0.0668247<br>
> 0.0704416 0.0681288 0.668247<br>
> FilterChain_2_threads 10 0.0353 0.0393441<br>
> 0.042048 0.0534279 0.393441<br>
> FilterChain_3_threads 10 0.025275 0.0265869<br>
> 0.0282408 0.0364931 0.265869<br>
> FilterChain_4_threads 10 0.0208249 0.0222288<br>
> 0.0235333 0.028336 0.222288<br>
> Image2D 10 0.026361 0.026421<br>
> 0.0278503 0.0266371 0.26421<br>
> Image3D 10 0.0263991 0.0266509<br>
> 0.0280954 0.0274949 0.266509<br>
> MeanSquares_1_threads 10 0.69005 0.69459<br>
> 0.732169 0.701676 6.9459<br>
> MeanSquares_2_threads 10 0.590396 0.699826<br>
> 0.739994 0.766702 6.99826<br>
> MeanSquares_3_threads 10 0.527146 0.538844<br>
> 0.568047 0.548791 5.38844<br>
> MeanSquares_4_threads 10 0.417008 0.454278<br>
> 0.480139 0.533593 4.54278<br>
> --------------------------------------------------------------<br>
> Rupert Brooks<br>
> <a href="mailto:rupert.brooks@gmail.com">rupert.brooks@gmail.com</a><br>
><br>
><br>
</div></div>> _______________________________________________<br>
> Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
><br>
> Visit other Kitware open-source projects at<br>
> <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
><br>
> Kitware offers ITK Training Courses, for more information visit:<br>
> <a href="http://kitware.com/products/protraining.php" target="_blank">http://kitware.com/products/protraining.php</a><br>
><br>
> Please keep messages on-topic and check the ITK FAQ at:<br>
> <a href="http://www.itk.org/Wiki/ITK_FAQ" target="_blank">http://www.itk.org/Wiki/ITK_FAQ</a><br>
><br>
> Follow this link to subscribe/unsubscribe:<br>
> <a href="http://www.itk.org/mailman/listinfo/insight-developers" target="_blank">http://www.itk.org/mailman/listinfo/insight-developers</a><br>
><br>
</blockquote></div><br></div></div></div>