<div>Hi,</div><div><br></div><div>I was commenting the other day that ITK4 seems to have lower performance than ITK3. I put together a little benchmark to test this objectively. Results for a couple of systems are below. The benchmark is neither particularly sophisticated nor exact, but i have tried to make it typical of what i use ITK for. I have not tried it with Hans' patch as that got abandoned. Will follow up separately on that.</div>
<div><br></div><div>In the numbers below, you will see that performance seems to have degraded across the board. One exception is the 2D image test. In this test i was lazy, and i left the image axis aligned with unit spacing. I am suspecting a special case optimization in ITK4. At the moment, this blocks my ability to upgrade my projects to ITK4, as they are performance critical. Any hints and tips to get those milliseconds back will be much appreciated.</div>
<div><br></div><div>The code is at <a href="https://github.com/rupertbrooks/itkbench">https://github.com/rupertbrooks/itkbench</a>, i'd be curious if others get the same results, and of course, criticism and improvements are welcome. </div>
<div><br></div><div>Note that the output about what processor / cores is on the system is bogus. The windows system is a core i7 with 4 cores, the linux one is a Core2 Quad. I suspect a bug in itksys::SystemInformation, but i will follow up on that separately also. </div>
<div><br></div><div>In all itk builds, ITK_USE_REVIEW is on. In the itk3 builds, USE_OPTIMZIZED_REGISTRATION, and the new statistics framework are turned on.</div><div><br></div><div>Cheers,</div><div>Rupert</div><div><br>
</div><div>Data follows.....</div>
<div><br></div><div>itk3 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo </div><div><div>System: CAMD5C5PMN1</div><div>Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache </div><div> Serial #: </div>
<div> Cache: -1</div><div> Clock: 2800</div><div> Cores: 4 cpus x 1 Cores = 4</div><div>OSName: Windows</div><div> Release: XP Professional</div><div> Version: Service Pack 3 (Build 2600)</div><div> Platform: x86</div>
<div> Operating System is 32 bit</div><div>ITK Version: 3.20.0</div><div>Virtual Memory: Total: 2047 Available: 2021</div><div>Physical Memory: Total:3581 Available: 1457</div><div> Probe Name: Count Min Mean Stdev Max Total </div>
<div> FilterChain_1_threads 10 0.0345688 0.0361805 0.0381523 0.0375099 0.361805</div><div> FilterChain_2_threads 10 0.0257301 0.0265144 0.0279574 0.0282478 0.265144</div>
<div> FilterChain_3_threads 10 0.022007 0.0232452 0.0245125 0.0240822 0.232452</div><div> FilterChain_4_threads 10 0.0199661 0.0221756 0.0235205 0.0288773 0.221756</div>
<div> Image2D 10 0.144909 0.14665 0.154586 0.148136 1.4665</div><div> Image3D 10 0.19125 0.193378 0.203846 0.196495 1.93378</div>
<div> MeanSquares_1_threads 10 0.612728 0.616159 0.649503 0.627789 6.16159</div><div> MeanSquares_2_threads 10 0.359703 0.368051 0.388106 0.392696 3.68051</div>
<div> MeanSquares_3_threads 10 0.268024 0.280672 0.296142 0.312557 2.80672</div><div> MeanSquares_4_threads 10 0.199249 0.213209 0.224918 0.2243 2.13209</div>
</div><div><br></div><div><div><br></div><div>itk4 Windows XP 32bit Visual Studio Pro 2005 Build RelWithDebInfo</div><br>
</div><div><div><div>System: CAMD5C5PMN1</div><div>Processor: Pentium III (0.18 micron) With 1 Or 2 MB On-Die L2 Cache </div><div> Serial #: </div><div> Cache: -1</div><div> Clock: 2800</div><div> Cores: 4 cpus x 1 Cores = 4</div>
<div>OSName: Windows</div><div> Release: XP Professional</div><div> Version: Service Pack 3 (Build 2600)</div><div> Platform: x86</div><div> Operating System is 32 bit</div><div>ITK Version: 4.3.0</div><div>Virtual Memory: Total: 2047 Available: 2008</div>
<div>Physical Memory: Total:3581 Available: 1574</div><div> Probe Name: Count Min Mean Stdev Max Total </div><div> FilterChain_1_threads 10 0.0797005 0.0808155 0.0851898 0.0820313 0.808155</div>
<div> FilterChain_2_threads 10 0.0670509 0.0921925 0.0991914 0.109177 0.921925</div><div> FilterChain_3_threads 10 0.0648499 0.113566 0.120975 0.125893 1.13566</div>
<div> FilterChain_4_threads 10 0.0536919 0.104662 0.114558 0.132114 1.04662</div><div> Image2D 10 0.0953979 0.0965282 0.101753 0.0978088 0.965282</div>
<div> Image3D 10 0.196468 0.198288 0.209017 0.200562 1.98288</div><div> MeanSquares_1_threads 10 0.982071 0.986089 1.03944 0.996445 9.86089</div>
<div> MeanSquares_2_threads 10 0.686028 0.728621 0.768821 0.799675 7.28621</div><div> MeanSquares_3_threads 10 0.534885 0.554387 0.5846 0.578133 5.54387</div>
<div> MeanSquares_4_threads 10 0.441551 0.463463 0.488903 0.502502 4.63463</div></div><div><br></div><div><br></div><div>itk3 Linux 64 bit gcc 4.6.1 Build Release</div><div>System: morrigan</div>
<div>Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz</div><div> Serial #:</div><div> Cache: 2048</div>
<div> Clock: 2003</div><div> Cores: 4 cpus x 4 Cores = 16</div><div>OSName: Linux</div><div> Release: 3.0.0-21-generic</div><div> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012</div><div> Platform: x86_64</div>
<div> Operating System is 64 bit</div><div>ITK Version: 3.20.1</div><div>Virtual Memory: Total: 20795 Available: 20795</div><div>Physical Memory: Total:8001 Available: 3416</div><div> Probe Name: Count Min Mean Stdev</div>
<div> Max Total</div><div> FilterChain_1_threads 10 0.0309269 0.031097 0.0327796 0.031528 0.31097</div><div> FilterChain_2_threads 10 0.021672 0.0248219 0.0261976 0.026264 0.248219</div>
<div> FilterChain_3_threads 10 0.0213931 0.0217796 0.0229634 0.0228851 0.217796</div><div> FilterChain_4_threads 10 0.0198419 0.0207421 0.0218725 0.0216701 0.207421</div>
<div> Image2D 10 0.0263629 0.0264159 0.0278451 0.0267591 0.264159</div><div> Image3D 10 0.0263779 0.0263906 0.0278181 0.026463 0.263906</div>
<div> MeanSquares_1_threads 10 0.463909 0.465901 0.491113 0.473644 4.65901</div><div> MeanSquares_2_threads 10 0.349138 0.40529 0.429438 0.487785 4.0529</div>
<div> MeanSquares_3_threads 10 0.331908 0.357685 0.377834 0.400941 3.57685</div><div> MeanSquares_4_threads 10 0.299428 0.336914 0.356878 0.395144 3.36914</div>
<div><br></div><div>itk4 Linux 64-bit gcc 4.6.1 Build Release</div></div><div>System: morrigan</div><div>Processor: Intel(R) Core(TM)2 Duo CPU T9500 @ 2.60GHz</div><div> Serial #:</div><div> Cache: 2048</div><div>
Clock: 2003</div><div> Cores: 4 cpus x 4 Cores = 16</div><div>OSName: Linux</div><div> Release: 3.0.0-21-generic</div><div> Version: #35-Ubuntu SMP Fri May 25 17:57:41 UTC 2012</div><div> Platform: x86_64</div>
<div> Operating System is 64 bit</div><div>ITK Version: 4.3.0</div><div>Virtual Memory: Total: 20795 Available: 20795</div><div>Physical Memory: Total:8001 Available: 3400</div><div> Probe Name: Count Min Mean Stdev</div>
<div> Max Total</div><div> FilterChain_1_threads 10 0.0663331 0.0668247 0.0704416 0.0681288 0.668247</div><div> FilterChain_2_threads 10 0.0353 0.0393441 0.042048 0.0534279 0.393441</div>
<div> FilterChain_3_threads 10 0.025275 0.0265869 0.0282408 0.0364931 0.265869</div><div> FilterChain_4_threads 10 0.0208249 0.0222288 0.0235333 0.028336 0.222288</div>
<div> Image2D 10 0.026361 0.026421 0.0278503 0.0266371 0.26421</div><div> Image3D 10 0.0263991 0.0266509 0.0280954 0.0274949 0.266509</div>
<div> MeanSquares_1_threads 10 0.69005 0.69459 0.732169 0.701676 6.9459</div><div> MeanSquares_2_threads 10 0.590396 0.699826 0.739994 0.766702 6.99826</div>
<div> MeanSquares_3_threads 10 0.527146 0.538844 0.568047 0.548791 5.38844</div><div> MeanSquares_4_threads 10 0.417008 0.454278 0.480139 0.533593 4.54278</div>
<div>--------------------------------------------------------------</div>Rupert Brooks<br><a href="mailto:rupert.brooks@gmail.com" target="_blank">rupert.brooks@gmail.com</a><br><br>