<div>Hello, Kana,</div><div>I might be wrong but as far as I remember, the filters related with RecursiveGaussian aren&#39;t compatible with the streaming filter :(</div><div><br></div><div>Regards</div><br><div class="gmail_quote">
On Thu, Mar 17, 2011 at 11:54 PM, Arunachalam Kana <span dir="ltr">&lt;<a href="mailto:Kana.Arunachalam@fh-wels.at">Kana.Arunachalam@fh-wels.at</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="DE-AT" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-GB">Hi ITK Users,</span></p><p><span lang="EN-GB">I need some advice on memory management and speed. </span></p><p><span lang="EN-GB">My system details: 64 bit windowsXP system and 4Gb RAM.</span></p>
<p><span lang="EN-GB">My goal: To obtain eigen vectors for 100Mb unsigned short image. Eigen vectors not for the whole image but for certain regions (approx occupies 40% of image).</span></p><p><span lang="EN-GB"> </span></p>
<p><span lang="EN-GB">I have to do some preprocessing too.</span></p><p><span lang="EN-GB">The pipeline is :</span></p><p><span lang="EN-GB">Original image -&gt; discretegaussian -&gt; gradientmagnitude -&gt; HessianRecursiveGaussianImageFilter-&gt;symmetric eigenvalue analysis -&gt; finally 3 vector image</span></p>
<p><span lang="EN-GB">For the above pipeline for a 100Mb image, i am running out of RAM.</span></p><p><span lang="EN-GB">After reading itkmails, i decided to use streamfilter to stream part of the image so that the RAM usage is low. I only tested half way and the RAM was overloaded.</span></p>
<p><span lang="EN-GB"> </span></p><p><span lang="EN-GB">I am using streamimagefilter to reduce the memory usage. After calculation of gradientmagnitude i have used 1.08Gb RAM. After this i need 1.2Gb RAM</span></p><p><span lang="EN-GB">For hessianimageoutput, so by using streamfilter i thought i would be reaching 2.5Gm RAM but i reach approx 6Gb(virtual memory is used).</span></p>
<p><span lang="EN-GB"><span>1.<span style="font:7.0pt &quot;Times New Roman&quot;">       </span></span></span><span lang="EN-GB">Am i doing something wrong with the stream filter ? (code given below)</span></p><p><span lang="EN-GB"><span>2.<span style="font:7.0pt &quot;Times New Roman&quot;">       </span></span></span><span lang="EN-GB">I tried another option: As i do not need hessian for the whole image, I used discretehessianfunction for hessian calculation but it is 30 times slower that the recursivehessian. Is discretegaussianimagefunction multithreaded? </span></p>
<p><span lang="EN-GB"><span>3.<span style="font:7.0pt &quot;Times New Roman&quot;">       </span></span></span><span lang="EN-GB">Is there any other way to achieve speed vs RAM compromised solution?</span></p><p><span lang="EN-GB"><span>4.<span style="font:7.0pt &quot;Times New Roman&quot;">       </span></span></span><span lang="EN-GB">The 100Mb data set is test data. The real data is 10Gb for which i will use 64 bit linux system (opensuse 11.2) . will the pipeline be executed in linux too?</span></p>
<p><span lang="EN-GB">Below is my code and after application of filter show the RAM usage:</span></p><p><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  char *infilename = &quot;StreamTest.mhd&quot;;  </span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::CovariantVector&lt;float,3&gt; VectorPixelType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::Vector&lt; VectorPixelType, 3 &gt; EV_PixelType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  //Image type</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB">  typedef itk::Image&lt;unsigned short, 3&gt;              InputImageType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::Image&lt;float, 3&gt;                       FloatImageType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::Image&lt;EV_PixelType,3&gt;                                                  EVImageType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB" style="color:red">**** RAM usage = 674Mb****</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  //reader initialisation and reading file</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB">  typedef itk::ImageFileReader&lt;InputImageType&gt;                     ReaderType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  ReaderType::Pointer reader = ReaderType::New();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  reader-&gt;SetFileName( infilename );</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  reader-&gt;Update();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  InputImageType::Pointer inImage = reader-&gt;GetOutput();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB" style="color:red">**** RAM usage = 783Mb****</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::DiscreteGaussianImageFilter&lt;InputImageType, FloatImageType&gt; RGIFType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  RGIFType::Pointer gaussfilter = RGIFType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  gaussfilter-&gt;SetInput(inImage);</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  gaussfilter-&gt;SetVariance(4.0);</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  gaussfilter-&gt;SetMaximumError(0.01);</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::CastImageFilter&lt;FloatImageType,InputImageType&gt; CIFType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  CIFType::Pointer castfilter = CIFType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  castfilter-&gt;SetInput(gaussfilter-&gt;GetOutput());</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::GradientMagnitudeImageFilter&lt;InputImageType,FloatImageType&gt; GMIFType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  GMIFType::Pointer gmfilter = GMIFType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  gmfilter-&gt;SetInput(castfilter-&gt;GetOutput());</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  gmfilter-&gt;Update();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB">  FloatImageType::Pointer gmImage = gmfilter-&gt;GetOutput();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB" style="color:red">**** RAM usage = 1290Mb****</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  gaussfilter-&gt;UnRegister();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
  castfilter-&gt;UnRegister();</p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span style="color:red">  **** RAM usage = 1080Mb****</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
 </p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">  <span lang="EN-GB">typedef itk::HessianRecursiveGaussianImageFilter&lt;FloatImageType&gt; HessianRecursiveGaussianFilterType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB">  typedef HessianRecursiveGaussianFilterType::OutputImageType HessianImageType;</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  HessianRecursiveGaussianFilterType::Pointer hessianfilter = HessianRecursiveGaussianFilterType::New();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  hessianfilter-&gt;SetInput(gmImage);</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  hessianfilter-&gt;SetSigma(1);</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB"> </span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  typedef itk::StreamingImageFilter&lt;HessianImageType,HessianImageType&gt; StreamerType;</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  StreamerType::Pointer streamer = StreamerType::New();</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  streamer-&gt;SetInput( hessianfilter-&gt;GetOutput() );</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  streamer-&gt;SetNumberOfStreamDivisions( 20 );</span></p><p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB">  streamer-&gt;Update();</span></p>
<p style="margin:0cm;margin-bottom:.0001pt;text-autospace:none"><span lang="EN-GB" style="color:red">**** RAM usage = 3600Mb**** after few seconds it came to **** RAM usage = 6120Mb****</span></p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt;text-autospace:none">
<span lang="EN-GB"> </span></p><p><span lang="EN-GB">Thank you in advance.</span></p><p><span lang="EN-GB">Regards,</span></p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Kana Arunachalam Kannappan</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">
Research Associate</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">FH OÖ Forschungs &amp; Entwicklungs GmbH</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Stelzhamer Strasse 23,</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">
4600 Wels,</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Austria.</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">Phone: +43 (0)7242 72811 -4420</p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt">
<a href="mailto:kana.arunachalam@fh-wels.at" target="_blank">kana.arunachalam@fh-wels.at</a></p><p class="MsoNormal" style="margin:0cm;margin-bottom:.0001pt"><a href="http://www.fh-ooe.at" target="_blank">www.fh-ooe.at</a>; <a href="http://www.3dct.at" target="_blank">www.3dct.at</a></p>
<p class="MsoNormal"> </p></div></div><br>_____________________________________<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at<br>
<a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Kitware offers ITK Training Courses, for more information visit:<br>
<a href="http://www.kitware.com/products/protraining.html" target="_blank">http://www.kitware.com/products/protraining.html</a><br>
<br>
Please keep messages on-topic and check the ITK FAQ at:<br>
<a href="http://www.itk.org/Wiki/ITK_FAQ" target="_blank">http://www.itk.org/Wiki/ITK_FAQ</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.itk.org/mailman/listinfo/insight-users" target="_blank">http://www.itk.org/mailman/listinfo/insight-users</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Sergio Vera<br><br> Alma IT Systems<br> C/ Vilana, 4B, 4º 1ª<br> 08022 Barcelona<br> T. (+34) 932 380 592<br> <a href="http://www.alma3d.com">www.alma3d.com</a><br>