[Insight-users] Evolution : overall pipeline memory print
Luis Ibanez
luis.ibanez at kitware.com
Sun Nov 11 06:48:00 EST 2007
Hi Bill,
This is great !
We can add a static variable to this new class and accumulate
in it all the memory allocation requests for image buffers.
This is also a nice use of the factories, since users could
enable this feature just by putting a share library in the
ITK_AUTOLOAD_PATH, without having to modify the code of their
application in any way, not even having to recompile their
application.
Luis
-------------------
Bill Lorensen wrote:
> Luis,
>
> I just checked in a test, Testing/Code/Common/itkObjectFactoryTest2,cxx
> that exercises the factory load from a library code in
> itkObjectFactoryBase.cxx. The test provides a factory-based
> ImportImageContainer that does some simple memory allocation reporting.
> It might stimulate some further ideas on how to report memory usage.
>
> We'll see if the object loading is portable across ITK systems. I
> already had to exclude the test from the Borland build because of
> problems. I'll leave it out of the Borland build until I can track down
> the problem.
>
> Bill
>
> On Nov 9, 2007 11:58 AM, Luis Ibanez <luis.ibanez at kitware.com
> <mailto:luis.ibanez at kitware.com>> wrote:
>
>
> Hi Julien,
>
>
> We will discuss this topic further today during the ITK tcon.
>
>
> I agree with Karthik that it is not a trivial problem to solve,
> but also agree with you that it is a problem worth looking at.
>
>
> Images are only going to get bigger...
> and they will do so a lot faster than computer memory is
> going to get cheaper. :-/
>
>
> We have also challenging situations when dealing with microscopy
> data from sources such as electron microscopy and confocal
> microscopy...
>
>
>
> One way of simulating the memory behavior of the pipeline
> is to overload/intercept the image method Allocate(), and
> have it report its memory request to a central log, instead
> of actually allocating the memory, then having the pixel
> container simulate that it actually have allocated the
> memory and return a constant (or a random) value for every
> pixel request. It may be done also by intercepting the
> method "AllocateElements(size)" in the itkImportImageContainer,
> which is where the memory allocation is ultimately done.
> (Note that even the itkVectorImage delegates memory allocation
> to its internal ImportImageContainer).
>
>
>
> By running your pipeline in such mode, it will be possible
> to collect the *actual* memory allocation requests, instead
> of the *estimated* ones from every filter, that as Karthik
> pointed out, will be very hard to do in a reliably way for
> the entire toolkit.
>
>
> In that mode, then you could experiment with different
> image partition strategies, for example the ones in the
> StreamingImageFilter. These strategies are managed by
> the RegionSplitter classes. You could by trial an error
> identify what size of image pieces would make possible
> to run the pipeline in the amount of memory that is
> actually available in the hardware.
>
>
> The drawback of this method is that it still will use the
> CPU time of actually computing the output of the filters,
> instead of just making a pass to estimate the memory
> footprint of the toolkit....
>
>
> You may have other ideas on how to implement other
> memory estimation strategies.
>
>
>
> We certainly agree that if such functionality is developed
> it will be much better to add it to ITK than to have it
> in your local copy of the toolkit.
>
>
>
> Regards,
>
>
> Luis
>
>
>
> -------------------------
> Julien Michel wrote:
> > Karthik Krishnan a écrit :
> >
> >> Julien:
> >>
> >> I can understand your issue with being bogged down by memory.
> However,
> >> I don't completely agree with your proposed solution. Here are my
> >> disagreements.
> >
> >
> > Karthik,
> >
> > Thanks a lot for your response. I agree with you on the fact that the
> > GetMemoryPrint() method will not apply to all DataObjects.
> Regarding the
> > step where to call the GetPipelineMemoryPrint() method, I was indeed
> > planning to do this after a
> > itk::ProcessObject::GenerateOutputInformation() and a
> > GenerateInputRequestedRegion() (otherwise it as no meaning at all).
> > I was hoping that minipipeline would be processed seamlessly by the
> > recursive call, but maybe I get it wrong.
> >
> > > In that case, we do not know about the run-time pixel sizes as
> > > illustrated in point 1. It would work for for images where the
> > > pixel-size information is known at build time.
> >
> > Well in our toolkit we recommend the use of VectorImage, from
> which we
> > can retrieve the pixel number of components afther an
> > UpdateOutputInformation(), at run time (but before the
> > UpdateOutputData()). This is not a problem for us because we
> always have
> > the same number of components for all pixels (our images beeing
> acquired
> > by remote sensors). Anyway I agree that this does not fit the
> generic case.
> >
> > We need to adress this memory problem. The best would be to know the
> > peak memory usage, but I think we could do well with the overall
> > pipeline memory usage, even if it is only an estimate that takes
> into
> > account only those DataObjects for which we can compute the memory
> > print. But put this way it sounds more like a workaround and I
> > understand that ITK would not like to have a workaround stuck
> into its
> > very base classes. We would welcome any other suggestion on idea to
> > solve this problem :o)
> >
> > Thanks a lot for your reply, and for ITK itself,
> >
> > Best regards,
> >
> > Julien MICHEL
> > _______________________________________________
> > Insight-users mailing list
> > Insight-users at itk.org <mailto:Insight-users at itk.org>
> > http://www.itk.org/mailman/listinfo/insight-users
> >
>
> _______________________________________________
> Insight-users mailing list
> Insight-users at itk.org <mailto:Insight-users at itk.org>
> http://www.itk.org/mailman/listinfo/insight-users
>
>
More information about the Insight-users
mailing list