FW: [Insight-users] MultiResMIRegistration Example
Lorensen, William E (Research)
lorensen@crd.ge.com
Wed, 10 Apr 2002 23:54:25 -0400
Lydia,
As you know, I've had success using MI for MRI-MRI and CT-CT.
Does your example normalize the images? This really affects performance,
Bill
-----Original Message-----
From: Lydia Ng [mailto:lng@insightful.com]
Sent: Wednesday, April 10, 2002 7:05 PM
To: Luis Ibanez; David Rich
Cc: insight-users@public.kitware.com
Subject: RE: FW: [Insight-users] MultiResMIRegistration Example
Luis, David:
In terms of memory - I think the multi-resolution image pyramid
is indeed the culprit.
Historically, it only computed the images at each level one at
at time. That was subsequently changed to computing all the level
following comments of the code review. To support large images I think
this will need to change back to outputing one level at a time.
-however-
What I am more concern about is that David is reporting
that the underlying algorithm is not working.
Luis: you are correct when you say that when dealing with
intra-modality one might be better off using something other
than MI. But both you and I have seen working demos of when the
same MI code can successfully register MR to MR with
significant translation and rotation.
David:
[1] How quantitatively bad were the results
when you only used 50 sample points?
[2] Have you tried tests where you have shifted and/or rotated
the image by a known amount? What happened?
[3] Would you mind posting your parameter file?
- Lydia
> -----Original Message-----
> From: Luis Ibanez [mailto:luis.ibanez@kitware.com]
> Sent: Wednesday, April 10, 2002 3:35 PM
> To: David Rich
> Cc: Lydia Ng; insight-users@public.kitware.com
> Subject: Re: FW: [Insight-users] MultiResMIRegistration Example
>
>
>
> Hi David,
>
> Thanks for your feedback on you experience with
> the toolkit.
>
> ITK is not intended only for education and research
> purposes. It has been designed to be used for
> commercial products as well. Particlar attention
> has been paid to the terms of the license so that
> commercial use becomes a reality.
>
> Performance both in memory and computing time are
> important elements in ITK design. The heavy use of
> templated code, for example, responds to the need of
> simplifying the maintenance of the code and improving
> its performance. Templates allow a lot of code to be
> inlined and also substitute the run-time mechanism
> of virtual function overloading with more expedite
> compile-time instantiations.
>
> However the toolkit needs to pass through the natural
> process of code maturation. In this sense, feedback
> from users is of fundamental importance and is always
> greately appreciated.
>
> Your report on memory consumption is quite interesting.
> We certainly want to track down the reasons for the
> excess of memory use and come up with a solution for it.
>
> The fact that you observed the memory growing as the
> registration ran, leads to suspect that the image pyramids
> may be implicated. At each stage of the registration
> only images at the same level of the pyramid are required
> to be in memory. It is likely that the current algorithm
> is not releasing the memory used by previous levels of
> the pyramid. Also, the subsampling is performed by first
> smoothing the images with Gaussian Blurring filters.
> These filters allocate memory for internal copies of the
> images for intermediate computations. It is also likely
> that this memory is failing to be released. If this is
> the case it should be relatively easy to fix. We will
> explore these possibilities.
>
>
> You also mention an option for increasing performance
> by computing the metric over a restricted region of
> interest in the image. This modification has recently
> being made on the metrics. In order to take advantage
> of it you just have to provide the metric with a region
> on the Fixed image. The rest of the Fixed image will
> be excluded from the computations required for registration.
>
> The use is like:
>
> metric->SetFixedImageRegion( region );
>
>
>
>
> A comment about Mutual Information: this metric is not
> particularly well suited for images of the same modality,
> and specially not for registering an image with itself.
> The reason is that when you produce the joint histogram
> of an image with itself (assuming they are already
> perfectly registered) it will result in filling only the
> diagonal elements of the joint histogram. In consequence
> only N out of NxN bins will be contributing information to
> the matching. Mutual Information behaves better when it is
> used with Multimodal images because in that case the joint
> histogram have a better distribution of samples (larger
> joint entropy). The registration will be naturally slow when
> the images get closer to the optimal position because in
> that case only the regions of the image with large gradients
> will contribute to variations in the metric (and hence to the
> computation of the metric derivatives that drive the gradient
> descent optimization method). The chances of random points
> to fall into gradient areas are pretty small in general, so
> most of the points will be only contributing to the diagonal
> of the joint histogram. That is, most of them are just
> reporting that the images seems to be registrered. That may
> explain why your test required a larger-than-normal population
> of points to progress and why it fails to register after
> getting to 1 or 2 pixels of distance from the optimal position.
> (which seems reasonable to expect for an image with itself).
> Multimodal images have better chances of producing joint
> entropy gradients with smaller populations of sampling points.
> Same modality images will register much better by using a
> MeanSquare metric for low resolutions and a PatternIntensity
> metric for high resolution. [I understand that you did this
> only for the purpouse of testing the code so this is just a
> comment for the record]
>
>
> We'll profile the memory consumption of the method and get
> back to you as soon as we identify the sources.
>
>
> Thanks a lot for your feedback
>
>
>
> Luis
>
>
> ============================================
> >>
> >>While watching this operate for over an hour, I observed that
> >>the executable started out at about 20MB but increased to
> >>almost 60MB before completion! Such excessive memory usage
> >>becomes intolerable in real-world applications-what if I were
> >>trying to register two 40MB images?
> >>
> >>So, my first question is whether or not ITK is intended to be
> >>used in commercial applications or is it only designed as a
> >>tool for people who want to experiment with many different
> >>models? The templated classes would certainly facilitate the
> >>latter but the complexity combined with size and time
> >>constraints do not contribute to the former.
> >>
> >>In an attempt to better understand the parameters that could
> >>be entered, I continued testing. The initial results in
> >>about 23 minutes of operation were:
> >>
> >>Final parameters:
> >>-0.000313974 -0.000876174 -0.000350401 1 -0.431262
> >>0.368347 -0.0612012
> >>
> >>The rotation parameters indicate a close fit although the
> >>translations are disconcerting, considering that an exact
> >>match should be more feasible (Note: the voxel size is 2.72
> >>in all directions).
> >>
> >>I changed the default input of 0.4 standard deviation for the
> >>registration to 0.2 and used 100 sample points, and the
> >>results were miserable:
> >>
> >>Final parameters:
> >>0.574853 -0.761886 -0.298448 -0.00139123 -19.8732
> >>28.4372 -79.6355
> >>
> >>With a standard deviation of 0.6 and 100 sample points the
> >>results are different but still miserable:
> >>
> >>Final parameters:
> >>-0.0671739 0.0841046 -0.994183 0.00385258 -14.183
> >>-1.85797 -1.66959
> >>
> >>The conclusion seems to be that with enough sample points the
> >>basic algorithm may provide satisfactory results. However,
> >>from past experience the time delay becomes critical for
> >>medical personnel. And if the algorithm requires a minimum
> >>of 500 data points, 60MB RAM, and 23 minutes on a 1.6MB image
> >>registered with itself, what would be required for two
> >>less-similar images of larger sizes?
> >>
> >>An examination of the source code to try to understand the
> >>parameters and whether or not the memory can be handled more
> >>efficiently again reminded me of the Microsoft ATL wizard.
> >>Only this time it seemed like it was necessary to understand
> >>the complexities of the ATL wizard for the purpose of
> >>creating a specific implementation. And again it occurred to
> >>me that the purpose of ITK seems to be that of creating a
> >>framework for research and experimentation for those who are
> >>not concerned with commercial requirements of speed and
> >>hardware constraints. Am I trying to use this contrary to
> >>the intent of the toolkit?
> >>
> >>On the other hand, is it possible that the ITK could be
> >>developed into something more like the ATL wizard? That is,
> >>ITK with the power of the templates built in, could be used
> >>to generate a basic code structure for the specific
> >>implementation requested. With such a code framework
> >>generated, it might be more feasible for the researcher as
> >>well as the commercial user to customize the specific
> >>implementation for speed and memory or to modify the
> >>mathematical calculations to fit specific requirements.
> >>
> >>At this point I feel little more than frustrated. Although I
> >>would like to be able to provide commercial support to public
> >>code, customizing it for customers but being able to leave
> >>the source with them, I can only recommend that ITK is not
> >>the right framework for commercial applications. It is slow,
> >>cumbersome, and requires significant hardware resources. If
> >>anyone wants to delve into the complexities of the code, they
> >>have to peel away multiple layers of typedefs and templated
> >>classes to try to evaluate implemented constructs and
> >>operations. If the end result in operation, size, and
> >>effectiveness were tremendous, the complexities could be
> >>overlooked. That does not seem to be the case.
> >>
> >>I would like to report more positive feedback. Can you help me?
> >>1) For the MultResMIRegistration example, how can I identify
> >>parameters that will be stable for generic image sets?
> >>2) How can I improve the time and memory requirements?
> >>3) Can I tell the algorithm to skip the blank slices at the
> >>front and/or back of the data of interest?
> >>4) Or, is this toolkit just not intended to be used in
> >>software used by medical personnel and others whose time is
> critical?
> >>
> >>Any suggestions that might help me understand the toolkit
> >>better or how to make it effective would be greatly appreciated.
> >>
> >>Dave Rich
> >>
> >>
> >>
>
>
>
_______________________________________________
Insight-users mailing list
Insight-users@public.kitware.com
http://public.kitware.com/mailman/listinfo/insight-users