[Insight-users] MultiResMIRegistration Example
Lydia Ng
lng@insightful.com
Wed, 10 Apr 2002 12:35:12 -0700
> > Ans: I altered the example code with the following line in=20
> > the header:
> >=20
> > typedef unsigned char InputPixelType;
>=20
> Do you dump out the pgms? Do they look okay?
>=20
> > Ans: Yes, those results would be reasonable, except that it=20
> > took 23 minutes and 60MB of RAM to generate these answers for=20
> > a small image in a situation where there should be no=20
> > ambiguity. According to the comments in the code, it should=20
> > work with only 50 sample points, yet this required 500 before=20
> > it would work.
>=20
> The cost of computing MI is O(N^2) where N is the number of samples.
> So it very costly to use a large number of sample points.
> I am still trying to understand why you would
> need 500 sample points. Another group using this code=20
> has independently found that it was fairly insensitive to
> the number of sample points.
>=20
> BTW are you using the beta tar-ball or downloading from cvs?
>=20
> > -----Original Message-----
> > From: David Rich [mailto:David.Rich@trw.com]
> > Sent: Wednesday, April 10, 2002 12:15 PM
> > To: Lydia Ng; insight-users@kitware.com
> > Subject: RE: [Insight-users] MultiResMIRegistration Example
> >=20
> >=20
> > See answers intermingled with questions below.
> > Dave
> >=20
> > >>> "Lydia Ng" <lng@insightful.com> 04/10/02 12:20PM >>>
> > Dear David,
> >=20
> > I very sympathize with your frustration. And yes, more work needs
> > to be done in terms of memory usage, speed and selecting regions
> > of interest etc to take the current code forward.
> >=20
> > I am still working through your email. I think as a first step,=20
> > we should iterate as to what when wrong with the example you try.=20
> > I have cc'd this email to insight-users to get input of others
> > as well.
> >=20
> > Here is the first set of questions for you -=20
> > sorry if some questions seem a bit silly -=20
> > it just helps me to get a picture of what you are trying to do:
> >=20
> > [1] I noticed in your email that your input image was "single-byte".
> > The example application only takes in signed-short (16bits) raw
> > data? Did you alter you image data to suit? Or did you alter the
> > example code to suit? If you altered the example code can you
> > post the altered code.
> >=20
> > Ans: I altered the example code with the following line in=20
> > the header:
> >=20
> > typedef unsigned char InputPixelType;
> >=20
> >=20
> > [2] What is the content of the image you were using. If is a real
> > medical image? If so, what modality and body part is it?=20
> > If is was synthetic image, does it have lots symmetry?=20
> > Is the pattern regular?
> >=20
> > Ans: MR image of the head
> >=20
> > [3] When you registered the image with itself, you say you got
> > the following results:
> > > Final parameters:=20
> > > -0.000313974 -0.000876174 -0.000350401 1 -0.431262 =20
> > > 0.368347 -0.0612012=20
> >=20
> > If your voxel size is 2.72 in all direction than the translation
> > is out by (0.16, 0.13, 0.02) of a pixel. Unless, I misinterpreted
> > your meaning I think that this is within range of published=20
> results of
> > MI? =20
> >=20
> > Ans: Yes, those results would be reasonable, except that it=20
> > took 23 minutes and 60MB of RAM to generate these answers for=20
> > a small image in a situation where there should be no=20
> > ambiguity. According to the comments in the code, it should=20
> > work with only 50 sample points, yet this required 500 before=20
> > it would work.
> >=20
> > The concern is what happens when the input images are less=20
> > similar. By reducing the sample size for evaluation of the=20
> > impact of changing some of the parameters, it was very=20
> > difficult to predict the impact of those changes. These=20
> > issues make it difficult to extrapolate to a complete=20
> > user-friendly code for a non-programmer to use. I need to be=20
> > confident of resolving these issues before attempting to=20
> > implement a complete code--and then find instability and=20
> > unacceptable constraints.
> >=20
> > Thanks for your help
> > Dave
> >=20
> > - Lydia
> >=20
> > > -----Original Message-----
> > > From: David Rich [mailto:David.Rich@trw.com]=20
> > > Sent: Wednesday, April 10, 2002 10:25 AM
> > > To: Lydia Ng
> > > Subject: RE: [Insight-users] MultiResMIRegistration Example
> > >=20
> > >=20
> > > Lydia,
> > >=20
> > > I think I need some help using ITK. I need to create a=20
> > > real-world application for registration. ITK looked like the=20
> > > perfect solution-something that could be embedded in other=20
> > > software, with source code in the public domain for others to=20
> > > be able to use as needed. =20
> > >=20
> > > I also liked the potential power built into ITK with all of=20
> > > the templated classes. This appears to be similar to=20
> > > Microsoft's ATL wizard, which almost automatically generates=20
> > > all of the customized code needed for COM objects and creates=20
> > > a ready-to-go framework for adding specific capabilities.
> > >=20
> > > The recent published works for registration indicate that=20
> > > mutual information-based methods work very well, especially=20
> > > when combined with multi-resolution calculations. A perfect=20
> > > fit for ITK. So I began to test the example code (as=20
> > > mentioned previously) for multi-resolution mutual information=20
> > > gradient descent registration. Skipping over miserable=20
> > > failures on early attempts, I settled on registering a single=20
> > > image with itself. This should result in a very rapid and=20
> > > obvious solution. The results were not very satisfying.
> > >=20
> > > I tried changing the translation scale from less than a pixel=20
> > > to much larger, but that only changes from one poor result to=20
> > > a drastically different one. I increased the number of=20
> > > sample points for calculations. 500 points seemed to work=20
> > > almost okay, although 1000 points worked better. (The image=20
> > > is 128 x 128 x 105, single-byte pixels, about 1.6MB total.)
> > >=20
> > > While watching this operate for over an hour, I observed that=20
> > > the executable started out at about 20MB but increased to=20
> > > almost 60MB before completion! Such excessive memory usage=20
> > > becomes intolerable in real-world applications-what if I were=20
> > > trying to register two 40MB images?=20
> > >=20
> > > So, my first question is whether or not ITK is intended to be=20
> > > used in commercial applications or is it only designed as a=20
> > > tool for people who want to experiment with many different=20
> > > models? The templated classes would certainly facilitate the=20
> > > latter but the complexity combined with size and time=20
> > > constraints do not contribute to the former.
> > >=20
> > > In an attempt to better understand the parameters that could=20
> > > be entered, I continued testing. The initial results in=20
> > > about 23 minutes of operation were:
> > >=20
> > > Final parameters:=20
> > > -0.000313974 -0.000876174 -0.000350401 1 -0.431262 =20
> > > 0.368347 -0.0612012 =20
> > >=20
> > > The rotation parameters indicate a close fit although the=20
> > > translations are disconcerting, considering that an exact=20
> > > match should be more feasible (Note: the voxel size is 2.72=20
> > > in all directions).
> > >=20
> > > I changed the default input of 0.4 standard deviation for the=20
> > > registration to 0.2 and used 100 sample points, and the=20
> > > results were miserable:
> > >=20
> > > Final parameters:=20
> > > 0.574853 -0.761886 -0.298448 -0.00139123 -19.8732 =20
> > > 28.4372 -79.6355 =20
> > >=20
> > > With a standard deviation of 0.6 and 100 sample points the=20
> > > results are different but still miserable:
> > >=20
> > > Final parameters:=20
> > > -0.0671739 0.0841046 -0.994183 0.00385258 -14.183 =20
> > > -1.85797 -1.66959 =20
> > >=20
> > > The conclusion seems to be that with enough sample points the=20
> > > basic algorithm may provide satisfactory results. However,=20
> > > from past experience the time delay becomes critical for=20
> > > medical personnel. And if the algorithm requires a minimum=20
> > > of 500 data points, 60MB RAM, and 23 minutes on a 1.6MB image=20
> > > registered with itself, what would be required for two=20
> > > less-similar images of larger sizes?
> > >=20
> > > An examination of the source code to try to understand the=20
> > > parameters and whether or not the memory can be handled more=20
> > > efficiently again reminded me of the Microsoft ATL wizard. =20
> > > Only this time it seemed like it was necessary to understand=20
> > > the complexities of the ATL wizard for the purpose of=20
> > > creating a specific implementation. And again it occurred to=20
> > > me that the purpose of ITK seems to be that of creating a=20
> > > framework for research and experimentation for those who are=20
> > > not concerned with commercial requirements of speed and=20
> > > hardware constraints. Am I trying to use this contrary to=20
> > > the intent of the toolkit?
> > >=20
> > > On the other hand, is it possible that the ITK could be=20
> > > developed into something more like the ATL wizard? That is,=20
> > > ITK with the power of the templates built in, could be used=20
> > > to generate a basic code structure for the specific=20
> > > implementation requested. With such a code framework=20
> > > generated, it might be more feasible for the researcher as=20
> > > well as the commercial user to customize the specific=20
> > > implementation for speed and memory or to modify the=20
> > > mathematical calculations to fit specific requirements. =20
> > >=20
> > > At this point I feel little more than frustrated. Although I=20
> > > would like to be able to provide commercial support to public=20
> > > code, customizing it for customers but being able to leave=20
> > > the source with them, I can only recommend that ITK is not=20
> > > the right framework for commercial applications. It is slow,=20
> > > cumbersome, and requires significant hardware resources. If=20
> > > anyone wants to delve into the complexities of the code, they=20
> > > have to peel away multiple layers of typedefs and templated=20
> > > classes to try to evaluate implemented constructs and=20
> > > operations. If the end result in operation, size, and=20
> > > effectiveness were tremendous, the complexities could be=20
> > > overlooked. That does not seem to be the case.
> > >=20
> > > I would like to report more positive feedback. Can you help me? =20
> > > 1) For the MultResMIRegistration example, how can I identify=20
> > > parameters that will be stable for generic image sets? =20
> > > 2) How can I improve the time and memory requirements?
> > > 3) Can I tell the algorithm to skip the blank slices at the=20
> > > front and/or back of the data of interest?
> > > 4) Or, is this toolkit just not intended to be used in=20
> > > software used by medical personnel and others whose time is=20
> > critical?
> > >=20
> > > Any suggestions that might help me understand the toolkit=20
> > > better or how to make it effective would be greatly appreciated.
> > >=20
> > > Dave Rich
> > >=20
> > >=20
> > > >>> "Lydia Ng" <lng@insightful.com> 03/21/02 09:06PM >>>
> > > Hi David,=20
> > > =20
> > > First a disclaimer: the objective of the example is to show=20
> > > how ITK components and classes can be used for registration.=20
> > > As such it only offers a limited interface, allowing the user=20
> > > to set only a subset of the parameters.=20
> > > =20
> > > As you would appreciate ITK is still under construction as are=20
> > > the examples so there are at times some inconsistency in the=20
> > > documentation.=20
> > > =20
> > > The original example did allow the user to set the translation=20
> > > scale at each resolution. I took that feature off in an attempt=20
> > > to simplfy the example :-)=20
> > > (NB the actual class still allows you to do this).=20
> > > The documentation has been cleaned up in the last round=20
> of changes.=20
> > > =20
> > > The purpose of the "translation scale" is an attempt to=20
> > > address the difference in scale between the parameters that=20
> > > represents the rotation (ranging from 0 to 1) and the=20
> > > parameters that represents the translation (ranging from 0 to=20
> > > the size of the image). In the code only the=20
> > > translation parameters (the last 3) is scaled.=20
> > > =20
> > > For example, in the readme the target image is=20
> > > ( 256 x 256 x 26 ) pixels or ( 320 x 320 x 104 ) mm.
> > > My heuristic is to set the translation scale to
> > > the max dimension in mm (ie 320).
> > >=20
> > > The "starting shrink factors" is related to the resolution
> > > reduction.
> > >=20
> > > Again for the target image in the readme:
> > > - at the first level the reduction is (4,4,1) resulting=20
> in an image
> > > of ( 64 x 64 x 26 )pixels
> > > - at the second level the reduction is (2,2,1) =3D=20
> (128x128x26)pixels
> > > - at the third,fourth and fifth level reduction is (1,1,1)=20
> > =3D full size
> > > Again the actual class allows you to have a bit freedom in=20
> > > specifying the downsampling schedule.
> > >=20
> > > Hope this helps.
> > > - Lydia
> > >=20
> > > =20
> > > -----Original Message-----=20
> > > From: David Rich=20
> > > Sent: Thu 3/21/2002 2:01 PM=20
> > > To: insight-users@public.kitware.com=20
> > > Cc:=20
> > > Subject: [Insight-users] MultiResMIRegistration Example
> > >=20
> > >=20
> > > I am trying to test the MultiResMIRegistration example on a=20
> > > couple of images. However, I do not understand some of the=20
> > > parameters of the input file. In particular, line 14=20
> > > specifies the translation scale at each resolution level. =20
> > > These seemed to affect my output in a strange way, so I=20
> > > checked into the code:
> > > =20
> > >=20
> > > // set the translation scale
> > > itk::Point<double,7> scales;
> > > scales.Fill(1.0);
> > > for( int j =3D 4; j < 7; j++ )
> > > {
> > > scales[j] =3D 1.0 / vnl_math_sqr( m_TranslationScales[0] );
> > > }
> > > =20
> > > internalRegistrator->GetOptimizer()->GetTransform()->SetScale(
> > > scales );
> > > =20
> > >=20
> > > First, I don't understand why the code reads in 5 different=20
> > > scales and then only uses the first one. =20
> > > =20
> > > Second, it appears that the scaling fills 3 parameters, 4=20
> > through 6. =20
> > > =20
> > > I would appreciate it if somenone could explane what this is=20
> > > supposed to do. I can't find any other documentation on it.
> > > =20
> > > Also, input lines 10 and 11 read in the starting level shrink=20
> > > factor for each of the two images. Is this the term used for=20
> > > resolution reduction?
> > > =20
> > > Thanks for any help.
> > > =20
> > > Dave Rich
> > >=20
> > >=20
> >=20
>=20