[Insight-users] Testing Robustness of Registration

[Luis Ibanez]

Hi Isabelle,

The CenteredTransformInitializer will set up the


         * Center
         * Translation


of your transform.

The Center is not part of the set of Parameters,
while the Translation is. This means that when you
set the parameters of the transform, after getting
it from the initializer, you will modify its Translation
but not its center of rotation.



That being said...



     The test of robustness that you are referring to is
     *purely anecdotic* and it doesn't serve any useful
     purpose, other than *to fool the reviewers* who still
     participate in the masquerade of the peer-review system
     that is still practiced in our domain.



Here are some arguments on why such test of robustness
is useless:



1) The rigid transform has a parametric space of 6 Dimensions

    If the point of convergence of the registration is a point Q
    in that six-dimensional space, you are trying to illustrate
    how large is the region of capture around that 6D point.

    The assumption is that you can find all the points P such
    that by starting at P in the parametric space, the optimization
    will converge to a very close neighborhood of Q.

    The loci of all points P could then be considered to be the
    area of capture of the registration. The larger this are is,
    the more resilient the registration process will be to variations
    on the initialization conditions of the transform.

    If we look at the Metric as a cost function defined in that 6D
    space, you are looking for the watershed that is associated to
    the point of convergence Q.  The ideal way of finding the region
    of capture will be to find the watershed associated to the scalar
    value of the image metric in that 6d space.

    The assumption above has the flaw that optimizer are not continuous.

    They do not follow smooth continuous paths on the parametric space.
    Instead, most of the optimizers perform sequences of discrete steps
    that create broken paths on the parametric space.  Even when the
    optimizer is initialized in a point that is inside the capture
    region, the optimizer settings such as step length, and number
    of iterations, may lead to early termination of the optimization
    before the path reaches a neighborhood of the point Q.

    You may find interesting to look at the diagram in the ITK Software
    Guide,

               http://www.itk.org/ItkSoftwareGuide.pdf

    that shows how the optimizer parameters and the metric parameters
    result in very different paths on the transform parametric space.

    See for example figure 8.15 in pdf-page 376.  See also the metric
    plots, and the plots of translations in the parametric space,
    that are shown for most of the registration in that same chapter.


    For an illustration of the notion of the capture region in the
    metric cost function, you may want to look at the figure 8.46.

    As opposed to all the "peer-reviewed" publications, in the ITK
    Software Guide you will find the actual code, images and full set
    of parameters that were used for generating these plots.

    You will also find the instructions for downloading the scripts
    that were used for generating the GNUPlot diagrams shown in this
    chapter.


    Note that the plot in figure 8.46 only explores 2 out of the 6
    dimensions that you will have to deal with, in a 3D rigid transform.

    This plot is a discretization of that space, (e.g. about 100 x 100 ).
    The equivalent plot in a 6D space will require you to do 100^6
    samples, that if you store in float numbers will be an 6D image of
    400 Megabytes. In order to find the value for each pixel of such
    6D image you will need to compute the metric of the two 3D images
    for the transform parameters associated to the 6D pixel.


    Even if you compute such image, and then run inside it a watershed
    from the point Q. That still doesn't guarantee that starting from
    any given point P inside the watershed will result in a convergence
    to the point Q.  The specific settings of the optimizer parameters
    may be or may not be appropriate for producing such discretized path.



2) In practice you are suggesting to have a very coarse sampling of
    the capture region by providing *some* paths, and based on the
    length of those paths, *induce* that the registration has a certain
    "degree of robustness".


    This is an interesting but still *ANECDOTIC* piece of information.


    It is as useful as to tell you that person X has travel through the
    Amazonian forest 27 times and has never been bitten by a snake.

    That doesn't mean much, regarding the chances of person Y to make
    a new trip in the Amazonian forest and being bitten by a snake.

    It certainly provides some degree of *psychological comfort* to be
    able to report that you have run the registration under a variety
    of perturbed conditions and yet achieved convergence. But, that's
    the only thing it is.... "psychological comfort". Because the number
    of perturbed conditions that you will be able to explore will be
    infinitesimally small compared to the number of potential paths
    in the 6D space that surrounds point Q in the parametric space.


    It is still a piece of information that will looks nice in typical
    decadent journal, and it will be comic to see reviewers buying into
    it. Since they never perform the reproduction of the experiments,
    they won't realize how useless and insignificant such measure
    of robustness will be.


    The measure may only start being significant if you manage to
    perform a dense-enough sampling of the 6D parametric space, and
    to mathematically evaluate the coverage of such sampling.

    E.g.
    A sampling that covers the equivalent of 60% of the parametric
    space at a range of  +/- 100mm of translations and +/- 10 degrees
    of otation.


     Now,... even if you manage to cover a large fraction of the
     parametric space you would have done so with only

                    *ONE SPECIFIC PAIR of IMAGES*

     as the input of the image metric.  The result can't hardly be
     extrapolated to *other images*, e.g. image with bias field,
     with other dynamic ranges of intensity, with other pixel spacing,
     with different levels of noise.




3) As Richard Feynman put it:


           "It is very hard to actually *know* something"


    Our imaging community is *very lax* when it comes to differentiate
    the *appearance* of knowledge from the actual *posession* of
    knowledge.




     Please don't be fooled by the many things that you see
     in Journals and Conference papers. Keep in mind that
     most of it has been published with the sole purpose of
     providing material for academic promotions and to fill-up
     established yearly quotas for intellectual production.


     As a reader, and actual practitioner of the imaging arts
     you ought to be *more critical* and exigent when it comes
     to the deduce / induce information from what other report
     in venues that do not require to demonstrate reproducibility.





    Regards,



       Luis



--------------------
IsabelleNg wrote:
> Thank you Karthik for your reply.
> 
> I just realize that I omitted some info. I am using an centered-initializer
> to initialize the transform (rigid 3d). In this case, would the following be
> a valid sequence?
> 
> rigidTransform->SetIdentity();
> resampler->SetTransform( randomXform );
> initializer->SetMovingImage( resample->Getoutput() );
> initializer->SetTransform( rigidTransform );
> 
> registration->SetInitialTransformParameters( rigidTransform->GetParameters()
> )
> 
> Thanks again,
> Isabelle
> 
> 
> 
> 
> 
> Karthik Krishnan-2 wrote:
> 
>>On 5/4/07, IsabelleNg <isabelleNg at homeworking.org> wrote:
>>
>>>
>>>ITK-users,
>>>
>>>I wish to test one of the registration algorithm by applying random
>>>transformations to the moving image. This is often done in papers that
>>>report registration results as tests of robustness and capture range. Is
>>>it
>>>valid to perform these tests by simply initializing the transform with
>>>random numbers? i..e by calling
>>>
>>>registration->SetInitialTransformationParameters( randomXform)?
>>
>>
>>Hi Isabelle
>>
>>Yes. It is identical.
>>
>>Or, do we actually need to physically write out the randomly misaligned
>>
>>>images and then feed back into the algorithm?
>>>
>>>How would results differ with these 2 approaches?
>>
>>
>>There wouldn't be a difference. There would be a marginal difference
>>depending on the transform used to resample the moving image before
>>writing
>>to the disk. In the first case, the initial transform is the same as the
>>one
>>used for registration.
>>
>>
>>-- 
>>Karthik Krishnan
>>R&D Engineer,
>>Kitware Inc.
>>
>>_______________________________________________
>>Insight-users mailing list
>>Insight-users at itk.org
>>http://www.itk.org/mailman/listinfo/insight-users
>>
>>
> 
>