[Insight-users] Question about Diffeomorphic Registration : GENERALIZATION == SUPERSTITION

Mon Jul 14 10:48:35 EDT 2008

Hi Kaven,

                    Thanks for your detailed email.

Before you go further,

Please avoid making the mistake of arriving to conclusions such as:

      "However, Thirion got much better registration result
       with lower residual than Diff. Demon."

This anecdotic description *only* applies to the pair of image that
you tried, and for the particular combinations of parameters that
you used for running each case.

It is *extremely* misleading to state that method A is "better" than
method B. This only serve the vanity purpose of running contests among
labs, and doesn't help to promote the progress of the field.

Note for example the misleading generalization at which you arrive
from the statement:

       "In addition, no matter how small step size is,
        in the Multi-resolution 1 and 2 (in Diff. Demon)
        it increases MSE instead of decreasing it."

Obviously you didn't tried *all possible* step sizes. You probably ran
a set of step sizes values, 5 of them ?, 10 ?,  50 ? 100 ? until you
got convinced that for that set, the result were consistent. The jump
from a finite set of observations, to a full generalization is a very
dangerous thing to do, and again, it is misleading for the community
to claim that you have somehow demonstrated such statement.

When you describe an experiment, you should *JUST* describe *the*
experiment. It shouldn't be contaminated with the half-cooked
conclusions, and lateral generalizations, that will soon become
superstitions.

Your description should have been something like:

for the following values of step size: {1.0, 0.8, 0.6 and 0.4} the
resulting values of MSE were = { 100, 148,159....}. A table of values,
or a plot of the values would have been an appropriate way of presenting
this information. The description of an experiment should only contain
information about reproducible *FACTS*.

The astronomical jump that you are doing to a generalization can not
be supported by a finite number of experimental observations.

This is a very common mistake in our field, and one of the is
perpetuated by our primitive approach to Journals and Conferences,
where the main goal is to demonstrate "innovation", and to claim
glory by pretending to be better than others.

There is no room for such retrograde practices in the ITK community.

When you describe your results to this list, please focus on the
facts, and provide the means for *replicating* your observations.

-----

As you certainly know, each one of the methods that you are running
has a relevant set of parameters including at least 10 numbers.

It is unlikely that you have exhaustively explored all possible
combination of these sets of parameters, and to do it with a population
of image large enough to support any generalization, or even the
hypothesis of a generalization.

------

This community practices Open Science, please join us on the effort for
dissipating the medieval darkness, and resist the temptation of looking
as the jury of a reality-TV contest for the "best" registration method.

The "DemonsRegistration" code that you are running, is it one of the
examples in the Insight/Examples/Registration ?

If not, could you please provide the source code ?

Please include a pointer to the "traditional C code" that you cite
as the benchmark.

Please make sure that you include in your description, *ALL* the
parameters needed to run the experiments that you report.

Here, we *DO* reproduce experiments.

     Regards,

         Luis

--------------------
Kaveh Kohan wrote:
> Hi Everybody,
> 
> I am a newbie to ITK community. I have couple of questions and I would 
> be thankful if anybody answers:
> 
> - Is there any fluid registration method implemented in ITK, either 
> LDDMM (Prof. Miller et al.) or fast diffeomorphism (Prof. Ashburner) ? 
> If not, is there any plan to implement such methods?
> 
> - As far as I know (so far) this paper and corresponding code is the 
> only diffeomorphic registration method implemented in ITK:
> 
> "Diffeomorphic Demons Using ITK's Finite Difference Solver Hierarchy"
> by: Tom Vercauteren 
> <http://www.insight-journal.org/midas/search.php?authorfirst=Tom&authorlast=Vercauteren>, 
> Xavier Pennec 
> <http://www.insight-journal.org/midas/search.php?authorfirst=Xavier&authorlast=Pennec>, 
> Aymeric Perchant 
> <http://www.insight-journal.org/midas/search.php?authorfirst=Aymeric&authorlast=Perchant>, 
> Nicholas Ayache 
> <http://www.insight-journal.org/midas/search.php?authorfirst=Nicholas&authorlast=Ayache>
> Jun-2007
> 
> I found it very interesting and tried it over traditional "C" example 
> which is commonly used as a bench mark for fluid methods. My 
> understanding is that this is not a large deformation method. I got 
> following results after trying many parameters and different update 
> strategies:
> 
> Fixed image:
> http://www.4shared.com/file/55079305/f6267afa/fixed.html
> Moving Image:
> http://www.4shared.com/file/55079348/ecfbc343/moving.html
> 
> --------------------
> Diffeomorphic Demon update:
> ../DemonsRegistration -f fixed.nii -m moving.mha -a 0 -o Out_Diff.mha -v 
> -i 200x4x4 -l6
> Result:
> http://www.4shared.com/file/55082051/db2be42f/Out_Diff.html
> ------------------
> Basic ITK Update:
>  ../DemonsRegistration -f fixed.nii -m moving.mha -a 1 -o Out_ITK.mha -v 
> -i 300x30x30 -l 5
> Result:
> http://www.4shared.com/file/55082650/a8a1a80b/Out_ITK.html
> -------------------
> Thirion Update:
> ../DemonsRegistration -f fixed.nii -m moving.mha -a 2 -o Out_Thr.mha -v 
> -i 300x30x30 -l 10
> Result:
> http://www.4shared.com/file/55080214/66a0c941/Out_Thr.html
> 
> to see warped grids and determinant of Jacobians for each method, please 
> visit:
> http://www.4shared.com/dir/8238152/7295688e/Registration_Results.html
> 
> Unlike basic ITK which produce negative Jacobian (but good 
> registration), both Diff. Demon and Thirion methods yiled non-negative 
> Jacobian. However, Thirion got much better registration result with 
> lower residual than Diff. Demon. In addition, no matter how small step 
> size is, in the Multi-resolution 1 and 2 (in Diff. Demon) it increases 
> MSE instead of decreasing it. I know that the step size I used for Diff. 
> Demon is very aggresive but even with default step size it increases MSE 
> in the Multi-res 1& 2.
> 
> I was wondering whether I am making a mistake or in general Thirion 
> update works better while procuding diffeomorphic transformation? I 
> mean, I think the advange of Diff. Demon is that it gaurantees to 
> produce Diffeomorphic transformation but does it necessarily sacrify 
> accuracy? or I am making a mistake somewhere? Do you think that combined 
> method (sometimes Thirion sometimes Diff. Demon) works better? Or this 
> method is not essentially proper for large deformation?
> 
> Any comment is appreciated.
> 
> Sincerely,
> Kaveh
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Insight-users mailing list
> Insight-users at itk.org
> http://www.itk.org/mailman/listinfo/insight-users