[Insight-developers] Scales estimation for registration

Sun Jun 5 00:21:05 EDT 2011

Dear All,

I just updated a patch about optimizer parameter estimation. The main 
changes include a Quasi-Newton method and Stefan et al.'s adaptive step.

As discussed in previous emails, scales estimation depends on the transform 
object and the images, while learning rate depends on the derivative from 
the metric object. In the tentative approach here, I put scales estimation 
in the optimizer's StartOptimization(), and learning rate in its 
AdvanceOneStep() .

Here is a brief description about the new classes:

- itkOptimizerParameterEstimatorTest: the main program of testing.

- itkOptimizerHelper: it is separated since the transform and image 
information are not available in the optimizer. It provides an optimizer 
with two virutual functions: EstimateScales and ComputeMaximumVoxelShift.
- itkOptimizerParameterEstimator: an actual implementation of 
itkOptimizerHelper.

- itkAutomaticGradientDescentOptimizer: an optimizer with simple estimation 
of scales and learning rate.
- itkQuasiNewtonOptimizer: a child class of 
itkAutomaticGradientDescentOptimizer, using a Quasi-Newton method to 
estimate the step sizes. It works fast with the testing data.
- itkAdaptiveGradientDescentOptimizer: a child class of 
itkAutomaticGradientDescentOptimizer, using a simplified version of the 
adaptive method.

The code might be reorganized in some better way. For example, scales 
estimation might be moved to ImageRegistrationMethod::Initialize(). But I 
hope to receive your feedbacks before further changes.

Thank you very much!

Bao

----- Original Message ----- 
From: Bill Lorensen
To: Baohua Wu
Cc: brian avants ; Nicholas Tustison ; Stefan Klein ; Marius Staring 
(registration) ; Stefan Klein ; Insight Developers
Sent: Monday, May 09, 2011 5:31 PM
Subject: Re: [Insight-developers] Scales estimation for registration

My concern is the estimation of the translation scales. It seems the global 
scale factor is used to scale the "estimated scales". I guess, indirectly, 
it interacts with the learning rate. So it is also of concern.

On Mon, May 9, 2011 at 1:42 PM, Baohua Wu <baowu99 at gmail.com> wrote:

hi brian,

thank you for the information.

to my knowledge, the test was passed with the synthetic data, and the scales 
worked well with several real images. i think bill's conern was mainly about 
how to set the global factor (or learning rate).

we have been trying to estimate this global factor as well. some strategies 
include the norm of gradient, the parameter space, etc.

baohua

----- Original Message ----- From: "brian avants" <stnava at gmail.com>
To: "Baohua Wu" <baowu99 at gmail.com>; "Nicholas Tustison" 
<ntustison at gmail.com>
Cc: "Stefan Klein" <s.klein at erasmusmc.nl>; "Bill Lorensen" 
<bill.lorensen at gmail.com>; "Marius Staring (registration)" 
<M.Staring at lumc.nl>; "Stefan Klein" <stefan at isi.uu.nl>; "Insight Developers" 
<insight-developers at itk.org>
Sent: Monday, May 09, 2011 10:05 AM

Subject: Re: [Insight-developers] Scales estimation for registration

hi baohua

nick may have some code that is relevant.

also, you may want to let people know about your other strategies for
estimating the parameter scales.  for instance,  the methods that take
into account the norm of the gradient itself, as well as as that build
estimates of the parameter space.

brian

On Mon, May 9, 2011 at 9:24 AM, Baohua Wu <baowu99 at gmail.com> wrote:

Stefan,

Thank you for the advice. I am glad to know your method works for scales
and learning rate. I will try it .

Baohua

----- Original Message ----- From: "Stefan Klein" <s.klein at erasmusmc.nl>
To: "Bill Lorensen" <bill.lorensen at gmail.com>
Cc: "Marius Staring (registration)" <M.Staring at lumc.nl>;
<baowu99 at gmail.com>; "Stefan Klein" <stefan at isi.uu.nl>; "Insight Developers"
<insight-developers at itk.org>
Sent: Monday, May 09, 2011 7:43 AM
Subject: Re: [Insight-developers] Scales estimation for registration

Hi Bill,

The method in elastix works well in my experience.

The method is explained here:
http://dx.doi.org/10.1007/s11263-008-0168-y

Equation 61/62
But that final expression should be squared because of the definition of
the scales in ITK.

The method needs to know the fixed image domain and the transform. So it
would fit in the same framework as Bao's class. The methods declared at
lines 147 to 161 should probably be protected by the way?

Just to clarify: the AutomaticScalesEstimation in elastix only computes
the scales, not the global LearningRate. I guess the GlobalScalingFactor
serves to determine the global LearningRate?

To automatically determine an optimal global learning rate, you would need
the CostFunction as well. (since multiplying the costfunction by 10 would
cause the optimal learning rate to be divided by 10, for example).

Good idea to make this a separate class!
Kind regards,
Stefan

On 06/05/2011 22:52, Bill Lorensen wrote:

Marius, Stefan and Bao,

One of the major weaknesses in ITK's registration framework is the
estimation of parameters. In particular, the estimation of the scales
seems to require special magic. My experience has been that the
selection of a proper scaling for translation is particularly
difficult. ITK traditionally (and currently) leaves the scaling
decision up to the application. There are currently two possible
approaches to this estimation.

1) Bao's itkParameterScaleEstimator proposed in this gerrit topic:
http://review.source.kitware.com/#change,1499
2) Elastix's (http://elastix.isi.uu.nl/) AutomaticScalesEstimation
method implementation in elxTransformBase.

For Marius and Stefan I have questions:

a) How successful is elastix's AutomaticScalesEstimation?
b) Currently this is implemented as a method in
elxTransformBase. Could it be implemented in an API similar to Bao's
itkParameterScaleEstimator?

For Bao's implementation, my concern is the addition of a free
parameter GlobalScalingFactor. It is not clear how to pick a proper
value. To me there is as much mystery in this parameter as there is in
picking a proper translation scaling

Bill

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.html

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://www.itk.org/mailman/listinfo/insight-developers

--
Stefan Klein
+31 10 7043049
http://www.bigr.nl/people/StefanKlein 