[Insight-users] Understanding OptimizerScales in its entirety

[Rupert Brooks]

Hi Levin,

Perhaps no one responded because understanding the optimizer scales in
their _entirety_ is a very tall order. :-) I'll take a shot at
answering the immediate question anyway.

The optimizer scales are, unfortunately, not consistently applied
across all the ITK optimizers.  However, VersorRigid3DOptimizer is a
subclass of RegularStepGradientDescentOptimizer and they both work the
same way.

In these optimizers, the gradient is divided by the scales.  Then the
step is this direction normalized to the step length.

This sounds simple but the effect is a bit counterintuitive.  This is
like scaling the transform parameters by the square roots of the
optimizer scale factors, and then limiting the step to a circle in the
original parameter space.  Which would be an ellipse in the new one.
Why square root? because you change the derivative by changing the
scales - and then consider it a direction in the original parameter
space.

I apologize in advance for self-promotion, but i just put an optimizer
on the Insight-Journal that may interest you, if you are digging into
this subject.  http://www.insight-journal.org/browse/publication/834
Different people have different theories about what the scales
accomplish - if you are up to some rather dry reading, i'll refer you
to Section 4.5 of my thesis
http://www.rupertbrooks.ca/downloads/Brooks_PhDThesis.pdf

And yes, different people have different heuristics for how to set
these scales.  In the thesis I argued that they should be chosen to
precondition the Hessian matrix of the cost function.  Others will
tell you they should roughly equalize the average pixel motion in the
image due to a unit shift of the parameters.  It turns out that these
are roughly the same thing.  Its important also to consider both how
the scales affect the optimizer path through parameter space, and how
they affect the stopping criteria.

Hope that helps a little,
Rupert


--------------------------------------------------------------
Rupert Brooks
rupert.brooks at gmail.com




On Thu, Aug 11, 2011 at 20:09, Wolf, Levin <Levin.Wolf at haw-hamburg.de> wrote:
>
> Hi everybody,
>
> i´m working on an 3D multisresolution Mutual Information registration and i would really like to understand the mechanism of the OptimizerScales in its entirety.
> I know, why I need them and how to use them. But I´m still not able to retrace the way of my scaling. How are they really working...
>
> In my case, I´m using:
>
> itk::MattesMutualInformationImageToImageMetric
> itk::VersorRigid3DTransform
> itk::VersorRigid3DTransformOptimizer
> itk::CenteredTransformInitializer
>
> and for example  the following settings:
>
>
> typedef OptimizerType::ScalesType       OptimizerScalesType;
>  OptimizerScalesType optimizerScales( transform->GetNumberOfParameters() );
>  const double translationScale = 1.0 / 2000.0;
>
>  optimizerScales[0] = 1.0;
>  optimizerScales[1] = 1.0;
>  optimizerScales[2] = 1.0;
>  optimizerScales[3] = translationScale;
>  optimizerScales[4] = translationScale;
>  optimizerScales[5] = translationScale;
>
>
>
>    if ( registration->GetCurrentLevel() == 0 )
>      {
>      optimizer->SetMaximumStepLength( 8.0 );
>      optimizer->SetMinimumStepLength( 1.0 );
>      }
>    else
>      {
>      optimizer->SetMaximumStepLength( optimizer->GetCurrentStepLength() / 2.0);
>      optimizer->SetMinimumStepLength( optimizer->GetMinimumStepLength() / 10.0 );
>      }
>
> What happens to the Scales??? Are they getting normalized somehow? Or is the steplength simply devided by the Scale?
>
> So how big would my first step be with these settings?
>
>
> I would be very grateful if someone could help me to understand the inside of the scaling.
>
>
>
> Thanks a lot
>
> Levin
>
>
>
>
>
> _____________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://www.kitware.com/products/protraining.html
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-users
>