[Insight-users] Regular Step Gradient Descent optimizer
Ming Chao
mingchao2005 at gmail.com
Tue May 2 19:32:20 EDT 2006
HI Luis,
Thanks for your prompt reply. Yes, I did set the optimizer (Regular Step
Gradient Descent optimizer) as the following way which I forgot to include
in last posting.
optimizer->SetMaximumStepLength( 0.50000 );
optimizer->SetMinimumStepLength( 0.01 );
optimizer->SetNumberOfIterations( 200 );
optimizer->SetGradientMagnitudeTolerance(
0.01*optimizer->GetGradientMagnitudeTolerance()
);
Later on I tried to increase the step lengths as:
optimizer->SetMaximumStepLength( 1.000 );
optimizer->SetMinimumStepLength( 0.05 );
And I had the following metric output:
0 -0.835581 [-0.0321978, -0.466639, 0.883861]
1 -0.840282 [0.399061, -0.885096, 1.68318]
2 -0.837371 [0.259052, -0.883767, 2.67333]
3 -0.824043 [0.75875, -0.889684, 2.68965]
4 -0.829629 [0.55809, -0.881664, 2.83855]
5 -0.82694 [0.679164, -0.884414, 2.80759]
6 -0.828347 [0.623941, -0.882636, 2.8368]
7 -0.827662 [0.566916, -0.881064, 2.86234]
The only change is the step length but the results are similar to the
previous one. By the way for the LBFGSB optimizer I used the following
conditions:
// (1) LBFGSB optimizer
OptimizerType::BoundSelectionType boundSelect(
transform->GetNumberOfParameters() );
OptimizerType::BoundValueType upperBound(
transform->GetNumberOfParameters() );
OptimizerType::BoundValueType lowerBound(
transform->GetNumberOfParameters() );
boundSelect.Fill( 0 );
upperBound.Fill( 10.0 );
lowerBound.Fill( -10.0 );
optimizer->SetBoundSelection( boundSelect );
optimizer->SetUpperBound( upperBound );
optimizer->SetLowerBound( lowerBound );
optimizer->SetMaximumNumberOfEvaluations( 200 );
optimizer->SetMaximumNumberOfCorrections( 200 );
Here I am not clear what you meant for best setting. I thought the
conditions I provided were reasonable. Do you see anything obviously
different which leads to different results?
Cheers,
Ming
On 5/2/06, Luis Ibanez <luis.ibanez at kitware.com> wrote:
>
>
> Hi Ming,
>
>
> Did you set the optimizer to do Minimization or Maximization ?
> that is, did you used any of the following:
>
>
> optimizer->MaximizeOn() ?
> optimizer->MaximizeOff() ?
> optimizer->MinimizeOn() ?
> optimizer->MinimizeOff() ?
>
> How much overlap did the images have at the end of the run
> with the RegularStepNormalized Correlation ?
>
> Note that the lasts iterations of the run with
> RegularStepGradientDescent are advancing at very
> small steps.
>
> You may want to start the optimizer with a larger initial
> value of the StepLength, and to change the default relaxation
> factor to be 0.7 or 0.9 instead of the default value 0.5.
>
> In this way, the step length will change to 0.7 of the previous
> value every time that the gradient changes directions.
>
> You will get a lot of insight about the registration process
> by plotting the trace of the translation in 3D. This will
> show you how much the Transform is changing at every iteration.
> This is not clearly conveyed just by looking at the numbers.
>
> You will find the 2D version of many of these types of plots
> in the ITK Software Guide,
>
> http://www.itk.org/ItkSoftwareGuide.pdf
>
>
> Note that it is not surprising that you get different
> results from different optimizers, but before you attempt
> to compare the results you should make sure that you are
> actually using both optimizers in their best settings and
> conditions. Otherwise it is just a bias and unfair
> comparison.
>
>
>
> Regards,
>
>
> Luis
>
>
>
> =================
> Ming Chao wrote:
> > Hi,
> >
> > When I used Regular Step Gradient Descent optimizer to register two
> > images I saw an abnormal behavior. The metric value first decreased but
> > after some iterations it became larger. See the following output:
> >
> > 0 -0.835581 [-0.00321978, -0.0466639, 0.0883861]
> > 1 -0.836504 [-0.00422488, -0.0933436, 0.176817]
> > 2 -0.83737 [-0.00293453, -0.139983, 0.265265]
> > 3 -0.838174 [0.000731481, -0.18652, 0.353701]
> > 4 -0.838908 [0.00685175, -0.232884, 0.442092]
> > 5 -0.839563 [0.015502, -0.278993, 0.530404]
> > 6 -0.840129 [0.0267537, -0.32476, 0.618602]
> > 7 -0.840598 [0.0406717, -0.370081, 0.706649]
> > 8 -0.84096 [ 0.0573129, -0.414844, 0.794509]
> > 9 -0.84121 [0.0767229, -0.458919, 0.882148]
> > 10 -0.841343 [0.0989338, -0.50216, 0.969537]
> > 11 -0.841358 [0.123961, -0.544401, 1.05665]
> > 12 -0.841256 [0.151799, -0.58545 , 1.14349]
> > 13 -0.841041 [0.182419, -0.625084, 1.23004]
> > 14 -0.840719 [0.215761, -0.663034, 1.31634]
> > 15 -0.840298 [0.214669, -0.680096, 1.41487]
> > 16 -0.839375 [0.216594, -0.696938, 1.51342]
> > 17 - 0.838339 [0.221686, -0.713544, 1.6119]
> > 18 -0.837215 [0.230089, -0.729893, 1.7102]
> > 19 -0.836032 [0.241937, -0.74596, 1.80819]
> > 20 -0.83482 [0.257347, -0.76171, 1.90573]
> > 21 -0.833613 [0.276409, - 0.7771, 2.00268]
> > 22 -0.832445 [0.299184, -0.792076, 2.09889]
> > 23 -0.831349 [0.325688, -0.806571, 2.19422]
> > 24 -0.830357 [0.35589, -0.820502, 2.28853]
> > 25 -0.829494 [0.389705, -0.833768, 2.3817]
> > 26 -0.828783 [0.426981, -0.84624, 2.47365]
> > 27 -0.828236 [0.467495, -0.857749, 2.56435]
> > 28 -0.827859 [0.510938, -0.868055, 2.65383]
> > 29 -0.827633 [0.556896, -0.876774, 2.74221]
> > 30 -0.827525 [ 0.60475, -0.883034, 2.8298]
> > 31 -0.827502 [0.650812, -0.875321, 2.91822]
> > 32 -0.827512 [0.629235, -0.881114, 2.87349]
> > 33 -0.827519 [0.603546, -0.883645, 2.83067]
> > 34 -0.827483 [0.620407, -0.880267 , 2.84881]
> > 35 -0.827564 [0.608674, -0.882388, 2.84506]
> > 36 -0.827459 [0.613038, -0.88246, 2.84059]
> > 37 -0.827529 [0.610107, -0.882339, 2.84166]
> > 38 -0.827493 [0.611332, -0.882386, 2.8407]
> >
> > However, if I change the optimizer to the LBFGSB optimizer, I got the
> > following output:
> >
> > 0 -0.840282 [-0.0321978, -0.466639, 0.883861]
> > 1 -0.840732 [0.0493198, -0.53362, 1.01199]
> > 2 -0.840735 [0.0592661, -0.540774, 1.03432]
> > 3 -0.840751 [0.100359, -0.570829, 1.11035]
> >
> > This looks reasonable. The setup for the registration is the following:
> >
> >
> > typedef itk::BSplineInterpolateImageFunction<ImageType, double >
> > InterpolatorType;
> > typedef itk::ImageRegistrationMethod<ImageType, ImageType >
> > RegistrationType;
> >
> > typedef
> > itk::NormalizedCorrelationImageToImageMetric<ImageType,ImageType >
> > MetricType;
> >
> > typedef itk::TranslationTransform< double, Dimension >
> > TransformType;
> >
> > Can anybody tell me why I have so different results with different
> > optimizers?
> >
> > Thanks,
> >
> > Ming
> >
> >
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Insight-users mailing list
> > Insight-users at itk.org
> > http://www.itk.org/mailman/listinfo/insight-users
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.kitware.com/pipermail/insight-users/attachments/20060502/86bd0fd6/attachment.html
More information about the Insight-users
mailing list