[Insight-users] SPSA Optimizer
Stefan Klein
stefan at isi.uu.nl
Thu Mar 24 05:08:36 EST 2005
Hi Mathieu, Daniel and others,
I put together a new SPSA optimizer, based on the three version that we
had. I also added a Test.
You may download the zip from:
http://www.isi.uu.nl/People/Stefan/
> - what should be the default values for a, A, C?
> - I used a=A=c=0.1 (Don't trust my implementation...)
> - Mathieu uses a=01.6, A = 60(!), C=0.01
> - Stefan uses a=A=c=1.0
> - Are we OK with Alpha = 0.602, Gamma = 0.101?
alpha = 0.602 and gamma = 0.101, as J. Spall suggests.
c depends on the amount of noise in your problem and the range of your
parameters, so could by anything. So, 1.0 seemed the most appropriate to me
as a default.
a depends on your specific problem as well, so is set to 1.0 by default.
A is set to 10.0, which is 10% of the default MaximumNumberOfIterations.
The Scales thing gave me quite a headache but I think I did it correct now.
In the .cxx I tried to explain why I did it like i did...
Please let me know what you think about it!
Regards,
Stefan.
At 10:04 18/03/05, Mathieu De Craene wrote:
>Le jeudi 17 mars 2005 à 09:10 -0500, Blezek, Daniel J (Research) a
>écrit :
> > Stefan, Marius, Mathieu,
> >
> > Excellent. I liked the try / catch blocks, and the stopping
> > criteria in Mathieu's implementation. Essentially, the code is the
> > same across all 3 implementations... As you would expect from the
> > nicely written paper.
> >
> > - what should be the default values for a, A, C?
> > - I used a=A=c=0.1 (Don't trust my implementation...)
> > - Mathieu uses a=01.6, A = 60(!), C=0.01
>
>
>In fact, this value should be changed to 0.5 or 1. I used a Set() method
>in the main code to change the default value. For rigid registration for
>instance, I doubt you will measure anything by changing the translations
>coefficients by 0.01 mm ;-)
>
>If the A coefficient is big, it means that the optimizer will make big
>steps at the initial iterations.
>An idea of the successive trials made by the spsa optimizer in rigid
>registration can be seen on this animation :
>http://www.tele.ucl.ac.be/~decraene/anims/RIGID.mpeg
>
>I think it is good to keep this parameter big enough if we want to keep
>the robust behavior of this optimizer.
>
> > - Stefan uses a=A=c=1.0
> > - Are we OK with Alpha = 0.602, Gamma = 0.101?
> >
>
>I have never experimented other set of values but these work fine for
>me !
>
> >
> > -dan
> > -----Original Message-----
> > From: Stefan Klein [mailto:stefan at isi.uu.nl]
> > Sent: Thursday, March 17, 2005 8:55 AM
> > To: Blezek, Daniel J (Research); Mathieu De Craene
> > Cc: Insight-users at itk.org; insight-developers at itk.org
> > Subject: RE: [Insight-users] some contributions
> >
> >
> > Eh, but then we'll have 2 SPSA-optimizers....
> > Wouldn't it be the best to have one that combines the good of
> > all 3 versions?
> >
> > plan:
> > - we include the scales thing and the test and some elements
> > from Mathieu's implementation
> > - we'll mail it to the list.
> > - you check it (if we combined all features correctly) and
> > correct it
> > - Daniel commits it to the itk.
> >
> > Marius&Stefan.
> >
> >
> >
> > At 08:39 17/03/05, Blezek, Daniel J (Research) wrote:
> > > Oddly enough, I also have an implementation of the SPSA
> > > algorithm! I offered to put Stefan's implementation in,
> > > provided he account for m_Scale and write a test. I'll make
> > > the same offer for you, Mathieu. I've put my code on the
> > > itk Wiki @ http://itk.org/Wiki/SPSAOptimizer however, I
> > > haven't tested my code very well, as it's new.
> > >
> > > -dan
> > >
> > > -----Original Message-----
> > > From: insight-users-bounces at itk.org
> > > [mailto:insight-users-bounces at itk.org]On Behalf Of Mathieu
> > > De Craene
> > > Sent: Thursday, March 17, 2005 8:20 AM
> > > To: Stefan Klein
> > > Cc: Insight-users at itk.org; insight-developers at itk.org
> > > Subject: Re: [Insight-users] some contributions
> > >
> > >
> > > Le mercredi 16 mars 2005 à 21:34 +0100, Stefan Klein a
> > > écrit :
> > > > Hi all,
> > > >
> > > > We have created some classes that are useful for us. We
> > > hope also for other
> > > > people, so we would like to contribute them. The code can
> > > be found for
> > > > download on:
> > > > http://www.isi.uu.nl/People/Stefan/
> > > >
> > > > - itkEulerTransform.h: This class is a wrap around the
> > > Euler2DTransform and
> > > > the Euler3DTransform. In this way it is templated over the
> > > dimension, like
> > > > other Transform-classes.
> > > >
> > > > - SimultaneousPerturbation.zip: This class implements the
> > > so-called
> > > > Simultaneous Perturbation method as an ITK optimizer. For
> > > more info on this
> > > > optimization method, go to:
> > > > http://www.jhuapl.edu/SPSA
> > > >
> > >
> > > I have also have some code for SPSA optimization. I found it
> > > quite
> > > useful for rigid registration (it seems to be pretty
> > > robust). The source
> > > code for a rigid registration application based on ITK is
> > > available on
> > >
> > > http://euterpe.tele.ucl.ac.be/Waleo2/vesale/
> > >
> > > I would like to compare our implementations of this
> > > optimizer.
> > >
> > > Regards,
> > >
> > > Mathieu.
> > >
> > > > - FullSearch.zip: This class implements a (semi-)Full
> > > Search optimization
> > > > routine. This can be useful for research, to scan the
> > > optimization surface,
> > > > or to evaluate the result of other optimizers. It allows
> > > the user to
> > > > specify which parameters should be evaluated, in which
> > > range.
> > > >
> > > > Since we have contributed 3 things now, we also have a
> > > feature request; as
> > > > a reward :-)
> > > >
> > > > Would it be possible to add a public function to the
> > > > MattesMutualInformationImageToImageMetric that forces the
> > > metric to select
> > > > new spatial samples? This would be useful if you would
> > > want new spatial
> > > > samples every iteration for example.
> > > >
> > > > We hope that the contributions are useful. Any comments
> > > are welcome!!
> > > >
> > > > Marius and Stefan.
> > > >
> > > >
> > > > Marius Staring, Stefan Klein
> > > > Image Sciences Institute
> > > > University Medical Centre Utrecht
> > > > Heidelberglaan 100, 3584 CX Utrecht, The Netherlands
> > > > phone: +31 (0)30 250 3186, fax: +31 (0)30 251 3399
> > > > {marius,stefan}@isi.uu.nl,
> > > > http://www.isi.uu.nl/People/Marius
> > > > http://www.isi.uu.nl/People/?stefan
> > > >
> > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.kitware.com/pipermail/insight-users/attachments/20050324/846001ad/attachment-0001.html
More information about the Insight-users
mailing list