[Insight-users] Literature about itk::RegularStepGradientDescentOptimizer ?

[Luis Ibanez]

Hi Motes

It is simply a gradient descent optimizer in which the
advancement step is updated at every step based on
whether the optimizer changed direction or not in the
parametric space.

The name "RegularStep" itself is something we made
up in the early development stages of the project  in
order to differentiate this optimizer from the other
"GradientDescent" which uses the gradient magnitude
in order to compute the step length via a multiplier
(learning rate).


If you simply search for Gradient Descent optimizers
and any scheme for computing the step length of such
optimizers you probably will find something that can
be used as a reference.


Are you looking for a citation to use in a paper ?

or are you looking for background to help you
understand how the optimizer works ?


    Regards


            Luis


---------------------------------------------------------------------------------------
On Tue, Nov 10, 2009 at 4:43 PM, motes motes <mort.motes at gmail.com> wrote:
> Where do I find articles, textbooks that describe the math behind
> itk::RegularStepGradientDescentOptimizer. When I google the term
> "Regular Step Gradient Descent Optimizer" I almost only get references
> to messages on the itk mailing list.
>
> I have the book: Numerical Optimization, Second Edition, Mocedal:
>
> http://www.amazon.com/Numerical-Optimization-Operations-Financial-Engineering/dp/0387303030/ref=sr_1_1?ie=UTF8&s=books&qid=1257889335&sr=8-1
>
> But I cannot find anything about the Regular Step Gradient Descent
> Optimizer. Any hints are much appreciated!
> _____________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://www.kitware.com/products/protraining.html
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-users
>