[Insight-users] Why itk::LBFGSBOptimizer and itk::LBFGSOptimizer are classed as single-variant

[Ali -]

Stefan,

Your description clears up the ambiguity. Since the Levenberg-Marquadt and the quasi-Newton optimisers are comparable methods, the quasi-Newton method could be implemented to be a multi-valued multi-variate optimiser just like the Levenberg-Marquadt implementation is. Therefore, this is not an implementation issue of VNL or ITK, it's just that the implementation of quasi-Newton in LBFGC is not multi-valued comparing with the implementation of the Levenberg-Marquadt method in MINPACK, where VNL is just a wrapper for both of them.




Hi Ali,


The itk::LBFGS(B)Optimizer is single-VALUED and multi-VARIATE. Their
implementation is correct, both in itk and vnl. 


the optimisaton problem is:


min_x f(x)


f = the cost function to be minimised

x = the input arguments/parameters


single-valued: the cost function f is a scalar (dim(f)=1)

multiple-valued: the cost function f is a vector (dim(f)>1)


univariate: the input argument x is a scalar (dim(x)=1)

multivariate: the input argument x is a vector (dim(x)>1)


The LBFGS(B)Optimizer can handle: single-valued f, both univariate x and
multivariate x

This is correctly implemented in itk/vnl


The LBFGS(B)Optimizer can NOT handle: multiple-valued f. 

The itk implementation should therefore not be modified to accept
MultipleValuedCostFunctions.


Sincerely,

Stefan




At 14:40 7-11-2007, Ali - wrote:

I guess Stefan is right. There is
no such thing as vnl_multiple_valued_cost_function, hence,
itk::LBFGSBOptimizer and itk::LBFGSOptimizer shouldn't have been
classified as single-variant optimisers in ITK and they should be
modified for accepting itk::MultipleValuedCostFunction.




>Hi,


>The itk/vxl::LBFGS(B)Optimizer implementation IS multivariate. The
'multivariate' label refers to the number of variables (input arguments
of the cost function) over which is optimised. 


>the optimisaton problem is:


>min_x f(x)


> On the site
http://www.alglib.net/optimization/
 a multivariate optimisation method means that dim(x) >1 is
allowed.


> The LBFGSOptimizer is not suitable for MultipleValuedCostFucnction,
as far as I know, only for SingleValuedCostFunctions.


> singlevalued means: dim(f) = 1

> multiplevalued means: dim(f) > 1


> The vnl_cost_function indeed inherits from the vnl_unary_function,
but the vnl_unary_function is templated over a vector of doubles, which
contains the input argument x:


> class vnl_cost_function : public vnl_unary_function<double,
vnl_vector<double> >


> with kind regards,

> stefan




> 

> 

> 

> Hi Ali,

> 

> 

> 1) It is not a "problem", it is an "implementation
feature". :-)

> 

> 

> 

> 2) It is not in ITK, it is in VXL,

> 

> Please look at the implementation in the file:

> 

> Insight/Utilities/vxl/core/vnl/vnl_lbfgs.h

> 

> Where the optimizer takes as input a vnl_cost_function

> object, that derives from the vnl_unary_function class.

> 

> 

> 

> 3) It will be great to have multi-variate implementations

> of these optimizers. If you are motivated to write one,

> it will make a great contribution to the Insight Journal.

> 

> 

> 

> Regards,

> 

> 

> Luis

> 

> 

> 

> -------------

> Ali - wrote:

> > Hi,

> > 

> > The VNL wrappers itk::LBFGSBOptimizer and itk::LBFGSOptimizer
are 

> > classified as single-variant optimiser, while being based on
the 

> > quasi-Newton method, they are multi-variant optimisers by
nature -- see 

> >
http://www.alglib.net/optimization/
. Is this the ITK wrapper problem or 

> > a VNL issue?

> > 

> > _______________________________________________

> > Insight-users mailing list

> > Insight-users at itk.org

> > http://www.itk.org/mailman/listinfo/insight-users



Do you know a place like the back of your hand? Share local knowledge with BackOfMyHand.com 

_______________________________________________

Insight-users mailing list

Insight-users at itk.org

http://www.itk.org/mailman/listinfo/insight-users




_________________________________________________________________
Feel like a local wherever you go.
http://www.backofmyhand.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.kitware.com/pipermail/insight-users/attachments/20071109/e04fedd1/attachment.html