[Insight-users] Why itk::LBFGSBOptimizer and itk::LBFGSOptimizer are classed as single-variant o

[Stefan Klein]

Hi Ali,

The itk::LBFGS(B)Optimizer is single-VALUED and multi-VARIATE. Their 
implementation is correct, both in itk and vnl.

the optimisaton problem is:

min_x f(x)

f = the cost function to be minimised
x = the input arguments/parameters

single-valued: the cost function f is a scalar (dim(f)=1)
multiple-valued: the cost function f is a vector (dim(f)>1)

univariate: the input argument x is a scalar (dim(x)=1)
multivariate: the input argument x is a vector (dim(x)>1)

The LBFGS(B)Optimizer can handle: single-valued f, both univariate x and 
multivariate x
This is correctly implemented in itk/vnl

The LBFGS(B)Optimizer can NOT handle: multiple-valued f.
The itk implementation should therefore not be modified to accept 
MultipleValuedCostFunctions.

Sincerely,
Stefan


At 14:40 7-11-2007, Ali - wrote:
>I guess Stefan is right. There is no such thing as 
>vnl_multiple_valued_cost_function, hence, itk::LBFGSBOptimizer and 
>itk::LBFGSOptimizer shouldn't have been classified as single-variant 
>optimisers in ITK and they should be modified for accepting 
>itk::MultipleValuedCostFunction.
>
>
> >Hi,
>
> >The itk/vxl::LBFGS(B)Optimizer implementation IS multivariate. The 
> 'multivariate' label refers to the number of variables (input arguments 
> of the cost function) over which is optimised.
>
> >the optimisaton problem is:
>
> >min_x f(x)
>
> > On the site http://www.alglib.net/optimization/  a multivariate 
> optimisation method means that dim(x) >1 is allowed.
>
> > The LBFGSOptimizer is not suitable for MultipleValuedCostFucnction, as 
> far as I know, only for SingleValuedCostFunctions.
>
> > singlevalued means: dim(f) = 1
> > multiplevalued means: dim(f) > 1
>
> > The vnl_cost_function indeed inherits from the vnl_unary_function, but 
> the vnl_unary_function is templated over a vector of doubles, which 
> contains the input argument x:
>
> > class vnl_cost_function : public vnl_unary_function<double, 
> vnl_vector<double> >
>
> > with kind regards,
> > stefan
>
>
> >
> >
> >
> > Hi Ali,
> >
> >
> > 1) It is not a "problem", it is an "implementation feature". :-)
> >
> >
> >
> > 2) It is not in ITK, it is in VXL,
> >
> > Please look at the implementation in the file:
> >
> > Insight/Utilities/vxl/core/vnl/vnl_lbfgs.h
> >
> > Where the optimizer takes as input a vnl_cost_function
> > object, that derives from the vnl_unary_function class.
> >
> >
> >
> > 3) It will be great to have multi-variate implementations
> > of these optimizers. If you are motivated to write one,
> > it will make a great contribution to the Insight Journal.
> >
> >
> >
> > Regards,
> >
> >
> > Luis
> >
> >
> >
> > -------------
> > Ali - wrote:
> > > Hi,
> > >
> > > The VNL wrappers itk::LBFGSBOptimizer and itk::LBFGSOptimizer are
> > > classified as single-variant optimiser, while being based on the
> > > quasi-Newton method, they are multi-variant optimisers by nature -- see
> > > http://www.alglib.net/optimization/ . Is this the ITK wrapper problem or
> > > a VNL issue?
> > >
> > > _______________________________________________
> > > Insight-users mailing list
> > > Insight-users at itk.org
> > > http://www.itk.org/mailman/listinfo/insight-users
>
>
>----------
>Do you know a place like the back of your hand? Share local knowledge with 
>BackOfMyHand.com
>_______________________________________________
>Insight-users mailing list
>Insight-users at itk.org
>http://www.itk.org/mailman/listinfo/insight-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.kitware.com/pipermail/insight-users/attachments/20071107/e7e5dc98/attachment.htm