#include <itkLBFGSOptimizerv4.h>

Inheritance diagram for itk::LBFGSOptimizerv4:

Collaboration diagram for itk::LBFGSOptimizerv4:

Detailed Description

Wrap of the vnl_lbfgs algorithm for use in ITKv4 registration framework. The vnl_lbfgs is a wrapper for the NETLIB fortran code by Nocedal [1].

LBFGS is a quasi-Newton method. Quasi-Newton methods use an approximate estimate of the inverse Hessian $(\nabla^2 f(x) )^{-1}$ to scale the gradient step:

$x_{n+1} = x_n - s (\nabla^2 f(x_n) )^{-1} \nabla f(x)$

with $ s $ the step size.

The inverse Hessian is approximated from the gradients of previous iteration and thus only the gradient of the objective function is required.

The step size $ s $ is determined through line search with the approach by More and Thuente [4]. This line search approach finds a step size such that

$\lVert \nabla f(x + s (\nabla^2 f(x_n) )^{-1} \nabla f(x) ) \rVert \le \nu \lVert \nabla f(x) \rVert$

The parameter $\nu$ is set through SetLineSearchAccuracy() (default 0.9) The default step length, i.e. starting step length for the line search, is set through SetDefaultStepLength() (default 1.0).

The optimization stops when either the gradient satisfies the condition

$\lVert \nabla f(x) \rVert \le \epsilon \max(1, \lVert X \rVert)$

or a maximum number of function evaluations has been reached. The tolerance $\epsilon$ is set through SetGradientConvergenceTolerance() (default 1e-5) and the maximum number of function evaluations is set through SetMaximumNumberOfFunctionEvaluations() (default 2000).

Note: The scaling of the optimization paramaters, set through SetScales(), should be set or left at one. Otherwise the Hessian approximation as well as the line search will be disturbed and the optimizer is unlikely to find a minima.

References:

[1] NETLIB lbfgs

[2] Jorge Nocedal. Updating Quasi-Newton Matrices with Limited Storage. Mathematics of Computation, Vol. 35, No. 151, pp. 773–782, 1980.

[3] Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming B, Vol. 45, No. 3, pp. 503-528, 1989.

[4] More, J. J. and D. J. Thuente. Line Search Algorithms with Guaranteed Sufficient Decrease. ACM Transactions on Mathematical Software 20, no. 3 (1994): 286–307.

Examples:: Examples/RegistrationITKv4/DeformableRegistration4.cxx.

Definition at line 85 of file itkLBFGSOptimizerv4.h.

Public Types
using	ConstPointer = SmartPointer< const Self >

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer< Self >

using	ScalesType = Superclass::ScalesType

using	Self = LBFGSOptimizerv4

using	Superclass = LBFGSOptimizerBasev4< vnl_lbfgs >

Public Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using	ConstPointer = SmartPointer< const Self >

using	InternalOptimizerType = LBFGSOptimizerBaseHelperv4< vnl_lbfgs >

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer< Self >

using	ScalesType = Superclass::ScalesType

using	Self = LBFGSOptimizerBasev4

using	StopConditionDescriptionType = Superclass::StopConditionDescriptionType

using	StopConditionReturnStringType = Superclass::StopConditionReturnStringType

using	Superclass = SingleValuedNonLinearVnlOptimizerv4

Public Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using	CommandType = ReceptorMemberCommand< Self >

using	ConstPointer = SmartPointer< const Self >

using	DerivativeType = Superclass::DerivativeType

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer< Self >

using	ScalesType = Superclass::ScalesType

using	Self = SingleValuedNonLinearVnlOptimizerv4

using	StopConditionDescriptionType = Superclass::StopConditionDescriptionType

using	StopConditionReturnStringType = Superclass::StopConditionReturnStringType

using	Superclass = ObjectToObjectOptimizerBase

Public Types inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
using	ConstPointer = SmartPointer< const Self >

using	DerivativeType = typename MetricType::DerivativeType

using	MeasureType = typename MetricType::MeasureType

using	MetricType = ObjectToObjectMetricBaseTemplate< double >

using	MetricTypePointer = typename MetricType::Pointer

using	NumberOfParametersType = typename MetricType::NumberOfParametersType

using	ParametersType = OptimizerParameters< double >

using	Pointer = SmartPointer< Self >

using	ScalesEstimatorType = OptimizerParameterScalesEstimatorTemplate< double >

using	ScalesType = OptimizerParameters< double >

using	Self = ObjectToObjectOptimizerBaseTemplate

using	StopConditionDescriptionType = std::ostringstream

using	StopConditionReturnStringType = std::string

using	Superclass = Object

Public Types inherited from itk::Object
using	ConstPointer = SmartPointer< const Self >

using	Pointer = SmartPointer< Self >

using	Self = Object

using	Superclass = LightObject

Public Types inherited from itk::LightObject
using	ConstPointer = SmartPointer< const Self >

using	Pointer = SmartPointer< Self >

using	Self = LightObject

Public Member Functions
virtual ::itk::LightObject::Pointer	CreateAnother () const

virtual double	GetDefaultStepLength () const

virtual double	GetLineSearchAccuracy () const

virtual const char *	GetNameOfClass () const

void	SetDefaultStepLength (double stp)

void	SetLineSearchAccuracy (double tol)

void	SetMetric (MetricType *metric) override

void	StartOptimization (bool doOnlyInitialization=false) override

void	VerboseOff ()

void	VerboseOn ()

Public Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
virtual double	GetGradientConvergenceTolerance () const

virtual unsigned int	GetMaximumNumberOfFunctionEvaluations () const

InternalOptimizerType *	GetOptimizer ()

const StopConditionReturnStringType	GetStopConditionDescription () const override

virtual bool	GetTrace () const

virtual void	SetGradientConvergenceTolerance (double gtol)

virtual void	SetMaximumNumberOfFunctionEvaluations (unsigned int n)

void	SetMetric (MetricType *metric) override

virtual void	SetTrace (bool flag)

void	StartOptimization (bool doOnlyInitialization=false) override

virtual void	TraceOff ()

virtual void	TraceOn ()

Public Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
virtual const DerivativeType &	GetCachedDerivative () const

virtual const ParametersType &	GetCachedCurrentPosition () const

Public Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
virtual SizeValueType	GetCurrentIteration () const

virtual const MeasureType &	GetCurrentMetricValue () const

virtual const ParametersType &	GetCurrentPosition () const

virtual SizeValueType	GetNumberOfIterations () const

virtual const ThreadIdType &	GetNumberOfWorkUnits () const

virtual const ScalesType &	GetScales () const

virtual const bool &	GetScalesAreIdentity () const

bool	GetScalesInitialized () const

virtual const MeasureType &	GetValue () const

virtual const ScalesType &	GetWeights () const

virtual const bool &	GetWeightsAreIdentity () const

virtual void	SetNumberOfIterations (SizeValueType _arg)

virtual void	SetNumberOfWorkUnits (ThreadIdType number)

virtual void	SetScalesEstimator (ScalesEstimatorType *_arg)

virtual void	SetWeights (ScalesType _arg)

virtual void	SetMetric (MetricType *_arg)

virtual MetricType *	GetModifiableMetric ()

virtual const MetricType *	GetMetric () const

virtual void	SetScales (const ScalesType &scales)

virtual void	SetDoEstimateScales (bool _arg)

virtual const bool &	GetDoEstimateScales () const

virtual void	DoEstimateScalesOn ()

virtual void	DoEstimateScalesOff ()

Public Member Functions inherited from itk::Object
unsigned long	AddObserver (const EventObject &event, Command *)

unsigned long	AddObserver (const EventObject &event, Command *) const

virtual void	DebugOff () const

virtual void	DebugOn () const

Command *	GetCommand (unsigned long tag)

bool	GetDebug () const

MetaDataDictionary &	GetMetaDataDictionary ()

const MetaDataDictionary &	GetMetaDataDictionary () const

virtual ModifiedTimeType	GetMTime () const

virtual const TimeStamp &	GetTimeStamp () const

bool	HasObserver (const EventObject &event) const

void	InvokeEvent (const EventObject &)

void	InvokeEvent (const EventObject &) const

virtual void	Modified () const

void	Register () const override

void	RemoveAllObservers ()

void	RemoveObserver (unsigned long tag)

void	SetDebug (bool debugFlag) const

void	SetReferenceCount (int) override

void	UnRegister () const noexceptoverride

void	SetMetaDataDictionary (const MetaDataDictionary &rhs)

void	SetMetaDataDictionary (MetaDataDictionary &&rrhs)

virtual void	SetObjectName (std::string _arg)

virtual const std::string &	GetObjectName () const

Public Member Functions inherited from itk::LightObject
virtual void	Delete ()

virtual int	GetReferenceCount () const

	itkCloneMacro (Self)

void	Print (std::ostream &os, Indent indent=0) const

Static Public Member Functions
static Pointer	New ()

Static Public Member Functions inherited from itk::Object
static bool	GetGlobalWarningDisplay ()

static void	GlobalWarningDisplayOff ()

static void	GlobalWarningDisplayOn ()

static Pointer	New ()

static void	SetGlobalWarningDisplay (bool flag)

Static Public Member Functions inherited from itk::LightObject
static void	BreakOnError ()

static Pointer	New ()

Protected Types
using	InternalOptimizerType = vnl_lbfgs

using	InternalParametersType = vnl_vector< double >

Protected Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using	CostFunctionAdaptorType = Superclass::CostFunctionAdaptorType

using	InternalOptimizerAutoPointer = std::unique_ptr< InternalOptimizerType >

Protected Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using	CostFunctionAdaptorType = SingleValuedVnlCostFunctionAdaptorv4

Protected Member Functions
	LBFGSOptimizerv4 ()

void	PrintSelf (std::ostream &os, Indent indent) const override

	~LBFGSOptimizerv4 () override

Protected Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
	LBFGSOptimizerBasev4 ()

void	PrintSelf (std::ostream &os, Indent indent) const override

	~LBFGSOptimizerBasev4 () override=default

Protected Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
const CostFunctionAdaptorType *	GetCostFunctionAdaptor () const

CostFunctionAdaptorType *	GetCostFunctionAdaptor ()

CostFunctionAdaptorType *	GetNonConstCostFunctionAdaptor () const

void	SetCostFunctionAdaptor (CostFunctionAdaptorType *adaptor)

	SingleValuedNonLinearVnlOptimizerv4 ()

	~SingleValuedNonLinearVnlOptimizerv4 () override

Protected Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
void	PrintSelf (std::ostream &os, Indent indent) const override

	ObjectToObjectOptimizerBaseTemplate ()

	~ObjectToObjectOptimizerBaseTemplate () override

Protected Member Functions inherited from itk::Object
	Object ()

bool	PrintObservers (std::ostream &os, Indent indent) const

virtual void	SetTimeStamp (const TimeStamp &time)

	~Object () override

Protected Member Functions inherited from itk::LightObject
virtual LightObject::Pointer	InternalClone () const

	LightObject ()

virtual void	PrintHeader (std::ostream &os, Indent indent) const

virtual void	PrintTrailer (std::ostream &os, Indent indent) const

virtual	~LightObject ()

Private Attributes
double	m_DefaultStepLength {1.0}

double	m_LineSearchAccuracy {0.9}

bool	m_Verbose {false}

Additional Inherited Members
Protected Attributes inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
double	m_CostFunctionConvergenceFactor

double	m_GradientConvergenceTolerance

double	m_InfinityNormOfProjectedGradient

unsigned int	m_MaximumNumberOfFunctionEvaluations

bool	m_OptimizerInitialized

std::ostringstream	m_StopConditionDescription

bool	m_Trace

InternalOptimizerAutoPointer	m_VnlOptimizer

Protected Attributes inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
SizeValueType	m_CurrentIteration

MeasureType	m_CurrentMetricValue

bool	m_DoEstimateScales

MetricTypePointer	m_Metric

SizeValueType	m_NumberOfIterations

ThreadIdType	m_NumberOfWorkUnits

ScalesType	m_Scales

bool	m_ScalesAreIdentity

ScalesEstimatorType::Pointer	m_ScalesEstimator

ScalesType	m_Weights

bool	m_WeightsAreIdentity

Protected Attributes inherited from itk::LightObject
std::atomic< int >	m_ReferenceCount

Member Typedef Documentation

using itk::LBFGSOptimizerv4::ConstPointer = SmartPointer< const Self >

Definition at line 95 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::InternalOptimizerType = vnl_lbfgs

protected

Internal optimizer type.

Definition at line 143 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::InternalParametersType = vnl_vector< double >

protected

InternalParameters type alias.

Definition at line 140 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::MetricType = Superclass::MetricType

Definition at line 97 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::ParametersType = Superclass::ParametersType

Definition at line 98 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::Pointer = SmartPointer< Self >

Definition at line 94 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::ScalesType = Superclass::ScalesType

Definition at line 99 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::Self = LBFGSOptimizerv4

Standard "Self" type alias.

Definition at line 92 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::Superclass = LBFGSOptimizerBasev4<vnl_lbfgs>

Definition at line 93 of file itkLBFGSOptimizerv4.h.

Constructor & Destructor Documentation

itk::LBFGSOptimizerv4::LBFGSOptimizerv4 ( )

protected

itk::LBFGSOptimizerv4::~LBFGSOptimizerv4 ( )

overrideprotected

Member Function Documentation

virtual::itk::LightObject::Pointer itk::LBFGSOptimizerv4::CreateAnother ( ) const

virtual

Create an object from an instance, potentially deferring to a factory. This method allows you to create an instance of an object that is exactly the same type as the referring object. This is useful in cases where an object has been cast back to a base class.

Reimplemented from itk::Object.

virtual double itk::LBFGSOptimizerv4::GetDefaultStepLength ( ) const

virtual

virtual double itk::LBFGSOptimizerv4::GetLineSearchAccuracy ( ) const

virtual

virtual const char* itk::LBFGSOptimizerv4::GetNameOfClass ( ) const

virtual

Run-time type information (and related methods).

Reimplemented from itk::LBFGSOptimizerBasev4< vnl_lbfgs >.

static Pointer itk::LBFGSOptimizerv4::New ( )

static

Method for creation through the object factory.

void itk::LBFGSOptimizerv4::PrintSelf	(	std::ostream &	os,
		Indent	indent
	)		const

overrideprotectedvirtual

Methods invoked by Print() to print information about the object including superclasses. Typically not called by the user (use Print() instead) but used in the hierarchical print process to combine the output of several classes.

Reimplemented from itk::Object.

void itk::LBFGSOptimizerv4::SetDefaultStepLength ( double stp )

Set/Get the default step size. This is a positive real number with a default value of 1.0 which determines the step size in the line search.

void itk::LBFGSOptimizerv4::SetLineSearchAccuracy ( double tol )

Set/Get the line search accuracy. This is a positive real number with a default value of 0.9, which controls the accuracy of the line search. If the function and gradient evalutions are inexpensive with respect to the cost of the iterations it may be advantageous to set the value to a small value (say 0.1).

void itk::LBFGSOptimizerv4::SetMetric ( MetricType * metric )

overridevirtual

Plug in a Cost Function into the optimizer

Implements itk::SingleValuedNonLinearVnlOptimizerv4.

void itk::LBFGSOptimizerv4::StartOptimization ( bool doOnlyInitialization = false )

overridevirtual

Start optimization with an initial value.

Reimplemented from itk::ObjectToObjectOptimizerBaseTemplate< double >.

void itk::LBFGSOptimizerv4::VerboseOff ( )

void itk::LBFGSOptimizerv4::VerboseOn ( )

Member Data Documentation

double itk::LBFGSOptimizerv4::m_DefaultStepLength {1.0}

private

Definition at line 148 of file itkLBFGSOptimizerv4.h.

double itk::LBFGSOptimizerv4::m_LineSearchAccuracy {0.9}

private

Definition at line 147 of file itkLBFGSOptimizerv4.h.

bool itk::LBFGSOptimizerv4::m_Verbose {false}

private

Definition at line 146 of file itkLBFGSOptimizerv4.h.

The documentation for this class was generated from the following file:

itkLBFGSOptimizerv4.h

Detailed Description

Public Types

Public Member Functions

Static Public Member Functions

Protected Types

Protected Member Functions

Private Attributes

Additional Inherited Members

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation