ITK  5.0.0
Insight Segmentation and Registration Toolkit
Public Types | Public Member Functions | Static Public Member Functions | Protected Types | Protected Member Functions | Private Attributes | List of all members
itk::LBFGSOptimizerv4 Class Reference

#include <itkLBFGSOptimizerv4.h>

+ Inheritance diagram for itk::LBFGSOptimizerv4:
+ Collaboration diagram for itk::LBFGSOptimizerv4:

Detailed Description

Wrap of the vnl_lbfgs algorithm for use in ITKv4 registration framework. The vnl_lbfgs is a wrapper for the NETLIB fortran code by Nocedal [1].

LBFGS is a quasi-Newton method. Quasi-Newton methods use an approximate estimate of the inverse Hessian $ (\nabla^2 f(x) )^{-1} $ to scale the gradient step:

\[ x_{n+1} = x_n - s (\nabla^2 f(x_n) )^{-1} \nabla f(x) \]

with $ s $ the step size.

The inverse Hessian is approximated from the gradients of previous iteration and thus only the gradient of the objective function is required.

The step size $ s $ is determined through line search with the approach by More and Thuente [4]. This line search approach finds a step size such that

\[ \lVert \nabla f(x + s (\nabla^2 f(x_n) )^{-1} \nabla f(x) ) \rVert \le \nu \lVert \nabla f(x) \rVert \]

The parameter $ \nu $ is set through SetLineSearchAccuracy() (default 0.9) The default step length, i.e. starting step length for the line search, is set through SetDefaultStepLength() (default 1.0).

The optimization stops when either the gradient satisfies the condition

\[ \lVert \nabla f(x) \rVert \le \epsilon \max(1, \lVert X \rVert) \]

or a maximum number of function evaluations has been reached. The tolerance $\epsilon$ is set through SetGradientConvergenceTolerance() (default 1e-5) and the maximum number of function evaluations is set through SetMaximumNumberOfFunctionEvaluations() (default 2000).

Note: The scaling of the optimization paramaters, set through SetScales(), should be set or left at one. Otherwise the Hessian approximation as well as the line search will be disturbed and the optimizer is unlikely to find a minima.

References:

[1] NETLIB lbfgs

[2] Jorge Nocedal. Updating Quasi-Newton Matrices with Limited Storage. Mathematics of Computation, Vol. 35, No. 151, pp. 773–782, 1980.

[3] Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming B, Vol. 45, No. 3, pp. 503-528, 1989.

[4] More, J. J. and D. J. Thuente. Line Search Algorithms with Guaranteed Sufficient Decrease. ACM Transactions on Mathematical Software 20, no. 3 (1994): 286–307.

Examples:
Examples/RegistrationITKv4/DeformableRegistration4.cxx.

Definition at line 85 of file itkLBFGSOptimizerv4.h.

Public Types

using ConstPointer = SmartPointer< const Self >
 
using MetricType = Superclass::MetricType
 
using ParametersType = Superclass::ParametersType
 
using Pointer = SmartPointer< Self >
 
using ScalesType = Superclass::ScalesType
 
using Self = LBFGSOptimizerv4
 
using Superclass = LBFGSOptimizerBasev4< vnl_lbfgs >
 
- Public Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using ConstPointer = SmartPointer< const Self >
 
using InternalOptimizerType = LBFGSOptimizerBaseHelperv4< vnl_lbfgs >
 
using MetricType = Superclass::MetricType
 
using ParametersType = Superclass::ParametersType
 
using Pointer = SmartPointer< Self >
 
using ScalesType = Superclass::ScalesType
 
using Self = LBFGSOptimizerBasev4
 
using StopConditionDescriptionType = Superclass::StopConditionDescriptionType
 
using StopConditionReturnStringType = Superclass::StopConditionReturnStringType
 
using Superclass = SingleValuedNonLinearVnlOptimizerv4
 
- Public Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using CommandType = ReceptorMemberCommand< Self >
 
using ConstPointer = SmartPointer< const Self >
 
using DerivativeType = Superclass::DerivativeType
 
using MetricType = Superclass::MetricType
 
using ParametersType = Superclass::ParametersType
 
using Pointer = SmartPointer< Self >
 
using ScalesType = Superclass::ScalesType
 
using Self = SingleValuedNonLinearVnlOptimizerv4
 
using StopConditionDescriptionType = Superclass::StopConditionDescriptionType
 
using StopConditionReturnStringType = Superclass::StopConditionReturnStringType
 
using Superclass = ObjectToObjectOptimizerBase
 
- Public Types inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
using ConstPointer = SmartPointer< const Self >
 
using DerivativeType = typename MetricType::DerivativeType
 
using MeasureType = typename MetricType::MeasureType
 
using MetricType = ObjectToObjectMetricBaseTemplate< double >
 
using MetricTypePointer = typename MetricType::Pointer
 
using NumberOfParametersType = typename MetricType::NumberOfParametersType
 
using ParametersType = OptimizerParameters< double >
 
using Pointer = SmartPointer< Self >
 
using ScalesEstimatorType = OptimizerParameterScalesEstimatorTemplate< double >
 
using ScalesType = OptimizerParameters< double >
 
using Self = ObjectToObjectOptimizerBaseTemplate
 
using StopConditionDescriptionType = std::ostringstream
 
using StopConditionReturnStringType = std::string
 
using Superclass = Object
 
- Public Types inherited from itk::Object
using ConstPointer = SmartPointer< const Self >
 
using Pointer = SmartPointer< Self >
 
using Self = Object
 
using Superclass = LightObject
 
- Public Types inherited from itk::LightObject
using ConstPointer = SmartPointer< const Self >
 
using Pointer = SmartPointer< Self >
 
using Self = LightObject
 

Public Member Functions

virtual ::itk::LightObject::Pointer CreateAnother () const
 
virtual double GetDefaultStepLength () const
 
virtual double GetLineSearchAccuracy () const
 
virtual const char * GetNameOfClass () const
 
void SetDefaultStepLength (double stp)
 
void SetLineSearchAccuracy (double tol)
 
void SetMetric (MetricType *metric) override
 
void StartOptimization (bool doOnlyInitialization=false) override
 
void VerboseOff ()
 
void VerboseOn ()
 
- Public Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
virtual double GetGradientConvergenceTolerance () const
 
virtual unsigned int GetMaximumNumberOfFunctionEvaluations () const
 
InternalOptimizerTypeGetOptimizer ()
 
const StopConditionReturnStringType GetStopConditionDescription () const override
 
virtual bool GetTrace () const
 
virtual void SetGradientConvergenceTolerance (double gtol)
 
virtual void SetMaximumNumberOfFunctionEvaluations (unsigned int n)
 
void SetMetric (MetricType *metric) override
 
virtual void SetTrace (bool flag)
 
void StartOptimization (bool doOnlyInitialization=false) override
 
virtual void TraceOff ()
 
virtual void TraceOn ()
 
- Public Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
virtual const DerivativeTypeGetCachedDerivative () const
 
virtual const ParametersTypeGetCachedCurrentPosition () const
 
- Public Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
virtual SizeValueType GetCurrentIteration () const
 
virtual const MeasureTypeGetCurrentMetricValue () const
 
virtual const ParametersTypeGetCurrentPosition () const
 
virtual SizeValueType GetNumberOfIterations () const
 
virtual const ThreadIdTypeGetNumberOfWorkUnits () const
 
virtual const ScalesTypeGetScales () const
 
virtual const bool & GetScalesAreIdentity () const
 
bool GetScalesInitialized () const
 
virtual const MeasureTypeGetValue () const
 
virtual const ScalesTypeGetWeights () const
 
virtual const bool & GetWeightsAreIdentity () const
 
virtual void SetNumberOfIterations (SizeValueType _arg)
 
virtual void SetNumberOfWorkUnits (ThreadIdType number)
 
virtual void SetScalesEstimator (ScalesEstimatorType *_arg)
 
virtual void SetWeights (ScalesType _arg)
 
virtual void SetMetric (MetricType *_arg)
 
virtual MetricTypeGetModifiableMetric ()
 
virtual const MetricTypeGetMetric () const
 
virtual void SetScales (const ScalesType &scales)
 
virtual void SetDoEstimateScales (bool _arg)
 
virtual const bool & GetDoEstimateScales () const
 
virtual void DoEstimateScalesOn ()
 
virtual void DoEstimateScalesOff ()
 
- Public Member Functions inherited from itk::Object
unsigned long AddObserver (const EventObject &event, Command *)
 
unsigned long AddObserver (const EventObject &event, Command *) const
 
virtual void DebugOff () const
 
virtual void DebugOn () const
 
CommandGetCommand (unsigned long tag)
 
bool GetDebug () const
 
MetaDataDictionaryGetMetaDataDictionary ()
 
const MetaDataDictionaryGetMetaDataDictionary () const
 
virtual ModifiedTimeType GetMTime () const
 
virtual const TimeStampGetTimeStamp () const
 
bool HasObserver (const EventObject &event) const
 
void InvokeEvent (const EventObject &)
 
void InvokeEvent (const EventObject &) const
 
virtual void Modified () const
 
void Register () const override
 
void RemoveAllObservers ()
 
void RemoveObserver (unsigned long tag)
 
void SetDebug (bool debugFlag) const
 
void SetReferenceCount (int) override
 
void UnRegister () const noexceptoverride
 
void SetMetaDataDictionary (const MetaDataDictionary &rhs)
 
void SetMetaDataDictionary (MetaDataDictionary &&rrhs)
 
virtual void SetObjectName (std::string _arg)
 
virtual const std::string & GetObjectName () const
 
- Public Member Functions inherited from itk::LightObject
virtual void Delete ()
 
virtual int GetReferenceCount () const
 
 itkCloneMacro (Self)
 
void Print (std::ostream &os, Indent indent=0) const
 

Static Public Member Functions

static Pointer New ()
 
- Static Public Member Functions inherited from itk::Object
static bool GetGlobalWarningDisplay ()
 
static void GlobalWarningDisplayOff ()
 
static void GlobalWarningDisplayOn ()
 
static Pointer New ()
 
static void SetGlobalWarningDisplay (bool flag)
 
- Static Public Member Functions inherited from itk::LightObject
static void BreakOnError ()
 
static Pointer New ()
 

Protected Types

using InternalOptimizerType = vnl_lbfgs
 
using InternalParametersType = vnl_vector< double >
 
- Protected Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using CostFunctionAdaptorType = Superclass::CostFunctionAdaptorType
 
using InternalOptimizerAutoPointer = std::unique_ptr< InternalOptimizerType >
 
- Protected Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using CostFunctionAdaptorType = SingleValuedVnlCostFunctionAdaptorv4
 

Protected Member Functions

 LBFGSOptimizerv4 ()
 
void PrintSelf (std::ostream &os, Indent indent) const override
 
 ~LBFGSOptimizerv4 () override
 
- Protected Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
 LBFGSOptimizerBasev4 ()
 
void PrintSelf (std::ostream &os, Indent indent) const override
 
 ~LBFGSOptimizerBasev4 () override=default
 
- Protected Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
const CostFunctionAdaptorTypeGetCostFunctionAdaptor () const
 
CostFunctionAdaptorTypeGetCostFunctionAdaptor ()
 
CostFunctionAdaptorTypeGetNonConstCostFunctionAdaptor () const
 
void SetCostFunctionAdaptor (CostFunctionAdaptorType *adaptor)
 
 SingleValuedNonLinearVnlOptimizerv4 ()
 
 ~SingleValuedNonLinearVnlOptimizerv4 () override
 
- Protected Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
void PrintSelf (std::ostream &os, Indent indent) const override
 
 ObjectToObjectOptimizerBaseTemplate ()
 
 ~ObjectToObjectOptimizerBaseTemplate () override
 
- Protected Member Functions inherited from itk::Object
 Object ()
 
bool PrintObservers (std::ostream &os, Indent indent) const
 
virtual void SetTimeStamp (const TimeStamp &time)
 
 ~Object () override
 
- Protected Member Functions inherited from itk::LightObject
virtual LightObject::Pointer InternalClone () const
 
 LightObject ()
 
virtual void PrintHeader (std::ostream &os, Indent indent) const
 
virtual void PrintTrailer (std::ostream &os, Indent indent) const
 
virtual ~LightObject ()
 

Private Attributes

double m_DefaultStepLength {1.0}
 
double m_LineSearchAccuracy {0.9}
 
bool m_Verbose {false}
 

Additional Inherited Members

- Protected Attributes inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
double m_CostFunctionConvergenceFactor
 
double m_GradientConvergenceTolerance
 
double m_InfinityNormOfProjectedGradient
 
unsigned int m_MaximumNumberOfFunctionEvaluations
 
bool m_OptimizerInitialized
 
std::ostringstream m_StopConditionDescription
 
bool m_Trace
 
InternalOptimizerAutoPointer m_VnlOptimizer
 
- Protected Attributes inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
SizeValueType m_CurrentIteration
 
MeasureType m_CurrentMetricValue
 
bool m_DoEstimateScales
 
MetricTypePointer m_Metric
 
SizeValueType m_NumberOfIterations
 
ThreadIdType m_NumberOfWorkUnits
 
ScalesType m_Scales
 
bool m_ScalesAreIdentity
 
ScalesEstimatorType::Pointer m_ScalesEstimator
 
ScalesType m_Weights
 
bool m_WeightsAreIdentity
 
- Protected Attributes inherited from itk::LightObject
std::atomic< int > m_ReferenceCount
 

Member Typedef Documentation

Definition at line 95 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::InternalOptimizerType = vnl_lbfgs
protected

Internal optimizer type.

Definition at line 143 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::InternalParametersType = vnl_vector< double >
protected

InternalParameters type alias.

Definition at line 140 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::MetricType = Superclass::MetricType

Definition at line 97 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::ParametersType = Superclass::ParametersType

Definition at line 98 of file itkLBFGSOptimizerv4.h.

Definition at line 94 of file itkLBFGSOptimizerv4.h.

using itk::LBFGSOptimizerv4::ScalesType = Superclass::ScalesType

Definition at line 99 of file itkLBFGSOptimizerv4.h.

Standard "Self" type alias.

Definition at line 92 of file itkLBFGSOptimizerv4.h.

Definition at line 93 of file itkLBFGSOptimizerv4.h.

Constructor & Destructor Documentation

itk::LBFGSOptimizerv4::LBFGSOptimizerv4 ( )
protected
itk::LBFGSOptimizerv4::~LBFGSOptimizerv4 ( )
overrideprotected

Member Function Documentation

virtual::itk::LightObject::Pointer itk::LBFGSOptimizerv4::CreateAnother ( ) const
virtual

Create an object from an instance, potentially deferring to a factory. This method allows you to create an instance of an object that is exactly the same type as the referring object. This is useful in cases where an object has been cast back to a base class.

Reimplemented from itk::Object.

virtual double itk::LBFGSOptimizerv4::GetDefaultStepLength ( ) const
virtual
virtual double itk::LBFGSOptimizerv4::GetLineSearchAccuracy ( ) const
virtual
virtual const char* itk::LBFGSOptimizerv4::GetNameOfClass ( ) const
virtual

Run-time type information (and related methods).

Reimplemented from itk::LBFGSOptimizerBasev4< vnl_lbfgs >.

static Pointer itk::LBFGSOptimizerv4::New ( )
static

Method for creation through the object factory.

void itk::LBFGSOptimizerv4::PrintSelf ( std::ostream &  os,
Indent  indent 
) const
overrideprotectedvirtual

Methods invoked by Print() to print information about the object including superclasses. Typically not called by the user (use Print() instead) but used in the hierarchical print process to combine the output of several classes.

Reimplemented from itk::Object.

void itk::LBFGSOptimizerv4::SetDefaultStepLength ( double  stp)

Set/Get the default step size. This is a positive real number with a default value of 1.0 which determines the step size in the line search.

void itk::LBFGSOptimizerv4::SetLineSearchAccuracy ( double  tol)

Set/Get the line search accuracy. This is a positive real number with a default value of 0.9, which controls the accuracy of the line search. If the function and gradient evalutions are inexpensive with respect to the cost of the iterations it may be advantageous to set the value to a small value (say 0.1).

void itk::LBFGSOptimizerv4::SetMetric ( MetricType metric)
overridevirtual

Plug in a Cost Function into the optimizer

Implements itk::SingleValuedNonLinearVnlOptimizerv4.

void itk::LBFGSOptimizerv4::StartOptimization ( bool  doOnlyInitialization = false)
overridevirtual

Start optimization with an initial value.

Reimplemented from itk::ObjectToObjectOptimizerBaseTemplate< double >.

void itk::LBFGSOptimizerv4::VerboseOff ( )
void itk::LBFGSOptimizerv4::VerboseOn ( )

Member Data Documentation

double itk::LBFGSOptimizerv4::m_DefaultStepLength {1.0}
private

Definition at line 148 of file itkLBFGSOptimizerv4.h.

double itk::LBFGSOptimizerv4::m_LineSearchAccuracy {0.9}
private

Definition at line 147 of file itkLBFGSOptimizerv4.h.

bool itk::LBFGSOptimizerv4::m_Verbose {false}
private

Definition at line 146 of file itkLBFGSOptimizerv4.h.


The documentation for this class was generated from the following file: