ITK  4.4.0
Insight Segmentation and Registration Toolkit
Public Types | Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes | Friends | List of all members
itk::QuasiNewtonOptimizerv4 Class Reference

#include <itkQuasiNewtonOptimizerv4.h>

+ Inheritance diagram for itk::QuasiNewtonOptimizerv4:
+ Collaboration diagram for itk::QuasiNewtonOptimizerv4:

Detailed Description

Implement a Quasi-Newton optimizer with BFGS Hessian estimation.

Second order approximation of the cost function is usually more efficient since it estimates the descent or ascent direction more precisely. However, computation of Hessian is usually expensive or unavailable. Alternatively Quasi-Newton methods can estimate a Hessian from the gradients in previous steps. Here a specific Quasi-Newton method, BFGS, is used to compute the Quasi-Newton steps.

The Quasi-Newton method doesn't produce a valid step sometimes, ex. when the metric function is not a convex locally. In this scenario, the gradient step is used after being scaled properly.

A helper member object, m_ScalesEstimator may be set to estimate parameter scales and step scales. A step scale measures the magnitude of a step and is used for learning rate computation.

When m_ScalesEstimator is set, SetMaximumNewtonStepSizeInPhysicalUnits() may be called to set the maximum step size. If it is not called, m_MaximumNewtonStepSizeInPhysicalUnits defaults to lambda * OptimizerParameterScalesEstimator::EstimateMaximumStepSize(), where lambda is in [1,5].

When m_ScalesEstimator is not set, the parameter scales and learning rates defaults to ones, or can be set by users manually.

Definition at line 60 of file itkQuasiNewtonOptimizerv4.h.

Public Types

typedef SmartPointer< const SelfConstPointer
typedef std::vector< HessianTypeHessianArrayType
typedef itk::Array2D
< InternalComputationValueType
typedef SmartPointer< SelfPointer
typedef QuasiNewtonOptimizerv4 Self
typedef GradientDescentOptimizerv4 Superclass
- Public Types inherited from itk::GradientDescentOptimizerv4
typedef SmartPointer< const SelfConstPointer
< double > 
typedef Superclass::DerivativeType DerivativeType
typedef Superclass::MeasureType MeasureType
typedef SmartPointer< SelfPointer
typedef GradientDescentOptimizerv4 Self
- Public Types inherited from itk::GradientDescentOptimizerBasev4
typedef SmartPointer< const SelfConstPointer
typedef MetricType::DerivativeType DerivativeType
typedef Superclass::MeasureType MeasureType
typedef Superclass::MetricType MetricType
typedef MetricType::Pointer MetricTypePointer
typedef SmartPointer< SelfPointer
typedef std::ostringstream StopConditionDescriptionType
typedef std::string StopConditionReturnStringType
enum  StopConditionType {
typedef ObjectToObjectOptimizerBase Superclass
- Public Types inherited from itk::ObjectToObjectOptimizerBase
typedef SmartPointer< const SelfConstPointer
typedef MetricType::MeasureType MeasureType
typedef ObjectToObjectMetricBase MetricType
typedef MetricType::Pointer MetricTypePointer
typedef OptimizerParameters
< double > 
typedef SmartPointer< SelfPointer
typedef OptimizerParameters
< double > 
typedef ObjectToObjectOptimizerBase Self
typedef Object Superclass
- Public Types inherited from itk::Object
typedef SmartPointer< const SelfConstPointer
typedef SmartPointer< SelfPointer
typedef Object Self
typedef LightObject Superclass
- Public Types inherited from itk::LightObject
typedef SmartPointer< const SelfConstPointer
typedef SmartPointer< SelfPointer
typedef LightObject Self

Public Member Functions

virtual ::itk::LightObject::Pointer CreateAnother (void) const
virtual const char * GetNameOfClass () const
virtual const DerivativeTypeGetNewtonStep ()
virtual void SetMaximumIterationsWithoutProgress (SizeValueType _arg)
virtual void SetMaximumNewtonStepSizeInPhysicalUnits (InternalComputationValueType _arg)
virtual void StartOptimization (bool doOnlyInitialization=false)
- Public Member Functions inherited from itk::GradientDescentOptimizerv4
virtual void EstimateLearningRate ()
virtual const
GetConvergenceValue ()
virtual const
GetLearningRate ()
virtual const
GetMaximumStepSizeInPhysicalUnits ()
virtual void ResumeOptimization ()
virtual void SetConvergenceWindowSize (SizeValueType _arg)
virtual void SetLearningRate (InternalComputationValueType _arg)
virtual void SetMaximumStepSizeInPhysicalUnits (InternalComputationValueType _arg)
virtual void SetMinimumConvergenceValue (InternalComputationValueType _arg)
virtual void SetScalesEstimator (OptimizerParameterScalesEstimator *_arg)
virtual void StopOptimization (void)
virtual void SetDoEstimateScales (bool _arg)
virtual const bool & GetDoEstimateScales ()
virtual void DoEstimateScalesOn ()
virtual void DoEstimateScalesOff ()
virtual void SetDoEstimateLearningRateAtEachIteration (bool _arg)
virtual const bool & GetDoEstimateLearningRateAtEachIteration ()
virtual void DoEstimateLearningRateAtEachIterationOn ()
virtual void DoEstimateLearningRateAtEachIterationOff ()
virtual void SetDoEstimateLearningRateOnce (bool _arg)
virtual const bool & GetDoEstimateLearningRateOnce ()
virtual void DoEstimateLearningRateOnceOn ()
virtual void DoEstimateLearningRateOnceOff ()
virtual void SetReturnBestParametersAndValue (bool _arg)
virtual const bool & GetReturnBestParametersAndValue ()
virtual void ReturnBestParametersAndValueOn ()
virtual void ReturnBestParametersAndValueOff ()
- Public Member Functions inherited from itk::GradientDescentOptimizerBasev4
virtual SizeValueType GetCurrentIteration () const
virtual const DerivativeTypeGetGradient ()
virtual const SizeValueTypeGetNumberOfIterations ()
virtual const StopConditionTypeGetStopCondition ()
virtual const
GetStopConditionDescription () const
virtual void SetNumberOfIterations (SizeValueType _arg)
virtual void ModifyGradientByScales ()
virtual void ModifyGradientByLearningRate ()
- Public Member Functions inherited from itk::ObjectToObjectOptimizerBase
virtual const MeasureTypeGetCurrentMetricValue ()
const ParametersTypeGetCurrentPosition ()
virtual const ThreadIdTypeGetNumberOfThreads ()
virtual const ScalesTypeGetScales ()
virtual const bool & GetScalesAreIdentity ()
const MeasureTypeGetValue ()
virtual const ScalesTypeGetWeights ()
virtual const bool & GetWeightsAreIdentity ()
virtual void SetNumberOfThreads (ThreadIdType number)
virtual void SetScales (ScalesType _arg)
virtual void SetWeights (ScalesType _arg)
virtual void SetMetric (MetricType *_arg)
virtual MetricTypeGetModifiableMetric ()
virtual const MetricTypeGetMetric () const
- Public Member Functions inherited from itk::Object
unsigned long AddObserver (const EventObject &event, Command *)
unsigned long AddObserver (const EventObject &event, Command *) const
virtual void DebugOff () const
virtual void DebugOn () const
CommandGetCommand (unsigned long tag)
bool GetDebug () const
MetaDataDictionaryGetMetaDataDictionary (void)
const MetaDataDictionaryGetMetaDataDictionary (void) const
virtual ModifiedTimeType GetMTime () const
virtual const TimeStampGetTimeStamp () const
bool HasObserver (const EventObject &event) const
void InvokeEvent (const EventObject &)
void InvokeEvent (const EventObject &) const
virtual void Modified () const
virtual void Register () const
void RemoveAllObservers ()
void RemoveObserver (unsigned long tag)
void SetDebug (bool debugFlag) const
void SetMetaDataDictionary (const MetaDataDictionary &rhs)
virtual void SetReferenceCount (int)
virtual void UnRegister () const
- Public Member Functions inherited from itk::LightObject
virtual void Delete ()
virtual int GetReferenceCount () const
 itkCloneMacro (Self)
void Print (std::ostream &os, Indent indent=0) const

Static Public Member Functions

static Pointer New ()
- Static Public Member Functions inherited from itk::GradientDescentOptimizerv4
static Pointer New ()

Protected Member Functions

virtual void AdvanceOneStep (void)
void CombineGradientNewtonStep (void)
virtual bool ComputeHessianAndStepWithBFGS (IndexValueType location)
virtual void EstimateNewtonStep ()
virtual void EstimateNewtonStepOverSubRange (const IndexRangeType &subrange)
void ModifyCombinedNewtonStep ()
virtual void PrintSelf (std::ostream &os, Indent indent) const
 QuasiNewtonOptimizerv4 ()
virtual void ResetNewtonStep (IndexValueType location)
virtual ~QuasiNewtonOptimizerv4 ()
- Protected Member Functions inherited from itk::GradientDescentOptimizerv4
 GradientDescentOptimizerv4 ()
virtual ~GradientDescentOptimizerv4 ()
virtual void ModifyGradientByScalesOverSubRange (const IndexRangeType &subrange)
virtual void ModifyGradientByLearningRateOverSubRange (const IndexRangeType &subrange)
- Protected Member Functions inherited from itk::GradientDescentOptimizerBasev4
 GradientDescentOptimizerBasev4 ()
virtual ~GradientDescentOptimizerBasev4 ()
- Protected Member Functions inherited from itk::ObjectToObjectOptimizerBase
 ObjectToObjectOptimizerBase ()
virtual ~ObjectToObjectOptimizerBase ()
- Protected Member Functions inherited from itk::Object
 Object ()
bool PrintObservers (std::ostream &os, Indent indent) const
virtual void SetTimeStamp (const TimeStamp &time)
virtual ~Object ()
- Protected Member Functions inherited from itk::LightObject
virtual LightObject::Pointer InternalClone () const
 LightObject ()
virtual void PrintHeader (std::ostream &os, Indent indent) const
virtual void PrintTrailer (std::ostream &os, Indent indent) const
virtual ~LightObject ()

Protected Attributes

SizeValueType m_BestIteration
ParametersType m_BestPosition
MeasureType m_BestValue
ParametersType m_CurrentPosition
HessianArrayType m_HessianArray
SizeValueType m_MaximumIterationsWithoutProgress
InternalComputationValueType m_MaximumNewtonStepSizeInPhysicalUnits
DerivativeType m_NewtonStep
std::vector< bool > m_NewtonStepValidFlags
std::string m_NewtonStepWarning
ParametersType m_OptimalStep
DerivativeType m_PreviousGradient
ParametersType m_PreviousPosition
MeasureType m_PreviousValue
- Protected Attributes inherited from itk::GradientDescentOptimizerv4
ParametersType m_BestParameters
ConvergenceMonitoringType::Pointer m_ConvergenceMonitoring
InternalComputationValueType m_ConvergenceValue
SizeValueType m_ConvergenceWindowSize
MeasureType m_CurrentBestValue
InternalComputationValueType m_LearningRate
InternalComputationValueType m_MaximumStepSizeInPhysicalUnits
InternalComputationValueType m_MinimumConvergenceValue
bool m_ReturnBestParametersAndValue
OptimizerParameterScalesEstimator::Pointer m_ScalesEstimator
- Protected Attributes inherited from itk::GradientDescentOptimizerBasev4
SizeValueType m_CurrentIteration
DerivativeType m_Gradient
GradientDescentOptimizerBasev4ModifyGradientByLearningRateThreader::Pointer m_ModifyGradientByLearningRateThreader
GradientDescentOptimizerBasev4ModifyGradientByScalesThreader::Pointer m_ModifyGradientByScalesThreader
SizeValueType m_NumberOfIterations
bool m_Stop
StopConditionType m_StopCondition
StopConditionDescriptionType m_StopConditionDescription
- Protected Attributes inherited from itk::ObjectToObjectOptimizerBase
MeasureType m_CurrentMetricValue
MetricTypePointer m_Metric
ThreadIdType m_NumberOfThreads
ScalesType m_Scales
bool m_ScalesAreIdentity
ScalesType m_Weights
bool m_WeightsAreIdentity

Private Member Functions

void operator= (const Self &)
 QuasiNewtonOptimizerv4 (const Self &)

Private Attributes

QuasiNewtonOptimizerv4EstimateNewtonStepThreader::Pointer m_EstimateNewtonStepThreader


class QuasiNewtonOptimizerv4EstimateNewtonStepThreader

Additional Inherited Members

- Protected Types inherited from itk::GradientDescentOptimizerBasev4

Member Typedef Documentation

Definition at line 68 of file itkQuasiNewtonOptimizerv4.h.

Type for an array of Hessian matrix for local support

Definition at line 82 of file itkQuasiNewtonOptimizerv4.h.

Type for Hessian matrix in the Quasi-Newton method

Definition at line 79 of file itkQuasiNewtonOptimizerv4.h.

typedef Superclass::InternalComputationValueType itk::QuasiNewtonOptimizerv4::InternalComputationValueType

Definition at line 74 of file itkQuasiNewtonOptimizerv4.h.

Definition at line 67 of file itkQuasiNewtonOptimizerv4.h.

Standard class typedefs.

Definition at line 65 of file itkQuasiNewtonOptimizerv4.h.

Definition at line 66 of file itkQuasiNewtonOptimizerv4.h.

Constructor & Destructor Documentation

itk::QuasiNewtonOptimizerv4::QuasiNewtonOptimizerv4 ( )
virtual itk::QuasiNewtonOptimizerv4::~QuasiNewtonOptimizerv4 ( )
itk::QuasiNewtonOptimizerv4::QuasiNewtonOptimizerv4 ( const Self )

Member Function Documentation

virtual void itk::QuasiNewtonOptimizerv4::AdvanceOneStep ( void  )

Advance one step using the Quasi-Newton step. When the Newton step is invalid, the gradient step will be used.

Reimplemented from itk::GradientDescentOptimizerv4.

void itk::QuasiNewtonOptimizerv4::CombineGradientNewtonStep ( void  )

Combine a gradient step with a Newton step. The Newton step will be used when it is valid. Otherwise the gradient step will be used.

virtual bool itk::QuasiNewtonOptimizerv4::ComputeHessianAndStepWithBFGS ( IndexValueType  location)

Estimate the next Hessian and step with BFGS method. The details of the method are described at .

virtual::itk::LightObject::Pointer itk::QuasiNewtonOptimizerv4::CreateAnother ( void  ) const

Create an object from an instance, potentially deferring to a factory. This method allows you to create an instance of an object that is exactly the same type as the referring object. This is useful in cases where an object has been cast back to a base class.

Reimplemented from itk::GradientDescentOptimizerv4.

virtual void itk::QuasiNewtonOptimizerv4::EstimateNewtonStep ( )

Estimate a Newton step

virtual void itk::QuasiNewtonOptimizerv4::EstimateNewtonStepOverSubRange ( const IndexRangeType subrange)

Estimate the quasi-newton step over a given index range.

virtual const char* itk::QuasiNewtonOptimizerv4::GetNameOfClass ( ) const

Run-time type information (and related methods).

Reimplemented from itk::GradientDescentOptimizerv4.

virtual const DerivativeType& itk::QuasiNewtonOptimizerv4::GetNewtonStep ( )

Get the most recent Newton step.

void itk::QuasiNewtonOptimizerv4::ModifyCombinedNewtonStep ( )

Estimate and apply the learning rate(s) for a combined Newton step. A combined Newton step uses the Newton step by default and the gradient step when the Newton step is not valid.

The learning rate is less than 1.0 and is restricted by m_MaximumNewtonStepSizeInPhysicalUnits.

static Pointer itk::QuasiNewtonOptimizerv4::New ( )

Method for creation through the object factory.

void itk::QuasiNewtonOptimizerv4::operator= ( const Self )
virtual void itk::QuasiNewtonOptimizerv4::PrintSelf ( std::ostream &  os,
Indent  indent 
) const

Methods invoked by Print() to print information about the object including superclasses. Typically not called by the user (use Print() instead) but used in the hierarchical print process to combine the output of several classes.

Reimplemented from itk::GradientDescentOptimizerv4.

virtual void itk::QuasiNewtonOptimizerv4::ResetNewtonStep ( IndexValueType  location)

Reset the Hessian to identity matrix and the Newton step to zeros.

virtual void itk::QuasiNewtonOptimizerv4::SetMaximumIterationsWithoutProgress ( SizeValueType  _arg)

Set the maximum tolerable number of iteration without any progress

virtual void itk::QuasiNewtonOptimizerv4::SetMaximumNewtonStepSizeInPhysicalUnits ( InternalComputationValueType  _arg)

Set the maximum step size.

When SetScalesEstimator is called by user, the optimizer will compute learning rates as m_MaximumNewtonStepSizeInPhysicalUnits / m_ScalesEstimator->EstimateStepScale(newtonStep).

If SetMaximumNewtonStepSizeInPhysicalUnits is not called by user, m_MaximumNewtonStepSizeInPhysicalUnits defaults to lambda * m_ScalesEstimator->EstimateMaximumStepSize(),

where EstimateMaximumStepSize returns one voxel spacing and lambda may be in [1,5] according to our experience.

virtual void itk::QuasiNewtonOptimizerv4::StartOptimization ( bool  doOnlyInitialization = false)

Start and run the optimization

Reimplemented from itk::GradientDescentOptimizerv4.

Friends And Related Function Documentation

Definition at line 187 of file itkQuasiNewtonOptimizerv4.h.

Member Data Documentation

SizeValueType itk::QuasiNewtonOptimizerv4::m_BestIteration

Definition at line 126 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_BestPosition

Definition at line 125 of file itkQuasiNewtonOptimizerv4.h.

MeasureType itk::QuasiNewtonOptimizerv4::m_BestValue

The best value so far and relevant information

Definition at line 124 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_CurrentPosition

The information about the current step

Definition at line 115 of file itkQuasiNewtonOptimizerv4.h.

QuasiNewtonOptimizerv4EstimateNewtonStepThreader::Pointer itk::QuasiNewtonOptimizerv4::m_EstimateNewtonStepThreader

Threader for Newton step estimation.

Definition at line 194 of file itkQuasiNewtonOptimizerv4.h.

HessianArrayType itk::QuasiNewtonOptimizerv4::m_HessianArray

The Hessian with local support

Definition at line 138 of file itkQuasiNewtonOptimizerv4.h.

SizeValueType itk::QuasiNewtonOptimizerv4::m_MaximumIterationsWithoutProgress

The maximum tolerable number of iteration without any progress

Definition at line 107 of file itkQuasiNewtonOptimizerv4.h.

InternalComputationValueType itk::QuasiNewtonOptimizerv4::m_MaximumNewtonStepSizeInPhysicalUnits

The maximum Quasi-Newton step size to restrict learning rates.

Definition at line 135 of file itkQuasiNewtonOptimizerv4.h.

DerivativeType itk::QuasiNewtonOptimizerv4::m_NewtonStep

The Quasi-Newton step

Definition at line 129 of file itkQuasiNewtonOptimizerv4.h.

std::vector<bool> itk::QuasiNewtonOptimizerv4::m_NewtonStepValidFlags

Valid flag for the Quasi-Newton steps

Definition at line 141 of file itkQuasiNewtonOptimizerv4.h.

std::string itk::QuasiNewtonOptimizerv4::m_NewtonStepWarning

Warning message during Quasi-Newton step estimation

Definition at line 132 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_OptimalStep

Definition at line 116 of file itkQuasiNewtonOptimizerv4.h.

DerivativeType itk::QuasiNewtonOptimizerv4::m_PreviousGradient

Definition at line 121 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_PreviousPosition

Definition at line 120 of file itkQuasiNewtonOptimizerv4.h.

MeasureType itk::QuasiNewtonOptimizerv4::m_PreviousValue

The information about the previous step

Definition at line 119 of file itkQuasiNewtonOptimizerv4.h.

The documentation for this class was generated from the following file: