ITK  4.3.0
Insight Segmentation and Registration Toolkit
Public Types | Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes | Friends | List of all members
itk::QuasiNewtonOptimizerv4 Class Reference

#include <itkQuasiNewtonOptimizerv4.h>

+ Inheritance diagram for itk::QuasiNewtonOptimizerv4:
+ Collaboration diagram for itk::QuasiNewtonOptimizerv4:

Detailed Description

Implement a Quasi-Newton optimizer with BFGS Hessian estimation.

Second order approximation of the cost function is usually more efficient since it estimates the descent or ascent direction more precisely. However, computation of Hessian is usually expensive or unavailable. Alternatively Quasi-Newton methods can estimate a Hessian from the gradients in previous steps. Here a specific Quasi-Newton method, BFGS, is used to compute the Quasi-Newton steps.

The Quasi-Newton method doesn't produce a valid step sometimes, ex. when the metric function is not a convex locally. In this scenario, the gradient step is used after being scaled properly.

A helper member object, m_ScalesEstimator may be set to estimate parameter scales and step scales. A step scale measures the magnitude of a step and is used for learning rate computation.

When m_ScalesEstimator is set, SetMaximumNewtonStepSizeInPhysicalUnits() may be called to set the maximum step size. If it is not called, m_MaximumNewtonStepSizeInPhysicalUnits defaults to lambda * OptimizerParameterScalesEstimator::EstimateMaximumStepSize(), where lambda is in [1,5].

When m_ScalesEstimator is not set, the parameter scales and learning rates defaults to ones, or can be set by users manually.

Definition at line 60 of file itkQuasiNewtonOptimizerv4.h.

Public Types

typedef SmartPointer< const SelfConstPointer
 
typedef std::vector< HessianTypeHessianArrayType
 
typedef itk::Array2D
< InternalComputationValueType
HessianType
 
typedef
Superclass::InternalComputationValueType 
InternalComputationValueType
 
typedef SmartPointer< SelfPointer
 
typedef QuasiNewtonOptimizerv4 Self
 
typedef GradientDescentOptimizerv4 Superclass
 
- Public Types inherited from itk::GradientDescentOptimizerv4
typedef SmartPointer< const SelfConstPointer
 
typedef
itk::Function::WindowConvergenceMonitoringFunction
< double > 
ConvergenceMonitoringType
 
typedef Superclass::DerivativeType DerivativeType
 
typedef
Superclass::InternalComputationValueType 
InternalComputationValueType
 
typedef Superclass::MeasureType MeasureType
 
typedef SmartPointer< SelfPointer
 
typedef GradientDescentOptimizerv4 Self
 
typedef
GradientDescentOptimizerBasev4 
Superclass
 
- Public Types inherited from itk::GradientDescentOptimizerBasev4
typedef SmartPointer< const SelfConstPointer
 
typedef MetricType::DerivativeType DerivativeType
 
typedef
Superclass::InternalComputationValueType 
InternalComputationValueType
 
typedef Superclass::MeasureType MeasureType
 
typedef Superclass::MetricType MetricType
 
typedef MetricType::Pointer MetricTypePointer
 
typedef SmartPointer< SelfPointer
 
typedef
GradientDescentOptimizerBasev4 
Self
 
typedef std::ostringstream StopConditionDescriptionType
 
typedef std::string StopConditionReturnStringType
 
enum  StopConditionType {
  MAXIMUM_NUMBER_OF_ITERATIONS,
  COSTFUNCTION_ERROR,
  UPDATE_PARAMETERS_ERROR,
  STEP_TOO_SMALL,
  QUASI_NEWTON_STEP_ERROR,
  CONVERGENCE_CHECKER_PASSED,
  OTHER_ERROR
}
 
typedef ObjectToObjectOptimizerBase Superclass
 
- Public Types inherited from itk::ObjectToObjectOptimizerBase
typedef SmartPointer< const SelfConstPointer
 
typedef
MetricType::InternalComputationValueType 
InternalComputationValueType
 
typedef MetricType::MeasureType MeasureType
 
typedef ObjectToObjectMetricBase MetricType
 
typedef MetricType::Pointer MetricTypePointer
 
typedef
MetricType::NumberOfParametersType 
NumberOfParametersType
 
typedef OptimizerParameters
< double > 
ParametersType
 
typedef SmartPointer< SelfPointer
 
typedef OptimizerParameters
< double > 
ScalesType
 
typedef ObjectToObjectOptimizerBase Self
 
typedef Object Superclass
 
- Public Types inherited from itk::Object
typedef SmartPointer< const SelfConstPointer
 
typedef SmartPointer< SelfPointer
 
typedef Object Self
 
typedef LightObject Superclass
 
- Public Types inherited from itk::LightObject
typedef SmartPointer< const SelfConstPointer
 
typedef SmartPointer< SelfPointer
 
typedef LightObject Self
 

Public Member Functions

virtual ::itk::LightObject::Pointer CreateAnother (void) const
 
virtual const char * GetNameOfClass () const
 
virtual const DerivativeTypeGetNewtonStep ()
 
virtual void SetMaximumIterationsWithoutProgress (SizeValueType _arg)
 
virtual void SetMaximumNewtonStepSizeInPhysicalUnits (InternalComputationValueType _arg)
 
virtual void StartOptimization (bool doOnlyInitialization=false)
 
- Public Member Functions inherited from itk::GradientDescentOptimizerv4
virtual void EstimateLearningRate ()
 
virtual const
InternalComputationValueType
GetConvergenceValue ()
 
virtual const
InternalComputationValueType
GetLearningRate ()
 
virtual const
InternalComputationValueType
GetMaximumStepSizeInPhysicalUnits ()
 
virtual void ResumeOptimization ()
 
virtual void SetConvergenceWindowSize (SizeValueType _arg)
 
virtual void SetLearningRate (InternalComputationValueType _arg)
 
virtual void SetMaximumStepSizeInPhysicalUnits (InternalComputationValueType _arg)
 
virtual void SetMinimumConvergenceValue (InternalComputationValueType _arg)
 
virtual void SetScalesEstimator (OptimizerParameterScalesEstimator *_arg)
 
virtual void StopOptimization (void)
 
virtual void SetDoEstimateScales (bool _arg)
 
virtual const bool & GetDoEstimateScales ()
 
virtual void DoEstimateScalesOn ()
 
virtual void DoEstimateScalesOff ()
 
virtual void SetDoEstimateLearningRateAtEachIteration (bool _arg)
 
virtual const bool & GetDoEstimateLearningRateAtEachIteration ()
 
virtual void DoEstimateLearningRateAtEachIterationOn ()
 
virtual void DoEstimateLearningRateAtEachIterationOff ()
 
virtual void SetDoEstimateLearningRateOnce (bool _arg)
 
virtual const bool & GetDoEstimateLearningRateOnce ()
 
virtual void DoEstimateLearningRateOnceOn ()
 
virtual void DoEstimateLearningRateOnceOff ()
 
virtual void SetReturnBestParametersAndValue (bool _arg)
 
virtual const bool & GetReturnBestParametersAndValue ()
 
virtual void ReturnBestParametersAndValueOn ()
 
virtual void ReturnBestParametersAndValueOff ()
 
- Public Member Functions inherited from itk::GradientDescentOptimizerBasev4
virtual SizeValueType GetCurrentIteration () const
 
virtual const DerivativeTypeGetGradient ()
 
virtual const SizeValueTypeGetNumberOfIterations ()
 
virtual const StopConditionTypeGetStopCondition ()
 
virtual const
StopConditionReturnStringType 
GetStopConditionDescription () const
 
virtual void SetNumberOfIterations (SizeValueType _arg)
 
virtual void ModifyGradientByScales ()
 
virtual void ModifyGradientByLearningRate ()
 
- Public Member Functions inherited from itk::ObjectToObjectOptimizerBase
virtual const MeasureTypeGetCurrentMetricValue ()
 
const ParametersTypeGetCurrentPosition ()
 
virtual const ThreadIdTypeGetNumberOfThreads ()
 
virtual const ScalesTypeGetScales ()
 
virtual const bool & GetScalesAreIdentity ()
 
const MeasureTypeGetValue ()
 
virtual const ScalesTypeGetWeights ()
 
virtual const bool & GetWeightsAreIdentity ()
 
virtual void SetNumberOfThreads (ThreadIdType number)
 
virtual void SetScales (ScalesType _arg)
 
virtual void SetWeights (ScalesType _arg)
 
virtual MetricTypeGetMetric ()
 
virtual void SetMetric (MetricType *_arg)
 
- Public Member Functions inherited from itk::Object
unsigned long AddObserver (const EventObject &event, Command *)
 
unsigned long AddObserver (const EventObject &event, Command *) const
 
virtual void DebugOff () const
 
virtual void DebugOn () const
 
CommandGetCommand (unsigned long tag)
 
bool GetDebug () const
 
MetaDataDictionaryGetMetaDataDictionary (void)
 
const MetaDataDictionaryGetMetaDataDictionary (void) const
 
virtual ModifiedTimeType GetMTime () const
 
virtual const TimeStampGetTimeStamp () const
 
bool HasObserver (const EventObject &event) const
 
void InvokeEvent (const EventObject &)
 
void InvokeEvent (const EventObject &) const
 
virtual void Modified () const
 
virtual void Register () const
 
void RemoveAllObservers ()
 
void RemoveObserver (unsigned long tag)
 
void SetDebug (bool debugFlag) const
 
void SetMetaDataDictionary (const MetaDataDictionary &rhs)
 
virtual void SetReferenceCount (int)
 
virtual void UnRegister () const
 
- Public Member Functions inherited from itk::LightObject
virtual void Delete ()
 
virtual int GetReferenceCount () const
 
 itkCloneMacro (Self)
 
void Print (std::ostream &os, Indent indent=0) const
 

Static Public Member Functions

static Pointer New ()
 
- Static Public Member Functions inherited from itk::GradientDescentOptimizerv4
static Pointer New ()
 

Protected Member Functions

virtual void AdvanceOneStep (void)
 
void CombineGradientNewtonStep (void)
 
virtual bool ComputeHessianAndStepWithBFGS (IndexValueType location)
 
virtual void EstimateNewtonStep ()
 
virtual void EstimateNewtonStepOverSubRange (const IndexRangeType &subrange)
 
void ModifyCombinedNewtonStep ()
 
virtual void PrintSelf (std::ostream &os, Indent indent) const
 
 QuasiNewtonOptimizerv4 ()
 
virtual void ResetNewtonStep (IndexValueType location)
 
virtual ~QuasiNewtonOptimizerv4 ()
 
- Protected Member Functions inherited from itk::GradientDescentOptimizerv4
 GradientDescentOptimizerv4 ()
 
virtual ~GradientDescentOptimizerv4 ()
 
virtual void ModifyGradientByScalesOverSubRange (const IndexRangeType &subrange)
 
virtual void ModifyGradientByLearningRateOverSubRange (const IndexRangeType &subrange)
 
- Protected Member Functions inherited from itk::GradientDescentOptimizerBasev4
 GradientDescentOptimizerBasev4 ()
 
virtual ~GradientDescentOptimizerBasev4 ()
 
- Protected Member Functions inherited from itk::ObjectToObjectOptimizerBase
 ObjectToObjectOptimizerBase ()
 
virtual ~ObjectToObjectOptimizerBase ()
 
- Protected Member Functions inherited from itk::Object
 Object ()
 
bool PrintObservers (std::ostream &os, Indent indent) const
 
virtual void SetTimeStamp (const TimeStamp &time)
 
virtual ~Object ()
 
- Protected Member Functions inherited from itk::LightObject
virtual LightObject::Pointer InternalClone () const
 
 LightObject ()
 
virtual void PrintHeader (std::ostream &os, Indent indent) const
 
virtual void PrintTrailer (std::ostream &os, Indent indent) const
 
virtual ~LightObject ()
 

Protected Attributes

SizeValueType m_BestIteration
 
ParametersType m_BestPosition
 
MeasureType m_BestValue
 
ParametersType m_CurrentPosition
 
HessianArrayType m_HessianArray
 
SizeValueType m_MaximumIterationsWithoutProgress
 
InternalComputationValueType m_MaximumNewtonStepSizeInPhysicalUnits
 
DerivativeType m_NewtonStep
 
std::vector< bool > m_NewtonStepValidFlags
 
std::string m_NewtonStepWarning
 
ParametersType m_OptimalStep
 
DerivativeType m_PreviousGradient
 
ParametersType m_PreviousPosition
 
MeasureType m_PreviousValue
 
- Protected Attributes inherited from itk::GradientDescentOptimizerv4
ParametersType m_BestParameters
 
ConvergenceMonitoringType::Pointer m_ConvergenceMonitoring
 
InternalComputationValueType m_ConvergenceValue
 
SizeValueType m_ConvergenceWindowSize
 
MeasureType m_CurrentBestValue
 
InternalComputationValueType m_LearningRate
 
InternalComputationValueType m_MaximumStepSizeInPhysicalUnits
 
InternalComputationValueType m_MinimumConvergenceValue
 
bool m_ReturnBestParametersAndValue
 
OptimizerParameterScalesEstimator::Pointer m_ScalesEstimator
 
- Protected Attributes inherited from itk::GradientDescentOptimizerBasev4
SizeValueType m_CurrentIteration
 
DerivativeType m_Gradient
 
GradientDescentOptimizerBasev4ModifyGradientByLearningRateThreader::Pointer m_ModifyGradientByLearningRateThreader
 
GradientDescentOptimizerBasev4ModifyGradientByScalesThreader::Pointer m_ModifyGradientByScalesThreader
 
SizeValueType m_NumberOfIterations
 
bool m_Stop
 
StopConditionType m_StopCondition
 
StopConditionDescriptionType m_StopConditionDescription
 
- Protected Attributes inherited from itk::ObjectToObjectOptimizerBase
MeasureType m_CurrentMetricValue
 
MetricTypePointer m_Metric
 
ThreadIdType m_NumberOfThreads
 
ScalesType m_Scales
 
bool m_ScalesAreIdentity
 
ScalesType m_Weights
 
bool m_WeightsAreIdentity
 

Private Member Functions

void operator= (const Self &)
 
 QuasiNewtonOptimizerv4 (const Self &)
 

Private Attributes

QuasiNewtonOptimizerv4EstimateNewtonStepThreader::Pointer m_EstimateNewtonStepThreader
 

Friends

class QuasiNewtonOptimizerv4EstimateNewtonStepThreader
 

Additional Inherited Members

- Protected Types inherited from itk::GradientDescentOptimizerBasev4
typedef
GradientDescentOptimizerBasev4ModifyGradientByScalesThreader::IndexRangeType 
IndexRangeType
 

Member Typedef Documentation

Definition at line 68 of file itkQuasiNewtonOptimizerv4.h.

Type for an array of Hessian matrix for local support

Definition at line 82 of file itkQuasiNewtonOptimizerv4.h.

Type for Hessian matrix in the Quasi-Newton method

Definition at line 79 of file itkQuasiNewtonOptimizerv4.h.

typedef Superclass::InternalComputationValueType itk::QuasiNewtonOptimizerv4::InternalComputationValueType

Definition at line 74 of file itkQuasiNewtonOptimizerv4.h.

Definition at line 67 of file itkQuasiNewtonOptimizerv4.h.

Standard class typedefs.

Definition at line 65 of file itkQuasiNewtonOptimizerv4.h.

Definition at line 66 of file itkQuasiNewtonOptimizerv4.h.

Constructor & Destructor Documentation

itk::QuasiNewtonOptimizerv4::QuasiNewtonOptimizerv4 ( )
protected
virtual itk::QuasiNewtonOptimizerv4::~QuasiNewtonOptimizerv4 ( )
protectedvirtual
itk::QuasiNewtonOptimizerv4::QuasiNewtonOptimizerv4 ( const Self )
private

Member Function Documentation

virtual void itk::QuasiNewtonOptimizerv4::AdvanceOneStep ( void  )
protectedvirtual

Advance one step using the Quasi-Newton step. When the Newton step is invalid, the gradient step will be used.

Reimplemented from itk::GradientDescentOptimizerv4.

void itk::QuasiNewtonOptimizerv4::CombineGradientNewtonStep ( void  )
protected

Combine a gradient step with a Newton step. The Newton step will be used when it is valid. Otherwise the gradient step will be used.

virtual bool itk::QuasiNewtonOptimizerv4::ComputeHessianAndStepWithBFGS ( IndexValueType  location)
protectedvirtual

Estimate the next Hessian and step with BFGS method. The details of the method are described at http://en.wikipedia.org/wiki/BFGS_method .

virtual::itk::LightObject::Pointer itk::QuasiNewtonOptimizerv4::CreateAnother ( void  ) const
virtual

Create an object from an instance, potentially deferring to a factory. This method allows you to create an instance of an object that is exactly the same type as the referring object. This is useful in cases where an object has been cast back to a base class.

Reimplemented from itk::GradientDescentOptimizerv4.

virtual void itk::QuasiNewtonOptimizerv4::EstimateNewtonStep ( )
protectedvirtual

Estimate a Newton step

virtual void itk::QuasiNewtonOptimizerv4::EstimateNewtonStepOverSubRange ( const IndexRangeType subrange)
protectedvirtual

Estimate the quasi-newton step over a given index range.

virtual const char* itk::QuasiNewtonOptimizerv4::GetNameOfClass ( ) const
virtual

Run-time type information (and related methods).

Reimplemented from itk::GradientDescentOptimizerv4.

virtual const DerivativeType& itk::QuasiNewtonOptimizerv4::GetNewtonStep ( )
virtual

Get the most recent Newton step.

void itk::QuasiNewtonOptimizerv4::ModifyCombinedNewtonStep ( )
protected

Estimate and apply the learning rate(s) for a combined Newton step. A combined Newton step uses the Newton step by default and the gradient step when the Newton step is not valid.

The learning rate is less than 1.0 and is restricted by m_MaximumNewtonStepSizeInPhysicalUnits.

static Pointer itk::QuasiNewtonOptimizerv4::New ( )
static

Method for creation through the object factory.

void itk::QuasiNewtonOptimizerv4::operator= ( const Self )
private
virtual void itk::QuasiNewtonOptimizerv4::PrintSelf ( std::ostream &  os,
Indent  indent 
) const
protectedvirtual

Methods invoked by Print() to print information about the object including superclasses. Typically not called by the user (use Print() instead) but used in the hierarchical print process to combine the output of several classes.

Reimplemented from itk::GradientDescentOptimizerv4.

virtual void itk::QuasiNewtonOptimizerv4::ResetNewtonStep ( IndexValueType  location)
protectedvirtual

Reset the Hessian to identity matrix and the Newton step to zeros.

virtual void itk::QuasiNewtonOptimizerv4::SetMaximumIterationsWithoutProgress ( SizeValueType  _arg)
virtual

Set the maximum tolerable number of iteration without any progress

virtual void itk::QuasiNewtonOptimizerv4::SetMaximumNewtonStepSizeInPhysicalUnits ( InternalComputationValueType  _arg)
virtual

Set the maximum step size.

When SetScalesEstimator is called by user, the optimizer will compute learning rates as m_MaximumNewtonStepSizeInPhysicalUnits / m_ScalesEstimator->EstimateStepScale(newtonStep).

If SetMaximumNewtonStepSizeInPhysicalUnits is not called by user, m_MaximumNewtonStepSizeInPhysicalUnits defaults to lambda * m_ScalesEstimator->EstimateMaximumStepSize(),

where EstimateMaximumStepSize returns one voxel spacing and lambda may be in [1,5] according to our experience.

virtual void itk::QuasiNewtonOptimizerv4::StartOptimization ( bool  doOnlyInitialization = false)
virtual

Start and run the optimization

Reimplemented from itk::GradientDescentOptimizerv4.

Friends And Related Function Documentation

Definition at line 187 of file itkQuasiNewtonOptimizerv4.h.

Member Data Documentation

SizeValueType itk::QuasiNewtonOptimizerv4::m_BestIteration
protected

Definition at line 126 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_BestPosition
protected

Definition at line 125 of file itkQuasiNewtonOptimizerv4.h.

MeasureType itk::QuasiNewtonOptimizerv4::m_BestValue
protected

The best value so far and relevant information

Definition at line 124 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_CurrentPosition
protected

The information about the current step

Definition at line 115 of file itkQuasiNewtonOptimizerv4.h.

QuasiNewtonOptimizerv4EstimateNewtonStepThreader::Pointer itk::QuasiNewtonOptimizerv4::m_EstimateNewtonStepThreader
private

Threader for Newton step estimation.

Definition at line 194 of file itkQuasiNewtonOptimizerv4.h.

HessianArrayType itk::QuasiNewtonOptimizerv4::m_HessianArray
protected

The Hessian with local support

Definition at line 138 of file itkQuasiNewtonOptimizerv4.h.

SizeValueType itk::QuasiNewtonOptimizerv4::m_MaximumIterationsWithoutProgress
protected

The maximum tolerable number of iteration without any progress

Definition at line 107 of file itkQuasiNewtonOptimizerv4.h.

InternalComputationValueType itk::QuasiNewtonOptimizerv4::m_MaximumNewtonStepSizeInPhysicalUnits
protected

The maximum Quasi-Newton step size to restrict learning rates.

Definition at line 135 of file itkQuasiNewtonOptimizerv4.h.

DerivativeType itk::QuasiNewtonOptimizerv4::m_NewtonStep
protected

The Quasi-Newton step

Definition at line 129 of file itkQuasiNewtonOptimizerv4.h.

std::vector<bool> itk::QuasiNewtonOptimizerv4::m_NewtonStepValidFlags
protected

Valid flag for the Quasi-Newton steps

Definition at line 141 of file itkQuasiNewtonOptimizerv4.h.

std::string itk::QuasiNewtonOptimizerv4::m_NewtonStepWarning
protected

Warning message during Quasi-Newton step estimation

Definition at line 132 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_OptimalStep
protected

Definition at line 116 of file itkQuasiNewtonOptimizerv4.h.

DerivativeType itk::QuasiNewtonOptimizerv4::m_PreviousGradient
protected

Definition at line 121 of file itkQuasiNewtonOptimizerv4.h.

ParametersType itk::QuasiNewtonOptimizerv4::m_PreviousPosition
protected

Definition at line 120 of file itkQuasiNewtonOptimizerv4.h.

MeasureType itk::QuasiNewtonOptimizerv4::m_PreviousValue
protected

The information about the previous step

Definition at line 119 of file itkQuasiNewtonOptimizerv4.h.


The documentation for this class was generated from the following file: