<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">I spent a while looking at the v4
optimization framework. I can follow your reasoning until
UpdateTransformParameters is called on the transform. However, at
this state, the old scaling is still done:<br>
<br>
itkTransform.hxx:<br>
if( factor == 1.0 )<br>
{<br>
for( NumberOfParametersType k = 0; k < numberOfParameters;
k++ )<br>
{<br>
this->m_Parameters[k] += update[k];<br>
}<br>
}<br>
else<br>
{<br>
for( NumberOfParametersType k = 0; k < numberOfParameters;
k++ )<br>
{<br>
this->m_Parameters[k] += update[k] * factor;<br>
}<br>
}<br>
<br>
which makes sense, since parameters scales are an optimizer
concept that transforms know nothing about.<br>
<br>
So (if I understand correctly), the code has been shuffled around
quite a bit, but the behavior is still the same.<br>
<br>
Is there something I'm missing?<br>
<br>
joel<br>
<br>
<br>
On 07/05/2013 16:40, brian avants wrote:<br>
</div>
<blockquote
cite="mid:CABWzF4W2cs-mY+qs-UcbcftVtgy_gw32iYV5DvCEzRoFCrY27A@mail.gmail.com"
type="cite">
<div dir="ltr">also - to take away a bit of the "mystery"
surrounding v4 optimization, let's see how the gradient descent
AdvanceOneStep function works:
<div><br>
</div>
<div>
<div>void</div>
<div>GradientDescentOptimizerv4</div>
<div>::AdvanceOneStep()</div>
<div>{</div>
<div> itkDebugMacro("AdvanceOneStep");</div>
<div><br>
</div>
<div> /* Begin threaded gradient modification.</div>
<div> * Scale by gradient scales, then estimate the learning</div>
<div> * rate if options are set to (using the scaled
gradient),</div>
<div> * then modify by learning rate. The m_Gradient
variable</div>
<div> * is modified in-place. */</div>
<div> this->ModifyGradientByScales();</div>
<div> this->EstimateLearningRate();</div>
<div> this->ModifyGradientByLearningRate();</div>
<div><br>
</div>
<div> try</div>
<div> {</div>
<div> /* Pass graident to transform and let it do its own
updating */</div>
<div> this->m_Metric->UpdateTransformParameters(
this->m_Gradient );</div>
<div> }</div>
<div> catch ( ExceptionObject & )</div>
<div> {</div>
<div> this->m_StopCondition = UPDATE_PARAMETERS_ERROR;</div>
<div> this->m_StopConditionDescription <<
"UpdateTransformParameters error";</div>
<div> this->StopOptimization();</div>
<div><br>
</div>
<div> // Pass exception to caller</div>
<div> throw;</div>
<div> }</div>
<div><br>
</div>
<div> this->InvokeEvent( IterationEvent() );</div>
<div>}</div>
<div><br>
</div>
<div><br>
</div>
<div style="">i hope this does not look too convoluted.
then the base metric class does this:</div>
<div><br>
</div>
<div>
<div>template<unsigned int TFixedDimension, unsigned int
TMovingDimension, class TVirtualImage></div>
<div>void</div>
<div>ObjectToObjectMetric<TFixedDimension,
TMovingDimension, TVirtualImage></div>
<div>::UpdateTransformParameters( const DerivativeType &
derivative, ParametersValueType factor )</div>
<div>{</div>
<div> /* Rely on transform::UpdateTransformParameters to
verify proper</div>
<div> * size of derivative */</div>
<div>
this->m_MovingTransform->UpdateTransformParameters(
derivative, factor );</div>
<div>}</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div style="">so the transform parameters should be updated in
a way that is consistent with: </div>
</div>
<div style=""><br>
</div>
<div style=""><span
style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px">newPosition[j]
= currentPosition[j] + transformedGradient[j] * factor /
scales[j];</span><br
style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px">
</div>
<div style=""><span
style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px"><br>
</span></div>
<div style=""><font face="arial, sans-serif" color="#500050">factor
defaults to 1 .... anyway, as you can infer from the above
discussion, even the basic gradient descent optimizer can be
used to take " regular steps " if you want.</font></div>
<div style=""><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div><br>
</div>
brian<br>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<br>
<div class="gmail_quote">On Tue, May 7, 2013 at 10:23 AM, brian
avants <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:stnava@gmail.com" target="_blank">stnava@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">brad
<div><br>
</div>
<div>did this issue ever go up on jira? i do remember
discussing with you at a meeting. our solution is in
the v4 optimizers.</div>
<div><br>
</div>
<div>the trivial additive parameter update doesnt work in
more general cases e.g. when you need to compose
parameters with parameter updates. </div>
<div><br>
</div>
<div>to resolve this limitation, the v4 optimizers pass
the update step to the transformations </div>
<div><br>
</div>
<div>this implements the idea that " the transforms know
how to update themselves " </div>
<div><br>
</div>
<div>there are several other differences, as nick pointed
out, that reduce the need for users to experiment with
scales .</div>
<div><br>
</div>
<div>for basic scenarios like that being discussed by
joel, i prefer the conjugate gradient optimizer with
line search. </div>
<div><br>
</div>
<div>
<div>itkConjugateGradientLineSearchOptimizerv4.h</div>
<div><br>
</div>
<div>when combined with the scale estimators, this leads
to registration algorithms with very few parameters to
tune. 1 parameter if you dont consider
multi-resolution.</div>
<span class="HOEnZb"><font color="#888888">
</font></span></div>
</div>
<div class="gmail_extra"><span class="HOEnZb"><font
color="#888888"><br clear="all">
<div>
<div><br>
</div>
brian<br>
<div><br>
</div>
<div><br>
</div>
</div>
</font></span>
<div>
<div class="h5">
<br>
<br>
<div class="gmail_quote">On Tue, May 7, 2013 at 9:27
AM, Nick Tustison <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:ntustison@gmail.com"
target="_blank">ntustison@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Brad,<br>
<br>
I certainly don't disagree with Joel's findings.
It seems like a<br>
good fix which should be put up on gerrit. There
were several<br>
components that we kept in upgrading the
registration framework.<br>
The optimizers weren't one of them.<br>
<br>
Also, could you elaborate a bit more on the
"convoluted" aspects<br>
of parameter advancement? There's probably a
reason for it and<br>
we could explain why.<br>
<span><font color="#888888"><br>
Nick<br>
</font></span>
<div>
<div><br>
<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>