<div dir="ltr">yes - there is something you are missing.   read the code below: <div><br></div><div><div style="font-family:arial,sans-serif;font-size:13px">  /* Begin threaded gradient modification.</div><div style="font-family:arial,sans-serif;font-size:13px">




   * Scale by gradient scales, then estimate the learning</div><div style="font-family:arial,sans-serif;font-size:13px">   * rate if options are set to (using the scaled gradient),</div><div style="font-family:arial,sans-serif;font-size:13px">




   * then modify by learning rate. The m_Gradient variable</div><div style="font-family:arial,sans-serif;font-size:13px">   * is modified in-place. */</div><div style="font-family:arial,sans-serif;font-size:13px">  this-&gt;ModifyGradientByScales();</div>




<div style="font-family:arial,sans-serif;font-size:13px">  this-&gt;EstimateLearningRate();</div><div style="font-family:arial,sans-serif;font-size:13px">  this-&gt;ModifyGradientByLearningRate();</div></div><div><br></div>




<div>the call to  <span style="font-family:arial,sans-serif;font-size:13px">  this-&gt;ModifyGradientByScales()</span><span style="font-family:arial,sans-serif;font-size:13px">;   changes the gradient according the scales, as the name suggests.   the v4 optimizers all behave in this general manner although this is taken from the gradient descent class.</span></div>




<div><br></div><div>so - the transform expects that the update was already modified by the scales so that the only thing the transform needs to do ( if anything at all) is multiply by a scalar</div><div><br></div>
<div>also, you are looking at the base class which is only used if the derived class did not implement UpdateTransformParameters.  for instance, the GaussianDisplacementField transform will also smooth the parameters when this function is called.</div>

<div><br></div><div>is this clear enough?</div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div><br></div>brian<br><div><br></div><div><br></div></div>
<br><br><div class="gmail_quote">On Tue, May 7, 2013 at 11:59 AM, Joël Schaerer <span dir="ltr">&lt;<a href="mailto:joel.schaerer@gmail.com" target="_blank">joel.schaerer@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">


  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    <div>I spent a while looking at the v4
      optimization framework. I can follow your reasoning until
      UpdateTransformParameters is called on the transform. However, at
      this state, the old scaling is still done:<br>
      <br>
      itkTransform.hxx:<br>
        if( factor == 1.0 )<br>
          {<br>
          for( NumberOfParametersType k = 0; k &lt; numberOfParameters;
      k++ )<br>
            {<br>
            this-&gt;m_Parameters[k] += update[k];<br>
            }<br>
          }<br>
        else<br>
          {<br>
          for( NumberOfParametersType k = 0; k &lt; numberOfParameters;
      k++ )<br>
            {<br>
            this-&gt;m_Parameters[k] += update[k] * factor;<br>
            }<br>
          }<br>
      <br>
      which makes sense, since parameters scales are an optimizer
      concept that transforms know nothing about.<br>
      <br>
      So (if I understand correctly), the code has been shuffled around
      quite a bit, but the behavior is still the same.<br>
      <br>
      Is there something I&#39;m missing?<br>
      <br>
      joel<div><div class="h5"><br>
      <br>
      <br>
      On 07/05/2013 16:40, brian avants wrote:<br>
    </div></div></div><div><div class="h5">
    <blockquote type="cite">
      <div dir="ltr">also - to take away a bit of the &quot;mystery&quot;
        surrounding v4 optimization, let&#39;s see how the gradient descent
        AdvanceOneStep function works:
        <div><br>
        </div>
        <div>
          <div>void</div>
          <div>GradientDescentOptimizerv4</div>
          <div>::AdvanceOneStep()</div>
          <div>{</div>
          <div>  itkDebugMacro(&quot;AdvanceOneStep&quot;);</div>
          <div><br>
          </div>
          <div>  /* Begin threaded gradient modification.</div>
          <div>   * Scale by gradient scales, then estimate the learning</div>
          <div>   * rate if options are set to (using the scaled
            gradient),</div>
          <div>   * then modify by learning rate. The m_Gradient
            variable</div>
          <div>   * is modified in-place. */</div>
          <div>  this-&gt;ModifyGradientByScales();</div>
          <div>  this-&gt;EstimateLearningRate();</div>
          <div>  this-&gt;ModifyGradientByLearningRate();</div>
          <div><br>
          </div>
          <div>  try</div>
          <div>    {</div>
          <div>    /* Pass graident to transform and let it do its own
            updating */</div>
          <div>    this-&gt;m_Metric-&gt;UpdateTransformParameters(
            this-&gt;m_Gradient );</div>
          <div>    }</div>
          <div>  catch ( ExceptionObject &amp; )</div>
          <div>    {</div>
          <div>    this-&gt;m_StopCondition = UPDATE_PARAMETERS_ERROR;</div>
          <div>    this-&gt;m_StopConditionDescription &lt;&lt;
            &quot;UpdateTransformParameters error&quot;;</div>
          <div>    this-&gt;StopOptimization();</div>
          <div><br>
          </div>
          <div>    // Pass exception to caller</div>
          <div>    throw;</div>
          <div>    }</div>
          <div><br>
          </div>
          <div>  this-&gt;InvokeEvent( IterationEvent() );</div>
          <div>}</div>
          <div><br>
          </div>
          <div><br>
          </div>
          <div>i hope this does not look too convoluted.
             then the base metric class does this:</div>
          <div><br>
          </div>
          <div>
            <div>template&lt;unsigned int TFixedDimension, unsigned int
              TMovingDimension, class TVirtualImage&gt;</div>
            <div>void</div>
            <div>ObjectToObjectMetric&lt;TFixedDimension,
              TMovingDimension, TVirtualImage&gt;</div>
            <div>::UpdateTransformParameters( const DerivativeType &amp;
              derivative, ParametersValueType factor )</div>
            <div>{</div>
            <div>  /* Rely on transform::UpdateTransformParameters to
              verify proper</div>
            <div>   * size of derivative */</div>
            <div> 
              this-&gt;m_MovingTransform-&gt;UpdateTransformParameters(
              derivative, factor );</div>
            <div>}</div>
          </div>
          <div><br>
          </div>
          <div><br>
          </div>
          <div>so the transform parameters should be updated in
            a way that is consistent with: </div>
        </div>
        <div><br>
        </div>
        <div><span style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px">newPosition[j]
            = currentPosition[j] + transformedGradient[j] * factor /
            scales[j];</span><br style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px">
        </div>
        <div><span style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px"><br>
          </span></div>
        <div><font face="arial, sans-serif" color="#500050">factor
            defaults to 1 ....  anyway, as you can infer from the above
            discussion, even the basic gradient descent optimizer can be
            used to take &quot; regular steps &quot;  if you want.</font></div>
        <div><br>
        </div>
      </div>
      <div class="gmail_extra"><br clear="all">
        <div>
          <div><br>
          </div>
          brian<br>
          <div><br>
          </div>
          <div><br>
          </div>
        </div>
        <br>
        <br>
        <div class="gmail_quote">On Tue, May 7, 2013 at 10:23 AM, brian
          avants <span dir="ltr">&lt;<a href="mailto:stnava@gmail.com" target="_blank">stnava@gmail.com</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">brad
              <div><br>
              </div>
              <div>did this issue ever go up on jira?  i do remember
                discussing with you at a meeting.   our solution is in
                the v4 optimizers.</div>
              <div><br>
              </div>
              <div>the trivial additive parameter update doesnt work in
                more general cases e.g. when you need to compose
                parameters with parameter updates. </div>
              <div><br>
              </div>
              <div>to resolve this limitation, the v4 optimizers pass
                the update step to the transformations </div>
              <div><br>
              </div>
              <div>this implements the idea that  &quot; the transforms know
                how to update themselves &quot; </div>
              <div><br>
              </div>
              <div>there are several other differences, as nick pointed
                out, that reduce the need for users to experiment with
                scales .</div>
              <div><br>
              </div>
              <div>for basic scenarios like that being discussed by
                joel, i prefer the conjugate gradient optimizer with
                line search. </div>
              <div><br>
              </div>
              <div>
                <div>itkConjugateGradientLineSearchOptimizerv4.h</div>
                <div><br>
                </div>
                <div>when combined with the scale estimators, this leads
                  to registration algorithms with very few parameters to
                  tune.   1 parameter if you dont consider
                  multi-resolution.</div>
                <span><font color="#888888">
                  </font></span></div>
            </div>
            <div class="gmail_extra"><span><font color="#888888"><br clear="all">
                  <div>
                    <div><br>
                    </div>
                    brian<br>
                    <div><br>
                    </div>
                    <div><br>
                    </div>
                  </div>
                </font></span>
              <div>
                <div>
                  <br>
                  <br>
                  <div class="gmail_quote">On Tue, May 7, 2013 at 9:27
                    AM, Nick Tustison <span dir="ltr">&lt;<a href="mailto:ntustison@gmail.com" target="_blank">ntustison@gmail.com</a>&gt;</span>
                    wrote:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      Hi Brad,<br>
                      <br>
                      I certainly don&#39;t disagree with Joel&#39;s findings.
                       It seems like a<br>
                      good fix which should be put up on gerrit.  There
                      were several<br>
                      components that we kept in upgrading the
                      registration framework.<br>
                      The optimizers weren&#39;t one of them.<br>
                      <br>
                      Also, could you elaborate a bit more on the
                      &quot;convoluted&quot; aspects<br>
                      of parameter advancement?  There&#39;s probably a
                      reason for it and<br>
                      we could explain why.<br>
                      <span><font color="#888888"><br>
                          Nick<br>
                        </font></span>
                      <div>
                        <div><br>
                          <br>
                        </div>
                      </div>
                    </blockquote>
                  </div>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div>