No subject

Mon Aug 10 18:29:21 EDT 2009

"At each iteration a limited memory BFGS approximation of the Hessian is
updated.
 This limited memory matrix is used to define a quadratic model of the
objective function 'f'.
 A search direction is then computed using a two-stage approach: first, the
gradient projection
 method is used to identify a set of active variables, i.e. variables taht
will be held at their bound;
 then the quadratic model is approximately minimized with respect to the
free variables. The
 search direction is defined to be the vector leading from the current
iterate to this approximate
 minimizer. Finally a line search is performed along the search direction
using the subroutine
 described in [17]. A novel feature of the algorithm is that the limited
memory BFGS matrices are
 represented in a compact form [7] that is efficient for bound constrained
problems."

BTW:
We have recently observed that for practical purposes
the LBFGSB optimizer doesn't provide any advantage with
respect to a simple gradient descent optimizer. The most
recent BSpline examples added to the toolkit use simply the
RegularStepGradientDescentOptimizer.

Regards,

     Luis

---------------------------------------------------------------------
On Mon, Sep 7, 2009 at 9:01 PM, Serena Fabbri <fabbri at u.washington.edu>wrote:

> Hi All,
> I write to ask some advice about the parameter of LBFGSB optimizer.
> I am registrating MRI and CT of the brain with BSpline.
>
> In particular I am using DeformableRegistration8 and I'd like to understand
> the strategy for the Upper and Lower Bound (in the example they are 0, it
> means the BSpline parameters have only 1 constraint).
> They are the max and minimun displacement that the parameters can assume.
> Is it correct?
> How is the best strategy to set them?
>
> Besides,
>
> What is the difference between SetMaximumNumberOfEvaluations
> SetMaximumNumberOfCorrections  ?
>
> Are they correlated with the update of hessian matrix?
>
> Any suggestions will be appreciated.
>
> thanks.
>
> Serena.
>
>
> _____________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-users
>

--00151773e42843a9c30473250e6a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Serena, Yes, the upper and lower bounds will be the extremes =
of the values that EACH component of the deformation vectors at EACH nod=
e, can assume. Note that due to symmetry you probably will have t=
o set those bounds 
to pairs=A0 ( -T, +T ). For details on the inner working of the =
LBFGSB optimizer, please refer to the following emails from the mailing =
list archives: <a href=3D"http://www.itk.org/pipermail/insight-users=
/2009-July/031468.html">http://www.itk.org/pipermail/insight-users/2009-Jul=
y/031468.html</a> 
 LBFGSB: <a href=3D"http://www.ece.northwestern.edu/%7Enocedal/lb=
fgsb.html" target=3D"_blank">http://www.ece.northwestern.edu/~nocedal/lbfgs=
b.html</a> 
References

<ul><li>
R. H. Byrd, P. Lu and J. Nocedal. <a href=3D"http://www.ece.northwestern.ed=
u/%7Enocedal/PSfiles/limited.ps.gz" target=3D"_blank">A
Limited Memory Algorithm for Bound Constrained Optimization</a>, (1995),
SIAM Journal on Scientific and Statistical Computing , 16, 5, pp. 1190-1208=
. <a href=3D"http://www.ece.northwestern.edu/%7Enocedal/PSfiles/limited.=
ps.gz" target=3D"_blank">http://www.ece.northwestern.edu/~nocedal/PSfiles/l=
imited.ps.gz</a> 
 
</li><li>
C. Zhu, R. H. Byrd and J. Nocedal. <a href=3D"http://www.ece.northwestern.e=
du/%7Enocedal/PSfiles/lbfgsb.ps.gz" target=3D"_blank">L-BFGS-B:
Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained
optimization</a> (1997), ACM Transactions on Mathematical Software, Vol
23, Num. 4, pp. 550 - 560. <a href=3D"http://www.ece.northwestern.edu/%7=
Enocedal/PSfiles/lbfgsb.ps.gz" target=3D"_blank">http://www.ece.northwester=
n.edu/~nocedal/PSfiles/lbfgsb.ps.gz</a> 
</li></ul>
 From the second reference: &quot;At each iteration a limited mem=
ory BFGS approximation of the Hessian is updated. =A0This limited memory=
 matrix is used to define a quadratic model of the objective function &#39;=
f&#39;. 
=A0A search direction is then computed using a two-stage approach: first, t=
he gradient projection 
=A0method is used to identify a set of active variables, i.e. variables tah=
t will be held at their bound; =A0then the quadratic model is approximat=
ely minimized with respect to the free variables. The =A0search directi=
on is defined to be the vector leading from the current iterate to this app=
roximate

=A0minimizer. Finally a line search is performed along the search direction=
 using the subroutine<br>=A0described in [17]. A novel feature of the algor=
ithm is that the limited memory BFGS matrices are<br>=A0represented in a co=
mpact form [7] that is efficient for bound constrained problems.&quot;<br>
<br><br><br>BTW: <br>We have recently observed that for practical purposes<=
br>the LBFGSB optimizer doesn&#39;t provide any advantage with<br>respect t=
o a simple gradient descent optimizer. The most<br>recent BSpline examples =
added to the toolkit use simply the<br>
RegularStepGradientDescentOptimizer.<br><br><br><br>Regards,<br><br><br>=A0=
=A0=A0=A0 Luis<br><br><br><br>---------------------------------------------=
------------------------<br><div class=3D"gmail_quote">On Mon, Sep 7, 2009 =
at 9:01 PM, Serena Fabbri <span dir=3D"ltr">&lt;<a href=3D"mailto:fabbri at u.=
washington.edu">fabbri at u.washington.edu</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi All,<br>
I write to ask some advice about the parameter of LBFGSB optimizer.<br>
I am registrating MRI and CT of the brain with BSpline.<br>
<br>
In particular I am using DeformableRegistration8 and I&#39;d like to unders=
tand the strategy for the Upper and Lower Bound (in the example they are 0,=
 it means the BSpline parameters have only 1 constraint).<br>
They are the max and minimun displacement that the parameters can assume. I=
s it correct?<br>
How is the best strategy to set them?<br>
<br>
Besides,<br>
<br>
What is the difference between SetMaximumNumberOfEvaluations<br>
SetMaximumNumberOfCorrections =A0?<br>
<br>
Are they correlated with the update of hessian matrix?<br>
<br>
Any suggestions will be appreciated.<br>
<br>
thanks.<br>
<br>
Serena.<br>
<br>
<br>
_____________________________________<br>
Powered by <a href=3D"http://www.kitware.com" target=3D"_blank">www.kitware=
.com</a><br>
<br>
Visit other Kitware open-source projects at<br>
<a href=3D"http://www.kitware.com/opensource/opensource.html" target=3D"_bl=
ank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the ITK FAQ at: <a href=3D"http://w=
ww.itk.org/Wiki/ITK_FAQ" target=3D"_blank">http://www.itk.org/Wiki/ITK_FAQ<=
/a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href=3D"http://www.itk.org/mailman/listinfo/insight-users" target=3D"_bl=
ank">http://www.itk.org/mailman/listinfo/insight-users</a><br>
</blockquote></div><br>

--00151773e42843a9c30473250e6a--