No subject
Sun Apr 3 09:54:08 EDT 2011
Kitware Robot has submitted this change and it was merged.
Change subject: COMP: Intermittent test failure
......................................................................
COMP: Intermittent test failure
itkInitializationBiasedParticleSwarmOptimizerTest is failing
intermittenly. This patch enables more verbose (but not too verbose)
output to help track down the problem.
It seems the test was failing a tolerance check on the results of the
optimization. This was due to hitting the max number of iterations
stop condition rather than the parameter tolerance stop
condition. This patch increases the max number of iterations, forcing
a parameter tolerance stop condition.
It is not clear whether this is a deterministic algorithm. Should we
expect consistent answers for each run?
This patch does not address the deterministic issue.
Change-Id: Ie2110c51639f491cb7b8a6287b0eaeb1370c50bd
---
M Modules/Numerics/Optimizers/test/itkInitializationBiasedParticleSwarmOpti=
mizerTest.cxx
1 file changed, 45 insertions(+), 90 deletions(-)
Approvals:
Hans J. Johnson: Verified; Looks good to me, approved
Bill Lorensen: Verified; Looks good to me, but someone else must approve
--
To view, visit <http://review.source.kitware.com/2793> http://review.source=
.kitware.com/2793
To unsubscribe, visit <http://review.source.kitware.com/settings> http://re=
view.source.kitware.com/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie2110c51639f491cb7b8a6287b0eaeb1370c50bd
Gerrit-PatchSet: 2
Gerrit-Project: ITK
Gerrit-Branch: master
Gerrit-Owner: Bill Lorensen <<mailto:bill.lorensen at gmail.com>bill.lorensen@=
gmail.com<mailto:bill.lorensen at gmail.com>>
Gerrit-Reviewer: Bill Lorensen <<mailto:bill.lorensen at gmail.com>bill.lorens=
en at gmail.com<mailto:bill.lorensen at gmail.com>>
Gerrit-Reviewer: Hans J. Johnson <<mailto:hans-johnson at uiowa.edu>hans-johns=
on at uiowa.edu<mailto:hans-johnson at uiowa.edu>>
Gerrit-Reviewer: Matt McCormick <<mailto:matt.mccormick at kitware.com>matt.mc=
cormick at kitware.com<mailto:matt.mccormick at kitware.com>>
Gerrit-Reviewer: Ziv Yaniv <<mailto:zyaniv at childrensnational.org>zyaniv at chi=
ldrensnational.org<mailto:zyaniv at childrensnational.org>>
________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by th=
e Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidenti=
al and may be legally privileged. If you are not the intended recipient, y=
ou are hereby notified that any retention, dissemination, distribution, or =
copying of this communication is strictly prohibited. Please reply to the =
sender that you have received the message in error, then delete it. Thank =
you.
________________________________
________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by th=
e Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidenti=
al and may be legally privileged. If you are not the intended recipient, y=
ou are hereby notified that any retention, dissemination, distribution, or =
copying of this communication is strictly prohibited. Please reply to the =
sender that you have received the message in error, then delete it. Thank =
you.
________________________________
--_000_0AAF9DB4B89049B5B27F248C166B67B8uiowaedu_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
</head>
<body bgcolor=3D"#FFFFFF">
<div>Agreed.<br>
<br>
Sent from my iPhone</div>
<div><br>
On Sep 16, 2011, at 7:24 AM, "Bill Lorensen" <<a href=3D"mailt=
o:bill.lorensen at gmail.com">bill.lorensen at gmail.com</a>> wrote:<br>
<br>
</div>
<div></div>
<blockquote type=3D"cite">
<div>This failures look like a bug or parameter issue to me.<br>
<br>
The large number of earlier failures were because the optimizer was termina=
ting on max iterations. My patch increased the max so that the termination =
should be due to function tolerance and parameter difference tolerance.<br>
<br>
Perhaps the function tolerance needs to be reduced.<br>
<br>
I think we'll have to let Ziv look at this.<br>
<br>
Bill<br>
<br>
<br>
<div class=3D"gmail_quote">On Fri, Sep 16, 2011 at 7:25 AM, Johnson, Hans J=
<span dir=3D"ltr">
<<a href=3D"mailto:hans-johnson at uiowa.edu"></a><a href=3D"mailto:hans-jo=
hnson at uiowa.edu">hans-johnson at uiowa.edu</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0pt 0pt 0pt 0.8ex; border=
-left:1px solid rgb(204,204,204); padding-left:1ex">
<div style=3D"word-wrap:break-word; color:rgb(0,0,0); font-size:14px; font-=
family:Calibri,sans-serif">
<div>Bill,</div>
<div><br>
</div>
<div>There were fewer failures, but there a still two similar failures:</di=
v>
<div><br>
</div>
<div><span style=3D"background-color:rgb(255,255,255); font-family:Arial,He=
lvetica,sans-serif">
<pre>Reason for stopping optimization:
InitializationBiasedParticleSwarmOptimizer: successfuly converged after 95=
iterations
Known parameters =3D [2, -2] Estimated parameters =3D [2.1757, -2.09753=
]
[Test 2 FAILURE]</pre>
<pre><br></pre>
<pre>Hans</pre>
<pre><br></pre>
</span></div>
<div><br>
</div>
<span>
<div style=3D"font-family:Calibri; font-size:11pt; text-align:left; color:b=
lack; border-width:1pt medium medium; border-style:solid none none; padding=
:3pt 0in 0in">
<span style=3D"font-weight:bold">From: </span>Bill Lorensen <<a href=3D"=
mailto:bill.lorensen at gmail.com" target=3D"_blank"></a><a href=3D"mailto:bil=
l.lorensen at gmail.com">bill.lorensen at gmail.com</a>><br>
<span style=3D"font-weight:bold">Date: </span>Thu, 15 Sep 2011 23:15:30 -04=
00<br>
<span style=3D"font-weight:bold">To: </span>"Yaniv, Ziv" <<a h=
ref=3D"mailto:ZYaniv at childrensnational.org" target=3D"_blank"></a><a href=
=3D"mailto:ZYaniv at childrensnational.org">ZYaniv at childrensnational.org</a>&g=
t;<br>
<span style=3D"font-weight:bold">Cc: </span>Matt McCormick <<a href=3D"m=
ailto:matt.mccormick at kitware.com" target=3D"_blank"></a><a href=3D"mailto:m=
att.mccormick at kitware.com">matt.mccormick at kitware.com</a>>, Hans Johnson=
<<a href=3D"mailto:hans-johnson at uiowa.edu" target=3D"_blank"></a><a hre=
f=3D"mailto:hans-johnson at uiowa.edu">hans-johnson at uiowa.edu</a>>,
ITK <<a href=3D"mailto:insight-developers at itk.org" target=3D"_blank"></=
a><a href=3D"mailto:insight-developers at itk.org">insight-developers at itk.org<=
/a>><br>
<span style=3D"font-weight:bold">Subject: </span>Re: Change in ITK[master]:=
COMP: Intermittent test failure<br>
</div>
<div>
<div></div>
<div class=3D"h5">
<div><br>
</div>
Ziv,<br>
<br>
The problem with the test was that sometimes it was terminating on iteratio=
ns. When this happened, the tolerance was not always satisfied.<br>
<br>
I upped the number of max iterations and I think the test should pass conos=
istently.<br>
<br>
We'll see.<br>
<br>
Usually for non-deterministic algorithms the test seeds the random number g=
enerator with a fixed number.<br>
<br>
<div class=3D"gmail_quote">On Thu, Sep 15, 2011 at 8:43 PM, Yaniv, Ziv <spa=
n dir=3D"ltr">
<<a href=3D"mailto:ZYaniv at childrensnational.org" target=3D"_blank"></a><=
a href=3D"mailto:ZYaniv at childrensnational.org">ZYaniv at childrensnational.org=
</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0pt 0pt 0pt 0.8ex; border=
-left:1px solid rgb(204,204,204); padding-left:1ex">
<u></u>
<div>
<p><font size=3D"2">Hi Bill,<br>
<br>
I was traveling and did not have internet connection for a day. Back online=
. The PSO family of algorithms is stochastic and does not ensure convergenc=
e to the local minimum. My initial implementation of the tests dealt with t=
his by running them multiple times
and if a certain percentage was successful then the test was declared succ=
essful. Perhaps this is not the best way to test such algorithms but I do n=
ot have a better idea. The nice feature of PSO is that it efficiently explo=
res a rugged parameter space. A
common practice is to run the PSO and then initialize a standard optimizer=
with the previous result, ensuring converges to the local minimum.<br>
<br>
What approach does ITK use to tests non-deterministic algorithms?<br>
<br>
&nb=
sp; regards<br>
&nb=
sp; Ziv<br>
<br>
--<br>
Ziv Yaniv, PhD.,<br>
Principal Investigator,<br>
The Sheikh Zayed Institute for Pediatric Surgical Innovation<br>
Children's National Medical Center<br>
6th Floor Main Hospital, Room M7740<br>
111 Michigan Ave., N.W<br>
Washington, D.C 20010<br>
Phone: <a href=3D"tel:202-476-1288" value=3D"+12024761288" target=3D"_b=
lank">202-476-1288</a><br>
email: <a href=3D"mailto:zyaniv at childrensnational.org" target=3D"_blank"></=
a><a href=3D"mailto:zyaniv at childrensnational.org">zyaniv at childrensnational.=
org</a></font></p>
<div>
<div></div>
<div><font size=3D"2"><br>
<br>
<br>
<br>
<br>
-----Original Message-----<br>
From: Code Review [<a href=3D"mailto:review at kitware.com" target=3D"_blank">=
</a><a href=3D"mailto:review at kitware.com">mailto:review at kitware.com</a>]<br=
>
Sent: Thu 9/15/2011 6:25 PM<br>
To: Bill Lorensen<br>
Cc: Matt McCormick; Hans J. Johnson; Yaniv, Ziv<br>
Subject: Change in ITK[master]: COMP: Intermittent test failure<br>
<br>
More information about the Insight-developers
mailing list