Dear All,<div><br></div><div> Thank you very much for your suggestions. </div><div><br></div><div> If we link the code directly to VXL and run the code in Debug and Release modes,</div><div>we get the difference in the decimal fractions in hundredths. I think this might due to </div>
<div>the difference in compiler mode as you suggested. </div><div><br></div><div> If we link the code to ITK VXL and run the code in Debug and Release modes,</div><div>we get the difference in decimal power 1 such as 2 and 3. I think this might be because</div>
<div>we do not link the code correctly or the version of VXL.</div><div><br></div><div> For both cases, I will track down the difference between running in Debug and in </div><div>Release. I will look into a book as Ian Scott suggested as well.</div>
<div><br></div><div> Any further suggestions are very welcome and appreciate.</div><div><br>Thank you very much,</div><div><br></div><div>Best regards,</div><div><br></div><div>Sureerat</div><div><br></div><div><br><div class="gmail_quote">
On Fri, Oct 5, 2012 at 4:54 AM, Ian Scott <span dir="ltr"><<a href="mailto:scottim@imorphics.com" target="_blank">scottim@imorphics.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Option 1: There is a bug in your code. Solution: track down the difference between the two runs and fix bug.<br>
<br>
Option 2: This is much more interesting. To start, you need to clarify how different the results are. A complicated numerical calculation like an optimisation will almost always give you a different numerical result if you change the optimisation settings of your compiler.<br>
<br>
If the result is massively different, there is the possibility that your cost function has a nearby singular. The compiler-optimised code will be running the maths in a slightly different order, with slightly different numerical round-off errors which get magnified by the high condition number near the singularity to give significantly different results. The solution here is to fix your cost function to be more stable.<br>
<br>
Sometimes your cost function isn't inherently singular, but its implementation is, which is harder to diagnose, but easier to fix. For further discussion, see any book on numerical precision, or for a brief introduction see Goldberg's "What every computer scientist should know about floating point" which is available all over the web.<br>
<br>
It is also possible (but less likely) that the Hessian of your cost function is nearly singular, leading to a similar divergence. This manifests as certain directions having almost zero gradient around the minimum. However, I would have expected the implementation of BFGS to be largely immune to this, but you never know.<br>
<br>
You can control how much the optimiser rearranges your maths code, or whether it stores a result in extra precision contrary to the advice of IEEE 754 using the /fp:string and related compiler options. This might help unify the two results, but if your cost function is singular, then the result will be inherently unreliable.<br>
<br>
Ian.<div class="im"><br>
<br>
<br>
On 04/10/2012 23:17, Sureerat Reaungamornrat wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
Dear All,<br>
We have a question regarding 'v3p_netlib_lbfgs_' function which was<br>
included in ITK version 4.0.0 in the file<br>
<br>
path-to-ITK-source\Modules\<u></u>ThirdParty\VNL\src\vxl\v3p\<u></u>netlib\opt\lbfgs.h<br>
We are not sure whether we should put this question to ITK or VXL<br>
mailing-lists.<br>
<br></div>
Our problem is that we use v3p_netlib_lbfgs_in our codeto optimize<div class="im"><br>
the same function and same data but<br>
we get different results when we ran our code in different<br>
modes:Release and Debug. We tested our code in Window 7<br>
using 64-bit Visual Studio 2008.<br>
<br>
Would anyone have any suggestions? Any suggestions are very appreciated.<br>
<br>
Thank you very much,<br>
<br>
Best regards,<br>
<br>
Sureerat<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br></div>
------------------------------<u></u>------------------------------<u></u>------------------<br>
Don't let slow site performance ruin your business. Deploy New Relic APM<br>
Deploy New Relic app performance management and know exactly<br>
what is happening inside your Ruby, Python, PHP, Java, and .NET app<br>
Try New Relic at no cost today and get our sweet Data Nerd shirt too!<br>
<a href="http://p.sf.net/sfu/newrelic-dev2dev" target="_blank">http://p.sf.net/sfu/newrelic-<u></u>dev2dev</a><br>
<br>
<br>
<br>
______________________________<u></u>_________________<br>
Vxl-users mailing list<br>
<a href="mailto:Vxl-users@lists.sourceforge.net" target="_blank">Vxl-users@lists.sourceforge.<u></u>net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/vxl-users" target="_blank">https://lists.sourceforge.net/<u></u>lists/listinfo/vxl-users</a><br>
<br>
</blockquote>
<br>
</blockquote></div><br></div>