[Insight-users] [Vxl-users] L-BFGS behaves differently in release with debug and release without debug

[Sureerat Reaungamornrat]

Dear All,

   Thank you very much for your suggestions.

    If we link the code directly to VXL and run the code in Debug and
Release modes,
we get the difference in the decimal fractions in hundredths. I think this
might due to
the difference in compiler mode as you suggested.

    If we link the code to ITK VXL and run the code in Debug and Release
modes,
we get the difference in decimal power 1 such as 2 and 3. I think this
might be because
we do not link the code correctly  or the version of VXL.

    For both cases, I will track down the difference between running in
Debug and in
Release. I will look into a book as Ian Scott suggested as well.

    Any further suggestions are very welcome and appreciate.

Thank you very much,

Best regards,

Sureerat


On Fri, Oct 5, 2012 at 4:54 AM, Ian Scott <scottim at imorphics.com> wrote:

> Option 1: There is a bug in your code. Solution: track down the difference
> between the two runs and fix bug.
>
> Option 2: This is much more interesting. To start, you need to clarify how
> different the results are. A complicated numerical calculation like an
> optimisation will almost always give you a different numerical result if
> you change the optimisation settings of your compiler.
>
> If the result is massively different, there is the possibility that your
> cost function has a nearby singular. The compiler-optimised code will be
> running the maths in a slightly different order, with slightly different
> numerical round-off errors which get magnified by the high condition number
> near the singularity to give significantly different results. The solution
> here is to fix your cost function to be more stable.
>
> Sometimes your cost function isn't inherently singular, but its
> implementation is, which is harder to diagnose, but easier to fix. For
> further discussion, see any book on numerical precision, or for a brief
> introduction see Goldberg's "What every computer scientist should know
> about floating point" which is available all over the web.
>
> It is also possible (but less likely) that the Hessian of your cost
> function is nearly singular, leading to a similar divergence. This
> manifests as certain directions having almost zero gradient around the
> minimum. However, I would have expected the implementation of  BFGS to be
> largely immune to this, but you never know.
>
> You can control how much the optimiser rearranges your maths code, or
> whether it stores a result in extra precision contrary to the advice of
> IEEE 754 using the /fp:string and related compiler options. This might help
> unify the two results, but if your cost function is singular, then the
> result will be inherently unreliable.
>
> Ian.
>
>
>
> On 04/10/2012 23:17, Sureerat Reaungamornrat wrote:
>
>> Dear All,
>>     We have a question regarding 'v3p_netlib_lbfgs_' function which was
>> included in ITK version 4.0.0 in the file
>>
>> path-to-ITK-source\Modules\**ThirdParty\VNL\src\vxl\v3p\**
>> netlib\opt\lbfgs.h
>>     We are not sure whether we should put this question to ITK or VXL
>> mailing-lists.
>>
>>     Our problem is that we use v3p_netlib_lbfgs_in our codeto optimize
>>
>> the same function and same data but
>> we get different results when we ran our code in different
>> modes:Release and Debug. We tested our code in Window 7
>> using 64-bit Visual Studio 2008.
>>
>>     Would anyone have any suggestions? Any suggestions are very
>> appreciated.
>>
>> Thank you very much,
>>
>> Best regards,
>>
>> Sureerat
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------**------------------------------**
>> ------------------
>> Don't let slow site performance ruin your business. Deploy New Relic APM
>> Deploy New Relic app performance management and know exactly
>> what is happening inside your Ruby, Python, PHP, Java, and .NET app
>> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
>> http://p.sf.net/sfu/newrelic-**dev2dev<http://p.sf.net/sfu/newrelic-dev2dev>
>>
>>
>>
>> ______________________________**_________________
>> Vxl-users mailing list
>> Vxl-users at lists.sourceforge.**net <Vxl-users at lists.sourceforge.net>
>> https://lists.sourceforge.net/**lists/listinfo/vxl-users<https://lists.sourceforge.net/lists/listinfo/vxl-users>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20121005/96d170aa/attachment.htm>