[Insight-developers] CSI-ITK: HDF5 intermittent failures

Wed Jun 8 12:26:28 EDT 2011

I have investigated it. HDF writes files in blocks, and apparently they
don't zero the blocks when they're allocated.  Since the block written is
larger than the data stored, it writes out some uninitialized memory at
the end of the block.

This valgrind error occurs whether or not the test succeeds.

And as 'intermittent' as failures are, I've not been able to generate a
single failing tests, on 10 different flavors of Linux, and OSX with gcc,
clang and icc.

Like I said, if anyone wants to give me a login to a machine with a
consistent failure I'm willing to debug it on that machine.

On 6/8/11 9:59 AM, "Luis Ibanez" <luis.ibanez at kitware.com> wrote:

>Hi Kent,
>
>Valgrind is still reporting uninitialized variables in HDF5
>
>http://www.cdash.org/CDash/viewDynamicAnalysisFile.php?id=2406474
>
>They seem to be related to the process of closing a file:
>
>==19114==    at 0x4E3DC40: __write_nocancel (syscall-template.S:82)
>==19114==    by 0x7AC5BC: H5FD_sec2_write (H5FDsec2.c:799)
>==19114==    by 0x7A12B9: H5FD_write (H5FDint.c:184)
>==19114==    by 0x785D23: H5F_accum_flush (H5Faccum.c:1000)
>==19114==    by 0x77E524: H5F_flush (H5F.c:1703)
>==19114==    by 0x77BCAE: H5F_dest (H5F.c:1005)
>==19114==    by 0x77EE75: H5F_try_close (H5F.c:1929)
>==19114==    by 0x77E7CE: H5F_close (H5F.c:1774)
>==19114==    by 0xA17067: H5I_dec_ref (H5I.c:1406)
>==19114==    by 0xA171C8: H5I_dec_app_ref (H5I.c:1449)
>==19114==    by 0x77F24D: H5Fclose (H5F.c:1990)
>==19114==    by 0x7153C0: H5::H5File::close() (H5File.cpp:770)
>
>
>They seem to be worth investigating...
>
>
>    Luis
>
>
>----------------------------
>On Wed, Jun 8, 2011 at 9:44 AM, Williams, Norman K
><norman-k-williams at uiowa.edu> wrote:
>> There are a lot of builds that fail, and they all (save one) have
>> something in common.  The test writes a file, but then cannot read it --
>> either it fails to open or it fails to read the first dataset.
>>
>>
>> I verified that the tests do in fact close the file they've created.
>>
>> I also notice that for many systems that fail, very similar systems
>> succeed.
>>
>> I can add a test that attempts to read a known-good image, and see if
>>that
>> fails as well.
>>
>> I've been loading up VMs in virtual boxes that are, where possible1 as
>> close as possible to the failing systems.  So far no failures.
>>
>> As I've said before, if someone can give me an account on any of the
>> machines with failures I'll gladly investigate.
>>
>> On 6/8/11 6:47 AM, "Brad King" <brad.king at kitware.com> wrote:
>>
>>>On 06/07/2011 10:51 AM, Brad King wrote:
>>>> On 06/04/2011 12:02 PM, Bill Lorensen wrote:
>>>>> Looks like an uninitialized variable somewhere? valgrind does not
>>>>> catch them all... Maybe you should run cppcheck on the HDF5
>>>>>thirdparty
>>>>> code.
>>>>
>>>> At Kent's request I've updated the HDF5 snapshot to version 1.8.7 from
>>>> upstream.  I manually built and tested it on Linux with GCC and
>>>>Windows
>>>> with MinGW and VS 9 tools.  We should all keep an eye out for new
>>>> warnings.
>>>>
>>>> The commit range of the changes is "e1f9e848..b76ce0bd".
>>>
>>>Well, there are quite a few places that the HDF5 tests now fail:
>>>
>>>http://www.cdash.org/CDash/testSummary.php?project=2&name=itkHDF5ImageIO
>>>Te
>>>st&date=2011-06-07
>>>http://www.cdash.org/CDash/testSummary.php?project=2&name=itkHDF5ImageIO
>>>St
>>>reamingReadWriteTest&date=2011-06-07
>>>
>>>However, they pass on all the Kitware machines and on all my own builds
>>>(except itkHDF5ImageIOTest on factory.kitware fails for an unrelated
>>> reason).
>>>
>>>Can anyone with a machine where it fails investigate, please?
>>>
>>>Thanks,
>>>-Brad
>>
>>
>>
>> ________________________________
>> Notice: This UI Health Care e-mail (including attachments) is covered
>>by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is
>>confidential and may be legally privileged.  If you are not the intended
>>recipient, you are hereby notified that any retention, dissemination,
>>distribution, or copying of this communication is strictly prohibited.
>>Please reply to the sender that you have received the message in error,
>>then delete it.  Thank you.
>> ________________________________
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.html
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-developers
>>

________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged.  If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited.  Please reply to the sender that you have received the message in error, then delete it.  Thank you.
________________________________