[Insight-users] Streaming Writer Broken?

Kevin H. Hobbs hobbsk at ohiou.edu
Fri Nov 19 13:00:44 EST 2010 

I'm trying to stream the full Visible Woman Head data, but it looks like
the something is still trying to allocate too much memory. Without
ulimit I just get "swap-death"

I run :

ulimit -v 6291456 # I don't want the OS to start swapping.
./StreamedVectorGradientMagnitudeImageFilter \
  ~/bubbles/VisibleWomanHead/RawRGB/VisibleWomanHeadFull.mhd \
  VisibleWomanHeadFull_GradMag.mhd \
  100 # The number of streamed pieces

100 pieces should keep me well below the 6 GiB limit yet :

VectorCastImageFilter 0.99
VectorCastImageFilter 1
VectorCastImageFilter 1

itk::MemoryAllocationError (0x7fc34c000ad0)
Location: "TElement* itk::ImportImageContainer<TElementIdentifier,
TElement>::AllocateElements(TElementIdentifier) const [with
TElementIdentifier = long unsigned int, TElement = itk::Vector<float, 3u>]"
File: /home/kevin/kitware/ITK/Code/Common/itkImportImageContainer.txx
Line: 191
Description: Failed to allocate memory for image.

So I tried it again without the 6GiB limit and in one piece :

./StreamedVectorGradientMagnitudeImageFilter \
  ~/bubbles/VisibleWomanHead/RawRGB/VisibleWomanHeadFull.mhd \
  VisibleWomanHeadFull_GradMag.mhd \
  1

Which just squeaks in under the available 12GB, swaps very little, and
completes successfully.

With my brand new 1.3GiB scalar image I tried to simply stream read and
write with no filter.

./StreamedScalarImageReadWrite \
  VisibleWomanHeadFull_GradMag.mhd \
  VisibleWomanHeadFull_GradMag_100.mhd \
  100 # The number of streamed pieces

I got this weirdness :

ImageFileWriter 0.99
WARNING: In /home/kevin/kitware/ITK/Code/IO/itkMetaImageIO.cxx, line 1453
MetaImageIO (0x10a0200): Unsupported metaData item
ITK_CoordinateOrientation of type
N3itk18SpatialOrientation31ValidCoordinateOrientationFlagsEfound, won't
be written to image file

ImageFileWriter 1
*** glibc detected *** ./StreamedScalarImageReadWrite: corrupted
double-linked list: 0x00000000010b9d10 ***

There was no swap death or excessive memory usage but the program didn't
exit and I had to <ctl>-C out.

I tried to stream the RGB data :

./StreamedRGBImageReadWrite \
  ~/bubbles/VisibleWomanHead/RawRGB/VisibleWomanHeadFull.mhd \
  VisibleWomanHeadFull.mhd \
  100

Which gave me :

ImageFileWriter 0.98
ImageFileWriter 0.99
ImageFileWriter 1
*** glibc detected *** ./StreamedRGBImageReadWrite: double free or
corruption (!prev): 0x000000000281b710 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3ee5a75676]
/home/kevin/kitware/ITK_Build/bin/libITKMetaIO.so.4.0(_ZN10MetaObject15ClearUserFieldsEv+0x2d)[0x7fc39c46548d]
/home/kevin/kitware/ITK_Build/bin/libITKMetaIO.so.4.0(_ZN10MetaObjectD1Ev+0x49)[0x7fc39c4665a9]
/home/kevin/kitware/ITK_Build/bin/libITKIO.so.4.0(_ZN3itk11MetaImageIOD1Ev+0x22)[0x7fc39fe7fa42]
/home/kevin/kitware/ITK_Build/bin/libITKIO.so.4.0(_ZN3itk11MetaImageIOD0Ev+0x9)[0x7fc39fe7fa79]
/home/kevin/kitware/ITK_Build/bin/libITKCommon.so.4.0(_ZNK3itk6Object10UnRegisterEv+0x4e)[0x7fc3a01c1bee]
./StreamedRGBImageReadWrite(_ZN3itk12SmartPointerINS_11ImageIOBaseEE10UnRegisterEv+0x35)[0x4280b1]
./StreamedRGBImageReadWrite(_ZN3itk12SmartPointerINS_11ImageIOBaseEED1Ev+0x18)[0x4264ac]
./StreamedRGBImageReadWrite(_ZN3itk15ImageFileWriterINS_5ImageINS_8RGBPixelIhEELj3EEEED2Ev+0x5f)[0x42d987]
./StreamedRGBImageReadWrite(_ZN3itk15ImageFileWriterINS_5ImageINS_8RGBPixelIhEELj3EEEED0Ev+0x18)[0x42da1e]
/home/kevin/kitware/ITK_Build/bin/libITKCommon.so.4.0(_ZNK3itk6Object10UnRegisterEv+0x4e)[0x7fc3a01c1bee]
./StreamedRGBImageReadWrite(_ZN3itk12SmartPointerINS_15ImageFileWriterINS_5ImageINS_8RGBPixelIhEELj3EEEEEE10UnRegisterEv+0x35)[0x4241c1]
./StreamedRGBImageReadWrite(_ZN3itk12SmartPointerINS_15ImageFileWriterINS_5ImageINS_8RGBPixelIhEELj3EEEEEED1Ev+0x18)[0x42361c]
./StreamedRGBImageReadWrite(main+0x25b)[0x42085f]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x3ee5a1ec5d]
./StreamedRGBImageReadWrite[0x420549]
======= Memory map: ========
00400000-00455000 r-xp 00000000 fd:00 1835307
 /home/kevin/Documents/StreamingTest/StreamedRGBImageReadWrite
00654000-00655000 rw-p 00054000 fd:00 1835307
 /home/kevin/Documents/StreamingTest/StreamedRGBImageReadWrite
00655000-00656000 rw-p 00000000 00:00 0
023e7000-028e1000 rw-p 00000000 00:00 0
 [heap]
3ee5600000-3ee561e000 r-xp 00000000 fd:01 12959
 /lib64/ld-2.12.1.so
3ee581d000-3ee581e000 r--p 0001d000 fd:01 12959
 /lib64/ld-2.12.1.so
3ee581e000-3ee581f000 rw-p 0001e000 fd:01 12959
 /lib64/ld-2.12.1.so
3ee581f000-3ee5820000 rw-p 00000000 00:00 0
3ee5a00000-3ee5b75000 r-xp 00000000 fd:01 18799
 /lib64/libc-2.12.1.so
3ee5b75000-3ee5d75000 ---p 00175000 fd:01 18799
 /lib64/libc-2.12.1.so
3ee5d75000-3ee5d79000 r--p 00175000 fd:01 18799
 /lib64/libc-2.12.1.so
3ee5d79000-3ee5d7a000 rw-p 00179000 fd:01 18799
 /lib64/libc-2.12.1.so
3ee5d7a000-3ee5d7f000 rw-p 00000000 00:00 0
3ee5e00000-3ee5e83000 r-xp 00000000 fd:01 19168
 /lib64/libm-2.12.1.so
3ee5e83000-3ee6082000 ---p 00083000 fd:01 19168
 /lib64/libm-2.12.1.so
3ee6082000-3ee6083000 r--p 00082000 fd:01 19168
 /lib64/libm-2.12.1.so
3ee6083000-3ee6084000 rw-p 00083000 fd:01 19168
 /lib64/libm-2.12.1.so
3ee6200000-3ee6202000 r-xp 00000000 fd:01 19541
 /lib64/libdl-2.12.1.so
3ee6202000-3ee6402000 ---p 00002000 fd:01 19541
 /lib64/libdl-2.12.1.so
3ee6402000-3ee6403000 r--p 00002000 fd:01 19541
 /lib64/libdl-2.12.1.so
3ee6403000-3ee6404000 rw-p 00003000 fd:01 19541
 /lib64/libdl-2.12.1.so
3ee6600000-3ee6617000 r-xp 00000000 fd:01 18802
 /lib64/libpthread-2.12.1.so
3ee6617000-3ee6816000 ---p 00017000 fd:01 18802
 /lib64/libpthread-2.12.1.so
3ee6816000-3ee6817000 r--p 00016000 fd:01 18802
 /lib64/libpthread-2.12.1.so
3ee6817000-3ee6818000 rw-p 00017000 fd:01 18802
 /lib64/libpthread-2.12.1.so
3ee6818000-3ee681c000 rw-p 00000000 00:00 0
3ee9600000-3ee9616000 r-xp 00000000 fd:01 23623
 /lib64/libgcc_s-4.4.4-20100630.so.1
3ee9616000-3ee9815000 ---p 00016000 fd:01 23623
 /lib64/libgcc_s-4.4.4-20100630.so.1
3ee9815000-3ee9816000 rw-p 00015000 fd:01 23623
 /lib64/libgcc_s-4.4.4-20100630.so.1
3eea200000-3eea2e9000 r-xp 00000000 fd:01 23624
 /usr/lib64/libstdc++.so.6.0.13
3eea2e9000-3eea4e9000 ---p 000e9000 fd:01 23624
 /usr/lib64/libstdc++.so.6.0.13
3eea4e9000-3eea4f0000 r--p 000e9000 fd:01 23624
 /usr/lib64/libstdc++.so.6.0.13
3eea4f0000-3eea4f2000 rw-p 000f0000 fd:01 23624
 /usr/lib64/libstdc++.so.6.0.13
3eea4f2000-3eea507000 rw-p 00000000 00:00 0
7fc39ad73000-7fc39b751000 rw-p 00000000 00:00 0
7fc39b751000-7fc39b765000 r-xp 00000000 fd:00 13766356
 /home/kevin/kitware/ITK_Build/bin/libitkzlib.so.4.0.0
7fc39b765000-7fc39b965000 ---p 00014000 fd:00 13766356
 /home/kevin/kitware/ITK_Build/bin/libitkzlib.so.4.0.0
7fc39b965000-7fc39b966000 rw-p 00014000 fd:00 13766356
 /home/kevin/kitware/ITK_Build/bin/libitkzlib.so.4.0.0
7fc39b966000-7fc39b968000 r-xp 00000000 fd:00 13766680
 /home/kevin/kitware/ITK_Build/bin/libITKznz.so.4.0.0
7fc39b968000-7fc39bb67000 ---p 00002000 fd:00 13766680
 /home/kevin/kitware/ITK_Build/bin/libITKznz.so.4.0.0
7fc39bb67000-7fc39bb68000 rw-p 00001000 fd:00 13766680
 /home/kevin/kitware/ITK_Build/bin/libITKznz.so.4.0.0
7fc39bb68000-7fc39bb69000 rw-p 00000000 00:00 0
7fc39bb69000-7fc39bb89000 r-xp 00000000 fd:00 13766805
 /home/kevin/kitware/ITK_Build/bin/libITKniftiio.so.4.0.0
7fc39bb89000-7fc39bd89000 ---p 00020000 fd:00 13766805
 /home/kevin/kitware/ITK_Build/bin/libITKniftiio.so.4.0.0
7fc39bd89000-7fc39bd8a000 rw-p 00020000 fd:00 13766805
 /home/kevin/kitware/ITK_Build/bin/libITKniftiio.so.4.0.0
7fc39bd8a000-7fc39bdb0000 r-xp 00000000 fd:00 13766427
 /home/kevin/kitware/ITK_Build/bin/libITKEXPAT.so.4.0.0
7fc39bdb0000-7fc39bfb0000 ---p 00026000 fd:00 13766427
 /home/kevin/kitware/ITK_Build/bin/libITKEXPAT.so.4.0.0
7fc39bfb0000-7fc39bfb2000 rw-p 00026000 fd:00 13766427
 /home/kevin/kitware/ITK_Build/bin/libITKEXPAT.so.4.0.0
7fc39bfb2000-7fc39bfda000 r-xp 00000000 fd:00 13766792
 /home/kevin/kitware/ITK_Build/bin/libITKDICOMParser.so.4.0.0Aborted

Without streaming both read/write programs succeed but the scalar
program prints the
"N3itk18SpatialOrientation31ValidCoordinateOrientationFlagsEfound"
warning once.

Code and image headers attached.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: StreamingTest.tar.bz2
Type: application/x-bzip
Size: 2741 bytes
Desc: not available
URL: <http://www.itk.org/pipermail/insight-users/attachments/20101119/9aa9a1a6/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://www.itk.org/pipermail/insight-users/attachments/20101119/9aa9a1a6/attachment.pgp>

More information about the Insight-users mailing list