[ITK-users] compressing MetaImages bigger than 4GB leads to data loss

[Dr. Roman Grothausmann]

On 25/06/14 16:14, Bill Lorensen wrote:
>  From the zlib FAQ:
> http://www.zlib.net/zlib_faq.html#faq32

Hm, I'm not sufficiently into the ITK MHD-compression to understand if this 
relates to the need of choosing ITK_USE_64BITS_IDS ON, i.e. making counters for 
inflate() and deflate() be 64 bits long or would specific chunk handling have to 
be implemented in ITK?
In short, can I resolve the problem by simply adjusting the ITK configuration 
with cmake or would it need changes to the code of ITK?


>
>
> On Wed, Jun 25, 2014 at 9:52 AM, Dr. Roman Grothausmann
> <grothausmann.roman at mh-hannover.de> wrote:
>> On 25/06/14 15:16, Bradley Lowekamp wrote:
>>>
>>> Please try compiling with ITK_USE_64BITS_IDS OFF, this flag is only needed
>>> for windows 64.
>>
>>
>> I just did, sadly the problem is still there.
>> I chose ITK_USE_64BITS_IDS ON because vtk needs it to be able to cope with
>> files bigger than 4GB. How comes linux doesn't need it?
>>
>>
>>> On Jun 25, 2014, at 8:59 AM, Dr. Roman Grothausmann
>>> <grothausmann.roman at mh-hannover.de> wrote:
>>>
>>>> On 25/06/14 14:51, Dr. Roman Grothausmann wrote:
>>>>>
>>>>> Dear Brad,
>>>>>
>>>>> On 25/06/14 14:24, Bradley Lowekamp wrote:
>>>>>>
>>>>>> That sounds like a bug to me. What OS? version of ITK? Are you using
>>>>>> ITK's zlib?
>>>>>
>>>>>
>>>>> Linux, Debian 7.2, self compiled itk-4.5.1 (problem also exists with
>>>>> itk-4.4.0),
>>>>> I tried both ITK's zlib and system zlib, same problem for both.
>>>>
>>>>
>>>> Forgot:
>>>> uname -a
>>>> Linux imgserv0 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64
>>>> GNU/Linux
>>>>
>>>> itk configured with:
>>>> ITK_USE_64BITS_IDS               ON
>>>>
>>>> So files bigger than 4GB can be handled, just the compression seems to be
>>>> affected.
>>>>
>>>>
>>>>>
>>>>> I also tested system libtiff. In that case the ITKwriter/libtiff throws
>>>>> an error
>>>>> that the file is too large for the TIFF-Standard, just as it should be.
>>>>>
>>>>> Many thanks for looking into this.
>>>>> Roman
>>>>>
>>>>>> On Jun 25, 2014, at 8:16 AM, Dr. Roman Grothausmann
>>>>>> <grothausmann.roman at mh-hannover.de> wrote:
>>>>>>
>>>>>>> Dear mailing list members,
>>>>>>>
>>>>>>>
>>>>>>> Working with a labelled tomogram that is larger than 4GB made me
>>>>>>> realize that
>>>>>>> when saving the data as a compressed MetaImage (mhd) leads to a
>>>>>>> complete data
>>>>>>> loss. It seems the way libz is used that files bigger than 4GB are
>>>>>>> compressed
>>>>>>> incorrectly. If that's not possible the itkFileWriter should issue an
>>>>>>> error
>>>>>>> in case an image is bigger than 4GB. So far it just goes on as if
>>>>>>> nothing
>>>>>>> would be wrong and only when opening the image again the reader
>>>>>>> reports that
>>>>>>> "Uncompress failed".
>>>>>>> Is this a bug?
>>>>>>>
>>>>>>> Just as a side node: writing the same file as a compressed TIFF works
>>>>>>> fine,
>>>>>>> though e.g. imagej/fiji opens the file with MSB swapped.
>>>>>>>
>>>>>>> Any help or hints are very much appreciated
>>>>>>> Roman
>>>>>>>
>>>>>>> --
>>>>>>> Dr. Roman Grothausmann
>>>>>>>
>>>>>>> Tomographie und Digitale Bildverarbeitung
>>>>>>> Tomography and Digital Image Analysis
>>>>>>>
>>>>>>> Institut für Funktionelle und Angewandte Anatomie, OE 4120
>>>>>>> Medizinische Hochschule Hannover
>>>>>>> Carl-Neuberg-Str. 1
>>>>>>> D-30625 Hannover
>>>>>>>
>>>>>>> Tel. +49 511 532-9574
>>>>>>> _____________________________________
>>>>>>> Powered by www.kitware.com
>>>>>>>
>>>>>>> Visit other Kitware open-source projects at
>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>
>>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>>> http://www.kitware.com/products/protraining.php
>>>>>>>
>>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>>>
>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>> http://public.kitware.com/mailman/listinfo/insight-users
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Dr. Roman Grothausmann
>>>>
>>>> Tomographie und Digitale Bildverarbeitung
>>>> Tomography and Digital Image Analysis
>>>>
>>>> Institut für Funktionelle und Angewandte Anatomie, OE 4120
>>>> Medizinische Hochschule Hannover
>>>> Carl-Neuberg-Str. 1
>>>> D-30625 Hannover
>>>>
>>>> Tel. +49 511 532-9574
>>>
>>>
>>
>> --
>> Dr. Roman Grothausmann
>>
>> Tomographie und Digitale Bildverarbeitung
>> Tomography and Digital Image Analysis
>>
>> Institut für Funktionelle und Angewandte Anatomie, OE 4120
>> Medizinische Hochschule Hannover
>> Carl-Neuberg-Str. 1
>> D-30625 Hannover
>>
>> Tel. +49 511 532-9574
>> _____________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://www.kitware.com/products/protraining.php
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/insight-users
>
>
>

-- 
Dr. Roman Grothausmann

Tomographie und Digitale Bildverarbeitung
Tomography and Digital Image Analysis

Institut für Funktionelle und Angewandte Anatomie, OE 4120
Medizinische Hochschule Hannover
Carl-Neuberg-Str. 1
D-30625 Hannover

Tel. +49 511 532-9574