[ITK Community] [Insight-users] Reading portion of image file

Bradley Lowekamp blowekamp at mail.nih.gov
Thu Nov 14 11:30:51 EST 2013 

The issue below is with HDF5 and streaming. I am not sure that you have the same one, it may similar be in nifti with files left open changing files and keeping the reader.

Are you performing streaming?
What is the error message you are getting?

I think your solution of creating a new ImageFilterReader for each file is best. That's the way I have done it.

Brad



On Nov 14, 2013, at 11:22 AM, Alexander Schmidt-Richberg <schmidtrichberg at me.com> wrote:

> Hi all,
> 
> I wonder if this problem has ever been solved/fixed.
> I'm experiencing (what I think is) exactly the same weird bug when reading a large number of nifti images.
> 
> The first option in the code below (sampleRegion = true) produces the same message after approx. 200 images, the second one (= false) works fine... Instantiating a new reader in every loop also works.  
> 
> Running the current git version.
> 
> Cheers
> Alex
> 
> for( unsigned int subject = 0; subject < numSubjects; ++subject )
> {
>   imageReader->SetFileName( filenames[subject].c_str() );
>   imageReader->Update();
> 
>   if( sampleRegion )
>   {
>     IteratorType it( imageReader->GetOutput(), samplingRegion );
>     for( it.GoToBegin(); !it.IsAtEnd(); ++it )
>     {
>       FeatureValueType value = it.Get(); // ...
>     }
>   }
>   else
>   {
>     IteratorType it( imageReader->GetOutput(), imageReader->GetOutput()->GetRequestedRegion() );
>     for( it.GoToBegin(); !it.IsAtEnd(); ++it )
>     {
>       FeatureValueType value = it.Get(); // ...
>     }
>   }
> }
> 
> 
> Am 08. März 2012 um 17:00 schrieb Bradley Lowekamp <blowekamp at mail.nih.gov>:
> 
>> Hello Matthias,
>> 
>> You are correct that the documentation for the ImageFileReader::EnlargeOutputRequestedRegion should be updated. However only certain ImageIO file formats support the "stream" or roi reading. The one derived from the StreamingImageIO base along with nift, and maybe others support this functionality. You'd have to look at ImageIO to know for sure.
>> 
>> Your usage of the ImageFileReader->ExtactImageFilter was scenario to motivate for the recent optimization of making the Extract filter run InPlace. That is very nice usage!
>> 
>> It looks like you have encountered a bug. Most likely some one is not closing a file, and you are running out of file descriptors for the process.
>> 
>> 
>> Here is the procedure for reporting a bug:
>> http://www.itk.org/Wiki/ITK/Procedure_for_Contributing_Bug_Fixes
>> 
>> If you have a "working" bit of code which exhibits the bug, that would help us reproduce the issue and should be included. This will help expedite the solution. If you have a patch to fix it even better.
>> 
>> I am suspicious that the new HDF5 ImageIO is not closing the file in the CanRead method.
>> 
>> Brad
>> 
>> On Mar 8, 2012, at 10:31 AM, Matthias Schneider wrote:
>> 
>>> Hi.
>>> 
>>> I'm trying to extract subregions of 2D image files. Ideally, I'd like to 
>>> read the required data only.
>>> In the ITK doc I came across the comment in
>>> itk::ImageFileReader::EnlargeOutputRequestedRegion() saying that:
>>> "ImageFileReader cannot currently read a portion of an image (since the 
>>> ImageIO objects cannot read a portion of an image)"
>>> 
>>> I'm not quite sure if this still holds. After a quick experiment, I was 
>>> pretty much confident that the readers are in fact able to read portions:
>>> 
>>> const std::string fname = "/path/to/my/image.nii"; // [2048x2048 px]
>>> typedef itk::Image<Pixel8uType, 2> ImageType;
>>> typedef itk::ImageFileReader<ImageType> ReaderType;
>>> 
>>> ImageType::RegionType roi;
>>> roi.SetSize(0, 512);
>>> roi.SetSize(1, 512);
>>> 
>>> for (int i = 0; i < N; ++i) {
>>>   std::cout << i << std::endl;
>>>   ReaderType::Pointer reader = ReaderType::New();
>>>   reader->SetFileName(fname);
>>>   reader->GetOutput()->SetRequestedRegion(roi);
>>>   reader->Update();
>>> }
>>> 
>>> At first glance, this works perfectly fine (I checked the buffered data 
>>> chunk) and runs much faster than reading the entire slice. Choosing N 
>>> very large (say 1024), the program crashes since the reader seems to not 
>>> close the file descriptors properly. I get the following error:
>>> 
>>> HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
>>>   #000: [...]/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5F.c line 
>>> 794 in H5Fis_hdf5(): unable to open file
>>>     major: Low-level I/O
>>>     minor: Unable to initialize object
>>>   #001: [...]/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FD.c line 
>>> 1086 in H5FD_open(): open failed
>>>     major: Virtual File Layer
>>>     minor: Unable to initialize object
>>>   #002: [...]/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FDsec2.c 
>>> line 348 in H5FD_sec2_open(): unable to open file: name = 
>>> '/path/to/my/image.nii', errno = 24, error message = 'Too many open 
>>> files', flags = 0, o_flags = 0
>>>     major: File accessability
>>>     minor: Unable to open file
>>> terminate called after throwing an instance of 
>>> 'itk::ImageFileReaderException'
>>>   what(): 
>>> [...]/ITK/Modules/IO/ImageBase/include/itkImageFileReader.hxx:143:
>>>  Could not create IO object for file /path/to/my/image.nii
>>> The file couldn't be opened for reading.
>>> Filename: /path/to/my/image.nii
>>> 
>>> 
>>> I also tried to explicitly use the ExtractImageFilter:
>>> 
>>> [... same as above ...]
>>> typedef itk::ExtractImageFilter<ImageType, ImageType> ExtractFilterType;
>>> 
>>> for (int i = 0; i < N; ++i) {
>>>   std::cout << i << std::endl;
>>>   ReaderType::Pointer reader = ReaderType::New();
>>>   reader->SetFileName(fname);
>>>   // reader->Update();
>>> 
>>>   ExtractFilterType::Pointer extractor =  ExtractFilterType::New();
>>>   extractor->SetInput(reader->GetOutput());
>>>   extractor->SetExtractionRegion(roi);
>>>   extractor->InPlaceOn(true);
>>>   extractor->Update();
>>> }
>>> 
>>> Without the reader->Update() the program still crashes when choosing N 
>>> sufficiently large (~1024). Updating the reader first of course reads 
>>> the entire slice (what I really want to avoid).
>>> So does this mean that the comment in the manual still holds and the 
>>> ImageIO is not able to correctly handle this case even though the 
>>> buffered data chunks are correctly imported?! This confuses me a bit or 
>>> maybe I'm missing something else?!
>>> 
>>> I'm happy about any comments or suggestions.
>>> 
>>> Thanks,
>>> Matthias
>>> _____________________________________
>>> Powered by www.kitware.com
>>> 
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>> 
>>> Kitware offers ITK Training Courses, for more information visit:
>>> http://www.kitware.com/products/protraining.html
>>> 
>>> Please keep messages on-topic and check the ITK FAQ at:
>>> http://www.itk.org/Wiki/ITK_FAQ
>>> 
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.itk.org/mailman/listinfo/insight-users
>> 
>> ========================================================
>> Bradley Lowekamp  
>> Medical Science and Computing for
>> Office of High Performance Computing and Communications
>> National Library of Medicine 
>> blowekamp at mail.nih.gov
>> 
>> 
>> _____________________________________
>> Powered by www.kitware.com
>> 
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>> 
>> Kitware offers ITK Training Courses, for more information visit:
>> http://www.kitware.com/products/protraining.html
>> 
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>> 
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/community/attachments/20131114/d389e161/attachment.html>
-------------- next part --------------
_____________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://www.itk.org/mailman/listinfo/insight-users

More information about the Community mailing list