[Insight-users] ImageSeriesReader memory allocation for large datasets

[Marius Erdt]

Hi,

I have tested some more data sets (.dcm, .png) on different systems, but the result is always the same. As soon as the number of slices gets too high, the memory allocation fails.

Does anyone have an idea how this problem can be fixed?

Thanks,

Marius



I added the output to the file and loaded a data set of 809 slices of 
16-Bit PNG images with a resolution of 573x330. This is the result:

Number of pixels requested = 152973810
Bytes per pixel = 4

so the size should be ~583 Mb in total. However, the memory could not be 
allocated.

regards,
Marius

Luis Ibanez wrote:
>/
/>/
/>/ Hi Marius,
/>/
/>/
/>/
/>/    Could you please help us to track this problem ?
/>/
/>/
/>/
/>/ Please do the following:
/>/
/>/
/>/ Edit the file:
/>/
/>/    Insight/Code/Common/itkImportImageContainer.txx
/>/
/>/ and in lines 182 - 196 you will find the following code:
/>/
/>/   try
/>/     {
/>/     data = new TElement[size];
/>/     }
/>/   catch(...)
/>/     {
/>/     data = 0;
/>/     }
/>/   if(!data)
/>/     {
/>/     // We cannot construct an error string here because we may be out
/>/     // of memory.  Do not use the exception macro.
/>/     throw MemoryAllocationError(__FILE__, __LINE__,
/>/                                 "Failed to allocate memory for image.",
/>/                                 ITK_LOCATION);
/>/     }
/>/
/>/
/>/
/>/ Could you please add a print out for the value of "size" in line 183,
/>/
/>/    std::cout << "Number of pixels requested = " << size << std::endl;
/>/
/>/ and also add a printout for the sizeof(TElement), just to make sure
/>/ that we know how much a pixel is taking in memory:
/>/
/>/    std::cout << "Bytes per pixel = " << sizeof(TElement) << std::endl;
/>/
/>/
/>/
/>/
/>/ Then attempt to load your series again....
/>/
/>/
/>/ Please let us know what numerical values you get from these two
/>/ print outs.
/>/
/>/
/>/
/>/      Thanks
/>/
/>/
/>/         Luis
/>/
/>/
/>/
/>/ -------------------
/>/ Marius Erdt wrote:
/>>/ Hello Luis,
/>>/
/>>/ I'm using a 32-Bit windows, but the process memory limit should not 
/>>/ be a problem since the data consumes just about 500 Mb in total. The 
/>>/ data is not copied or processed in advance, but just loaded using the 
/>>/ itk image series reader.
/>>/
/>>/ The number of pixels of my data is 573x330 in all slices. However, I 
/>>/ experience the same problem when loading big medical data sets of 
/>>/ 512x512. I don't know if it is maybe a problem with the total amount 
/>>/ of slices that can be handled by the itk image series reader or if it 
/>>/ is another issue.
/>>/ Does anyone know if the itk reader has some built in memory limit?
/>>/
/>>/ Regards,
/>>/ Marius
/>>/
/>>/
/>>/ Luis Ibanez wrote:
/>>/
/>>>/ Are you running this in a Windows 64 bits machine ?
/>>>/
/>>>/ Otherwise, Windows 32 processes can only manage 2Gb of RAM
/>>>/ regardless whether you have more RAM available in your machine.
/>>>/
/>>>/ BTW: you mention the number of slices, and the number of bits
/>>>/      per pixel, but not the number of pixels per slice.
/>>>/      Could you please clarify ?
/>>>/
/>>>>/
/>>>>/ I'm getting the following memory allocation error together with the 
/>>>>/ itk::ImageSeriesReader when loading series of png images:
/>>>>/
/>>>>/ d:\itk\src\insighttoolkit-3.4.0\code\common\itkImportImageContainer.txx:188: 
/>>>>/
/>>>>/ Failed to allocate memory for image.
/>>>>/
/>>>>/ However, the data (about 1000 slices of 16Bit png or ~560 Mb) 
/>>>>/ should fit into main memory (4Gbyte). Since the data is just 
/>>>>/ loaded, the Windows data allocation limit for a single process 
/>>>>/ should also not be the problem.
/>>>>/
/>>>>/ I'm experiencing this behavior not just for png images but for all 
/>>>>/ big datasets I want to load with itk. Does anyone know what exactly 
/>>>>/ the problem is or how to load such big data sets with itk?
/>>>>/
/>>>>/ Thanks for help,
/>>>>/ Marius
/>>>>/ _______________________________________________
/>>>>/ Insight-users mailing list
/>>>>/ Insight-users at itk.org <http://www.itk.org/mailman/listinfo/insight-users>
/>>>>/ http://www.itk.org/mailman/listinfo/insight-users
/>>>>/
/>>>/
/>>/
/>>/
/