[Insight-users] accelerate reading (and writing) (Nightly tests on large images (~2Gb))

[Luis Ibanez]

Thorsten,


For your reference:

We run two Nightly tests (on 64bits machines) that read and write
files larger than 2Gb.


See for example:
http://www.cdash.org/CDash/testSummary.php?project=2&name=itkLargeImageWriteReadTest1&date=2008-10-15
http://www.cdash.org/CDash/testSummary.php?project=2&name=itkLargeImageWriteReadTest2&date=2008-10-15


These were run in "Redwall", a quad-core machine, 64bits,
running Windows Vista, and compiling ITK with VS8

Their timings are:

     itkLargeImageWriteReadTest1    15.37 sec
     itkLargeImageWriteReadTest2    62.88 sec


You will find the code of the tests at:

    Insight/Testing/Code/IO/itkLargeImageWriteReadTest.cxx


It generates an image, writes it, reads it back, and
check all the pixels against the original image.


Here is the output and timing details from test 1

====================================================================
Trying to allocate an image of size 1716 Mb
Initializing pixel values
Trying to write the image to disk
Trying to read the image back from disk
Comparing the pixel values.. :
           Probe Tag    Starts    Stops             Time (s)
             Allocate           1            1      1.90735e-005
              Compare           1            1           2.83842
         Initializing           1            1           3.71668
                 Read           1            1           4.09584
                Write           1            1           5.68875

Test PASSED !
====================================================================

and for test 2 (Note the difference on the image size)

====================================================================
Trying to allocate an image of size 3051 Mb
Initializing pixel values
Trying to write the image to disk
Trying to read the image back from disk
Comparing the pixel values.. :
           Probe Tag    Starts    Stops             Time (s)
             Allocate           1            1      5.34058e-005
              Compare           1            1           5.08393
         Initializing           1            1           4.04456
                 Read           1            1           7.47683
                Write           1            1           45.8443

Test PASSED !
====================================================================


    Regards,


       Luis



--------------------------------------------------------------
Thorsten W. wrote:
> Hello,
> 
> the solution was easier than thought. I have compiled this project in
> debug mode! When i compile it in release mode, reading takes only 1
> minute!
> 
> Regards,
> Thorsten
> 
> Am Dienstag, den 14.10.2008, 12:43 -0400 schrieb Bradley Lowekamp:
> 
>>Hello,
>>
>>
>>The other thing to look at is the file format. Other file formats, are
>>a header followed by raw binary data i.e. VTK NRRD. If you have hard
>>drive space, I'd convert the files overnight. First check to see what
>>the performance of reading these other file types are and make sure
>>it's not slow because of memory swapping.
>>
>>
>>Brad
>>
>>On Oct 13, 2008, at 4:38 PM, Thorsten W. wrote:
>>
>>
>>>Hi,
>>>you know, i'am working with large image matrices. 
>>>I reading a 1.88GB 26000x26000x24Bit BMP File. It takes 5 minutes!
>>>Writing tooks 4 minutes.
>>>
>>>Is there any possibility to update this process aside from update my
>>>pc?
>>>
>>>My current specifications:
>>>AMD Athlon 64 3200+
>>>New Windows XP 64Bit
>>>4GB DDR-Ram
>>>
>>>I'am sorry for asking so many questions and remain
>>>
>>>kind regards,
>>>Thorsten
>>>
>>>_______________________________________________
>>>Insight-users mailing list
>>>Insight-users at itk.org
>>>http://www.itk.org/mailman/listinfo/insight-users
>>>
>>
>>========================================================
>>
>>Bradley Lowekamp  
>>
>>Lockheed Martin Contractor for
>>
>>Office of High Performance Computing and Communications
>>
>>National Library of Medicine 
>>
>>blowekamp at mail.nih.gov
>>
>>
>>
>>
> 
> 
> _______________________________________________
> Insight-users mailing list
> Insight-users at itk.org
> http://www.itk.org/mailman/listinfo/insight-users
>