[Insight-developers] Going from Non-Streaming to Streaming I/O for a class

Tue Apr 26 05:47:54 EDT 2011

Le 25 avr. 11 à 22:57, Luis Ibanez a écrit :

> Hi Kent,
>
> Brad L. will probably have a lot to say about this topic,
> but let me just contribute some quick guesses here.
>
> On Mon, Apr 25, 2011 at 2:44 PM, Williams, Norman K
> <norman-k-williams at uiowa.edu> wrote:
>> For HDF5, I've implemented writing and reading images in chunks, as a
>> pre-requisite for Streaming I/O.  Now I have a
>> couple of questions about how that works.
>>
>> 1.Is there any particular reason for inheriting from  
>> itk::StreamingIOBase?
>> I notice that some streamable I/O classes
>> inherit itk::ImageIOBase -- in fact most of them.
>>
>
> This is due to historical reasons.
>
> The itk::StreamingIOBase class was added recently.
>
> We probably should re-root the other ImageIO classes
> that do streaming, to derive from this new class.
>
>> 2. It appears that streamed reading has two requirements -- are  
>> there any
>> others?:
>>   A) implement GenerateStreamableReadRegionFromRequestedRegion.
>>   B) In Read, use this->GetIORegion() to decide where in the file  
>> to read.
>>
>
> You will also need to implement:
>
>      virtual bool CanStreamRead()
>
>

If your ImageIO is a subclass of StreamingIOBase, you only have to use  
GetIORegion() in Read.
The other methods are already implemented.

> (that probably should be "const"...)
>
>
>> 3. Streamed Writing seems to depend on implementation of
>> GetActualNumberOfSplitsForWriting, but it's a bit
>>   confusing what to implement.  What seems to be happening is this:
>>   A) Implement GetActualNumberOfSplitsForWriting, and return the # of
>> chunks to write that the driver is capable of.
>>      In the case of HDF5, this can be anything from a single voxel  
>> to the
>> whole image.

Yes but it's probably more efficient to match the hdf5 chuncks,  
especially when compression is unable.
For the final version, it would be nice to return the number of chunks  
in the hdf5 files, at least when the compression in on.

>>  I'm not sure though, what
>>      the pastedRegion and largestPossibleRegion are for -- are they
>> simply advisory, saying 'I'd like to be able to
>>      paste the region pastedRegion, and oh by the way, here's the
>> largestPossibleRegion for the file I want to write',
>>      or ... what?
>
> you will also need:
>
> virtual bool CanStreamWrite()
>
>
> The "paste" functionality is intended for a case that is a bit more
> generic than streaming. E.g. in Streaming, you could imagine the
> output file to be growing progressively as you write data out.
> In the "paste" mode, the full file may already be out there in disk,
> and we are patching a section of it.
>
>
>>   B) In Write, write out the region requested by m_IORegion, if  
>> possible
>> or throw an exception?
>>
>
> Yeap, This is more or less what is done in the MetaImageIO class
> for example.
>
>
>> A good thing about HDF5 is that it can handle scatter/gather I/O --  
>> you
>> set up the chunk size, and then you can write the image data all at  
>> once
>> and it divides it into chunks and writes it, optionally compressing  
>> each
>> chunk. Or you can write out a chunk at a time, out of order.
>> --
>
> Nice !
>
> This is going to be really useful.
>
>
>      Luis
>
>
>> Kent Williams norman-k-williams at uiowa.edu
>>
>>
>>
>>
>>
>>
>> ________________________________
>> Notice: This UI Health Care e-mail (including attachments) is  
>> covered by the Electronic Communications Privacy Act, 18 U.S.C.  
>> 2510-2521, is confidential and may be legally privileged.  If you  
>> are not the intended recipient, you are hereby notified that any  
>> retention, dissemination, distribution, or copying of this  
>> communication is strictly prohibited.  Please reply to the sender  
>> that you have received the message in error, then delete it.  Thank  
>> you.
>> ________________________________
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.html
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-developers
>>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.html
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers

-- 
Gaëtan Lehmann
Biologie du Développement et de la Reproduction
INRA de Jouy-en-Josas (France)
tel: +33 1 34 65 29 66    fax: 01 34 65 29 09
http://voxel.jouy.inra.fr  http://www.itk.org
http://www.mandriva.org  http://www.bepo.fr

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: Ceci est une signature ?lectronique PGP
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20110426/98f997d8/attachment.pgp>