[Insight-developers] Going from Non-Streaming to Streaming I/O for a class

[Luis Ibanez]

Hi Kent,

Brad L. will probably have a lot to say about this topic,
but let me just contribute some quick guesses here.

On Mon, Apr 25, 2011 at 2:44 PM, Williams, Norman K
<norman-k-williams at uiowa.edu> wrote:
> For HDF5, I've implemented writing and reading images in chunks, as a
> pre-requisite for Streaming I/O.  Now I have a
> couple of questions about how that works.
>
> 1.Is there any particular reason for inheriting from itk::StreamingIOBase?
> I notice that some streamable I/O classes
> inherit itk::ImageIOBase -- in fact most of them.
>

This is due to historical reasons.

The itk::StreamingIOBase class was added recently.

We probably should re-root the other ImageIO classes
that do streaming, to derive from this new class.

> 2. It appears that streamed reading has two requirements -- are there any
> others?:
>   A) implement GenerateStreamableReadRegionFromRequestedRegion.
>   B) In Read, use this->GetIORegion() to decide where in the file to read.
>

You will also need to implement:

      virtual bool CanStreamRead()


(that probably should be "const"...)


> 3. Streamed Writing seems to depend on implementation of
> GetActualNumberOfSplitsForWriting, but it's a bit
>   confusing what to implement.  What seems to be happening is this:
>   A) Implement GetActualNumberOfSplitsForWriting, and return the # of
> chunks to write that the driver is capable of.
>      In the case of HDF5, this can be anything from a single voxel to the
> whole image.  I'm not sure though, what
>      the pastedRegion and largestPossibleRegion are for -- are they
> simply advisory, saying 'I'd like to be able to
>      paste the region pastedRegion, and oh by the way, here's the
> largestPossibleRegion for the file I want to write',
>      or ... what?

you will also need:

virtual bool CanStreamWrite()


The "paste" functionality is intended for a case that is a bit more
generic than streaming. E.g. in Streaming, you could imagine the
output file to be growing progressively as you write data out.
In the "paste" mode, the full file may already be out there in disk,
and we are patching a section of it.


>   B) In Write, write out the region requested by m_IORegion, if possible
> or throw an exception?
>

Yeap, This is more or less what is done in the MetaImageIO class
for example.


> A good thing about HDF5 is that it can handle scatter/gather I/O -- you
> set up the chunk size, and then you can write the image data all at once
> and it divides it into chunks and writes it, optionally compressing each
> chunk. Or you can write out a chunk at a time, out of order.
> --

Nice !

This is going to be really useful.


      Luis


> Kent Williams norman-k-williams at uiowa.edu
>
>
>
>
>
>
> ________________________________
> Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged.  If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited.  Please reply to the sender that you have received the message in error, then delete it.  Thank you.
> ________________________________
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.html
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-developers
>