[Insight-developers] Image memory model

[Karthik Krishnan]

Volken, Dan:

Thanks for the detailed explanation. Comments are inlined.

On Sat, May 24, 2008 at 1:28 AM, Dan Mueller <dan.muel at gmail.com> wrote:
> Thanks for engaging in this discussion with me.
>
> To cut to the chase, my actual problem is politics -- the "not
> invented here syndrome". I am trying to introduce ITK into my project,
> but have encountered resistance from other developers. The memory
> model is their biggest concern; they seem blinded to the plethora of
> advantages ITK offers. As such I am investigating the possibility of
> removing this stumbling block. I guess suggesting a breaking change to
> ITK due to the internal politics at my work is probably not a very
> good idea :P Let me describe the problem further to see if there is
> another way round.
>
> Some background. We have an extensive image processing application
> which uses a slice-contiguous model (based on DICOM). Images are read
> from disk, loaded into memory, image processing operations performed,
> and results rendered to screen or written back to disk. There may be 5
> or more 1GB images open at a time (many situations require less
> memory, some require more). To integrate ITK I am currently 1.
> importing each slice using the Import filter, copying the slices into
> contiguous memory using the Tile filter, applying the desired ITK
> algorithm(s), and converting the contiguous memory back to slices (if
> required). The conversions consume unnecessary time and (more
> importantly) memory, which is not acceptable in various circumstances
> (having to consume an extra 5GB of memory to duplicate image slices is
> not very nice). Some of the ITK operations I want to use in this
> setting include (but are not limited to): registration, level set
> segmentation, intensity-based segmentation, distance transforms, and
> mesh operations (image to mesh, mesh to image).
>
> So, how exactly would the proposed itk::SliceContiguousImage subclass
> work? I guess I would need to override the PixelContainer type (by
> adding a new template argument to itk:Image, which defaults to
> ImportImageContainer) and override the Set/GetPixel(..) methods. Then
> I guess I would need to create a new ImportSlicedImagePixelContainer
> to replace ImportImageContainer. I'm not sure I understand how the new
> image type would interact with linear iterators -- these iterators use
> m_Buffer+m_Offset throughout. Could I somehow override
> ImageBase::ComputeOffsetTable(..) to return the correct offset based
> on the slice memory model?

> -OR-
> 2. attempt to get linear *and* neighbor iterators to work with a slice
> contiguous memory model.
>

Very much possible with the iterators.

We already do this with the itk::VectorImage. More on it here
  http://www.itk.org/Wiki/Proposals:VectorImage

Any access to any pixel via the iterators goes through another class
called the "PixelAccessor" and the "PixelAccessorFunctor".

Each image may define its own PixelAccessor. ie its a trait of the Image.

For instance the itk::VectorImage, which also internally has a
different memory layout from a standard itk::Image, this is done by
defining our own PixelAccessor. Please see
itk::DefaultVectorPixelAccessor

------

If you follow the code of Iterator's Set method:

1. Iterator delegates the actual job of access to the pixel to a
"PixelAccessorFunctor".

   PixelType iterator.Get()   reads
    { return pixelAccessorFunctor.Get(*(m_Buffer+m_Offset)); }


2. PixelAccessorFunctor delegates it to the appropriate
"PixelAccessor" for the image. For instance for the
DefaultVectorPixelAccessorFunctor, the one used for the VectorImage,
this reads:

  inline ExternalPixelType Get( const InternalPixelType &input ) const
    {
    return m_PixelAccessor.Get( input, &input  - m_Begin );
    }

Note that we pass the offset in:  (&input  - m_Begin)


3. The PixelAccessor for a vector image uses the offset to compute the
correct index into its memory layout and return the right pixel on the
fly.

  ExternalType Get( const InternalType & input, const unsigned long
offset ) const
    {
    ExternalType output( (&input)+(offset*m_OffsetMultiplier) ,
m_VectorLength );
    return output;
    }

--------

The same holds true for the iterator.Set(..) methods.

You can see that PixelAccessors allow you to:

(a) Plug in your own way of accessing the image, by computing the
actual index from the iterator's offset (m_Offset).

(b) You don't need this, but FYI. You can construct a different return
type/ input type as well, so as to implement a facade. For the
VectorImage, we not only have a different memory layout, but we wish
to exhibit a facade to the actual pixel type (much like
ImageAdaptors). Pixels retreived/set via Get/Set are VectorPixels,
however the memory organization is that of multi-component Image<
pixels >. (more on the wiki).

Note that all the methods above are inlined for rapid access despite
the delegation.

--------

So the gist is:

  (a) You would write your own Image class: itk::SliceImage.

  (b) You would write your own SliceImagePixelAccessor and
SliceImagePixelAccessorFunctor

  (c) These classes would be traits of your image.

That's it. I also take back statements the statement we might have
made that you would have to write your own iterator. I don't think
this will be needed. All the current iterators in ITK will work fine
on your image as they do for VectorImage.

------

GetBufferPointer() as such is never (I think) used within ITK
directly. However it provides a convinient way of exporting data to
other libraries.

Hope this helps.

--
Karthik Krishnan
R&D Engineer,
Kitware Inc.