[Insight-developers] ImageSpatialObject Bug0006340

Tue Aug 5 16:28:23 EDT 2008

Hi Rupert,

Am I understanding the problem correctly...

ImageType::Pointer img = ImageType::New();
img->SetDirections( someNonIdentityMatrix );
img->TransformPhysicalPointToIndex(pnt, indx);
std::cout << img->GetDirections() << std::endl;

Will produce the exact same output (i.e., have the same directions) if
ImageType is an itkImage or an itkOrientedImage, BUT the value of indx
(which isn't printed) will potentially be very different.   That is,
the images will have the same set of directions, but one will use it
and one won't.

We actually won't know what the code does unless we know what cmake
options were set, right?

Regretfully the concept of image direction was added after spatial
objects were created.   ImageSpatialObjects (like many other spatial
objects) have transforms which move from index space to object space
and from object space to world space.   For imagespatialobjects, in
the index space to object space transform, voxel spacing is
considered.  For object space to world transform, the object (and its
children-objects, e.g., extracted surfaces) are transformed to place
them in the scene.

So, the question becomes, should the directions in an image be applied
to objects extracted from the image, or not?

As specific examples, consider meshes or paths extracted from an image
using one of the existing filters.   When you extract a mesh or a path
from an image, in what space does it live: index space, object space,
or world coordinates?   When considering this, it is particularly
interesting to look at how  normals / gradients at node points are
computed (e.g., how paths are steered as they are extracted from an
image).  Worst case, if the image coordinates are transformed by the
directions, but the gradients at those coordinates aren't specifically
transformed by the directions, then those entities live in a hybrid
space with some info (coordinates) in world space and some info
(gradients) in object space.

If we assume all is good, then we should apply the directions when
going from index to object space, otherwise, the directions should be
applied when going from object space to world space, so that the
directions are applied to children objects as well.

Stephen

On Tue, Aug 5, 2008 at 3:31 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
> More questions:
>
> Can ImageSpatialObject be rewritten to use the Transform methods of
> Image and OrientedImage?
>
> Bill
>
> On Tue, Aug 5, 2008 at 3:26 PM, Stephen Aylward
> <Stephen.Aylward at kitware.com> wrote:
>> So,
>>
>> Should an itkImage always return the identity matrix for its directions?
>>
>> or
>>
>> Should we allow inconsistency between recorded information and actual
>> operation (record, but do not use direction in itkImage)?
>>
>> or
>>
>> Should we do-away with the itkImage without orientation, and should
>> all images be itkOrientedImages?
>>
>>
>> Stephen
>>
>>
>>
>> On Tue, Aug 5, 2008 at 3:19 PM, Rupert Brooks <rupe.brooks at gmail.com> wrote:
>>> Hi Bill,
>>>
>>> Thats just the thing - the ImageSpatialObject does NOT use
>>> TransformXtoY calls, because it has its own internal transformation
>>> system.  It copies the information out of the image that it is given
>>> and uses that as the spatial object IndexToWorld transform.  The bug
>>> was caused because previously it did not copy the direction - this is
>>> easily fixed.
>>>
>>> The tricky issue is that itk::Image always behaves as though its
>>> direction is identity - but this is not enforced.  Its entirely
>>> possible to put a non-identity direction in itk::Image.  This is not
>>> used in the TransformXtoY calls of the image - but it is returned by
>>> the GetDirection method.  So when i copy the direction information, if
>>> there is some in the itk::Image, it breaks the test.
>>>
>>> As you can probably guess, i discovered that by accident.
>>>
>>> I agree that itk::Image should not behave differently inside and
>>> outside a Spatial object.   As I see it, theres 4 options - I think
>>> its a design decision which is why im bringing it up here.
>>>
>>> 1. Explicitly test for the itk::Image case inside itk::SpatialObject,
>>> and ignore its directions
>>>    This will run into problems with subclasses of itk::Image - and
>>> not all subclasses can be treated the same way.  Which leads to #2
>>>
>>> 2. Explicitly test for the itk::OrientedImage case inside
>>> itk::SpatialObject and only use the direction in that case
>>>    This runs into the reverse problem, that if another subclass of
>>> itk::Image with direction information is used, it will break when used
>>> in the itk::ImageSpatialObject.    Perhaps I am biased against this
>>> because i actually have such a class in my code.   No such class
>>> currently exists in ITK, that i know of.
>>>
>>> 3. Consider this a conceptual bug in itk::Image, and force the
>>> itk::Image GetDirection method to always return identity.
>>>    I think this is the most correct answer, but i fear backward
>>> compatibility consequences.
>>>
>>> 4. Ignore the issue, and assume/hope that users wont put direction
>>> cosines in an itk::Image anyway.
>>>    This is by far the easiest approach, but i don't like knowing a
>>> potential bug exists and not fixing it.  These type of sins always
>>> seem to come back to haunt me :-)
>>>
>>> Rupert
>>>
>>> On Tue, Aug 5, 2008 at 2:08 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>> Rupert,
>>>>
>>>> In general, itkImage directions are ignored (assumed identity).
>>>> itkOrientedImage directions are used. I have not looked at the
>>>> ImageSpatialObject yet. If it uses Transform{X}To{Y} type calls, then
>>>> Image and OrientedImage should behave properly. I don't think itkImage
>>>> should behave differently inside and outside of itkImageSpatialObject.
>>>>
>>>> Bill
>>>>
>>>> On Tue, Aug 5, 2008 at 1:58 PM, Rupert Brooks <rupe.brooks at gmail.com> wrote:
>>>>> Hi Everyone,
>>>>>
>>>>> As Luis wisely suggested i waited until after the 3.8 release to work
>>>>> on this  :-)  I still need some big picture advice thoug
>>>>>> Please note that the first action to take, even before experimenting
>>>>>> with a fix, is to add a tests that illustrates the failure. E.g. to
>>>>>> add a test that exercise the condition you have identified as
>>>>>> problematic.
>>>>>
>>>>> I've added the test (itkImageMaskSpatialObjectTest2), and you all
>>>>> should notice it failing in your dashboards shortly :-)
>>>>>
>>>>> Theres more than one way to patch the problem - i'll refer back to my
>>>>> previous email.  I think theres a conceptual bug, or at least
>>>>> confusion in the itk::Image. I cant decide how i should make
>>>>> ImageSpatialObject behave when handed an ITK image that has
>>>>> non-identity directions.
>>>>>
>>>>>>> Briefly, the image spatial object takes the coordinate transform from
>>>>>>> the image that it is given, and uses that as its IndexToWorld
>>>>>>> transformation.  Previously, it was ignoring the direction.  The fix
>>>>>>> was a matter of adding the necessary lines to copy the direction
>>>>>>> information also.
>>>>>>>
>>>>>>> However, this fix raises a further issue, and I'd like advice:
>>>>>>>
>>>>>>> The itk::Image class may have a non-identity direction component, but
>>>>>>> it always ignores it.  However, a naive implementation of my fix would
>>>>>>> use that direction information, giving a different behavior than the
>>>>>>> itk::Image.  I could write code to check if its an ITK image, and then
>>>>>>> behave differently, but this seems inelegant at best.   Is this
>>>>>>> actually another bug - should an itk::Image always have an identity
>>>>>>> direction, to conform with its behavior?  Can i ignore the issue,
>>>>>>> assuming the user will be bright enough to only put direction
>>>>>>> information in an itk::Image for a very good reason.
>>>>>
>>>>> Does anyone have some insights into how the itkImageSpatialObject
>>>>> should behave when handed an itk::Image which has non-identity
>>>>> direction cosines?
>>>>>
>>>>> (the bug in question is this one
>>>>> http://public.kitware.com/Bug/view.php?id=0006340)
>>>>>
>>>>> Cheers
>>>>> Rupert B.
>>>>>
>>>>>
>>>>> --
>>>>> --------------------------------------------------------------
>>>>> Rupert Brooks
>>>>> McGill Centre for Intelligent Machines (www.cim.mcgill.ca)
>>>>> Ph.D Student, Electrical and Computer Engineering
>>>>> http://www.cyberus.ca/~rbrooks
>>>>> _______________________________________________
>>>>> Insight-developers mailing list
>>>>> Insight-developers at itk.org
>>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>>>
>>>> _______________________________________________
>>>> Insight-developers mailing list
>>>> Insight-developers at itk.org
>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>>
>>>
>>>
>>>
>>> --
>>> --------------------------------------------------------------
>>> Rupert Brooks
>>> McGill Centre for Intelligent Machines (www.cim.mcgill.ca)
>>> Ph.D Student, Electrical and Computer Engineering
>>> http://www.cyberus.ca/~rbrooks
>>> _______________________________________________
>>> Insight-developers mailing list
>>> Insight-developers at itk.org
>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>
>>
>>
>>
>> --
>> Stephen R. Aylward, Ph.D.
>> Chief Medical Scientist
>> Kitware, Inc. - Chapel Hill Office
>> http://www.kitware.com
>> (518) 371-3971 x300
>>
>

-- 
Stephen R. Aylward, Ph.D.
Chief Medical Scientist
Kitware, Inc. - Chapel Hill Office
http://www.kitware.com
(518) 371-3971 x300