[ITK-users] automated splitting and pasting of volumes for use in external programs

Mon Apr 13 10:58:07 EDT 2015

Dear Brad,

Many thanks for Your reply. Just switching back to GenerateData(), according to 
Your suggestion, did the trick of getting streaming to work for this example. 
However, the mini-pipeline filters (AddImageFilter in this case) seem not to run 
multi-threaded (even though the AddImageFilter is based on 
itkBinaryFunctorImageFilter, which is multi-threaded).
Any ideas why?

What confuses me in the User Guide is on page 195 point 4:
"It subdivides data into subpieces for multithreading. (Note that the division 
of data into subpieces is exactly same problem as dividing data into pieces for 
streaming; hence multithreading comes for free as part of the streaming 
architecture.)"
Is the subdivision also done if the Filter has no ThreadedGenerateData but only 
a GenerateData?

Another problem I stumbled into now is that if I define the InputPixelType not 
to equal the OutputPixelType the program aborts with:
Description: itk::ERROR: Image(0x19f8770): itk::Image::Graft() cannot cast 
PKN3itk10DataObjectE to PKN3itk5ImageIdLj3EEE

Why is that? My filter itkLabelShiftImageFilter is templated over the input and 
the output image and so is the internal AddImageFilter.

If I set the NumberOfStreamDivisions to the number of slices in the 3D Stack 
Input, is it guaranteed that each slice will be processed on its own?
Or would I have to use an approach as in 
itkJoinSeriesImageFilterStreamingTest.cxx employing itkExtractImageFilter for 
each slice?

Many thanks again for Your help.
Roman

On 01/04/15 16:53, Bradley Lowekamp wrote:
> Hello,
>
> Looking at your code the first big problem is that you are trying to do a
> mini-pipeline in the ThreadedGenerateData method. You have having each thread
> modify the m_AddImageFilter concurrently.
>
> Generally mini-pipelines should be done in the more general
> GenerateDataMethod as is done int he ITKExample Minipipeline. Then the
> mini-piple filters will be multi-threaded.
>
> ( a couple more comments below )
>
> HTH, Brad
>
> On Apr 1, 2015, at 10:30 AM, Dr. Roman Grothausmann <grothausmann.roman at mh-hannover.de> wrote:
>
>> It seems the the Minipipeline-example (http://www.itk.org/Wiki/ITK/Examples/Developer/Minipipeline) and the Itk User Guide (http://www.itk.org/Doxygen/html/classitk_1_1ImageSource.html#ab988dcc743020c2f4381996ba6503306) differ concerning grafting.
>> With the User Guide grafting of the input it works up to some percent of the StreamingImageFilter execution progress but then crashes with a Segmentation fault. (see https://github.com/romangrothausmann/ITK-CLIs/commit/51d6a3ffc3a74cfb2ccf711b1e3b18f0c65a9876)
>> With the infos given in the User Guide I'm confused what of a general Streaming Filter is needed for a simple Minipipeline Filter (Composit Filter). What I would need is a combination of the Streaming/Threaded example and the Minipipeline-example or the info what needs to be added/modified in the Minipipeline-example.
>> Is my understanding correct that grep-ing the itk sources for ThreadedGenerateData will list all filters that are capable of streaming unless they have special conditions on the InputRequestedRegion?
>
> By default the ITK pipeline is streamable for the ImageSource derived filters. To make it a filter not streamable the regions do need to be modified.
>
> There pipeline methods are general and customizable. There do exists streamable filters which are not multi-threaded. And there exists filters which need their entire output but can an arbitrary output region.
>
>> Will a multi-threaded filter pipeline be only single-threaded if connected to itkStreamingImageFilter?
>
> The pipeline should not be multi-threaded only the individual filters are
>
>>
>> Thanks for any help or hints.
>> Roman
>>
>>
>> On 31/03/15 21:10, Dr. Roman Grothausmann wrote:
>>> Dear Matt,
>>>
>>>
>>> Thanks for Your quick reply. If I got Your hint right, You suggest writing my
>>> own filter (that e.g. does the writing of the chunk to a file) and put an
>>> itkStreamingImageFilter after it supplying it with the # of chunks and the type
>>> of splitter. If I understand the ITK Software Guide correctly, such a filter
>>> needs to be threaded, i.e. have a ThreadedGenerateData.
>>> I tried that with a simple AddFilter that should just add a constant according
>>> to the chunk number to the region it is supposed to process (files attached).
>>> However it stops due to region miss-matches. I can't see why. With just
>>> GenerateData it seems like it is not streaming, according to the
>>> PipelineMonitorImageFilter.
>>> What would be the correct way to go?
>>>
>>> Many thanks for looking into this.
>>> Roman
>>>
>>> On 30/03/15 16:02, Matt McCormick wrote:
>>>> Hi Roman,
>>>>
>>>> You probably want to use the streaming capabilities of the ITK
>>>> pipeline.  This is covered in more detailed in the ITK Software Guide.
>>>> The pipeline internally uses ImageRegionSplitter classes. Overlaps are
>>>> handled by propagation of the RequestedRegion throughout the pipeline.
>>>> To monitor the changes in the regions, use the
>>>> PipelineMonitorImageFilter [1].
>>>>
>>>> HTH,
>>>> Matt
>>>>
>>>> [1] http://www.itk.org/Doxygen/html/classitk_1_1PipelineMonitorImageFilter.html
>>>>
>>>> On Mon, Mar 30, 2015 at 5:25 AM, Dr. Roman Grothausmann
>>>> <grothausmann.roman at mh-hannover.de> wrote:
>>>>> Dear mailing list members,
>>>>>
>>>>>
>>>>> A dataset I work on is too big to load completely into itksnap for manual
>>>>> adjustment of the automatic pre-segementation with watersheds. The full
>>>>> resolution is needed, so I cannot work with a re-sampled version. The only
>>>>> possible easy solution I see at the moment is to split up the 3D image into
>>>>> smaller blocks of sizes as equal as possible, do the itsnap interaction, and
>>>>> finally paste all blocks of the resulting segmentation back together.
>>>>> Would itkImageRegionSplitterMultidimensional be the right filter for the
>>>>> splitting of blocks as equal in size as possible?
>>>>> Is there a "logger" like e.g. itkXMLFileOutputWindow (as used in
>>>>> itkStreamingImageFilterTest2.cxx) that could be used to store the regions
>>>>> corresponding to each block-file to use that for itkPasteImageFilter to
>>>>> combine these blocks later again?
>>>>> Not needed now, but perhaps lager: Is there some nice way to handle overlaps
>>>>> in this case that is already implemented in itk, something like
>>>>> itkImageRegionSplitterMultidimensional::SetOverlap()?
>>>>>
>>>>> Many thanks for any help or hints.
>>>>> Roman
>>>>>

-- 
Dr. Roman Grothausmann

Tomographie und Digitale Bildverarbeitung
Tomography and Digital Image Analysis

Institut für Funktionelle und Angewandte Anatomie, OE 4120
Medizinische Hochschule Hannover
Carl-Neuberg-Str. 1
D-30625 Hannover

Tel. +49 511 532-9574