ITK/Roadmap 2005 2006

Introduction

The future developement of ITK relies heavily on feedback from the community of users and developers as well as guidelines from the clinical and medical research community.

This page is intended to gather feedback regarding the future direction of ITK and in particular to list specific features and functionalities that would make the toolkit more useful for the medical imaging community.

Users/Developers Support

This section relates to the services that will help beginners to start with ITK as well as material that will help experienced users to take better advantage of the functionalities available in the Insight toolkit.

Tutorials

Conferences

Tutorials have been typically presented in conferences related to Medical Imaging

• MICCAI
• SPIE Medical Imaging
• IEEE Visualization

Advanced Tutorials

The tutorials so far have been focused on Introductory material. The typical introductory tutorial is presented in 8 hours and does not allow for hands-on experience with the segmentation and registration methods. Now that the toolkit has a considerable user-base it seems appropriate to develop tutorials in more advanced topics and present those tutorials at conferences. Advanced tutorial could cover topics such as

• How to write an ITK filter
  • How to deal with Traits and Multi-platform
  • How to deal with Multi-threading
• Advanced registration
  • Fine tuning parameters
  • Profiling metrics
  • Improving performance
  • Specific clinical uses
• Advanced segmentation
  • Fine tuning parameters
  • Specific clinical uses

Workshops

Workshops allow for a more interactive experience than tutorials. In the context of a workshop, attendants are expected to contribute material and share it with colleagues. The framework of a workshop could provide a more effective atmosphere for discussing the addition of new classes to the toolkit and for improving the implementation of existing ones.

Documentation

Documentation for ITK is provided at several levels. It includes the user's manual level provided by the ITK Software Guide, the reference manual level as provided by Doxygen and the community forum as provided by the Wiki.

Doxygen

• How to make Doxygen a more effective tool for assisting users.
• Identify classes with poor documentation level and take correction action.
• Increase the crosslinking of Doxygen with the software guide.

Software Guide

• Adding more 3D Examples
• Adding some 4D Examples
• Adding more links to papers

WIKI

• How to make the wiki more effective tool for the community
• Flag particular sections of the Wiki that may have become outdated and correct them
• Reduce the duplication of information (e.g. ITK installation is explained in the tutorials material, the software guide, the wiki and the HTML pages).

The Insight Journal

The Insight Journal will come up online very soon.

We anticipate it to be the standard mechanism for

• Bringing new algorithms into ITK.
• Evaluating existing algorithms.
• Sharing experiences and "how-to" knowledge among the community.

In particular, the Insight Journal will become a repository on specific recipies on how to process specific types of data sets. This will bring ITK a layer above of the plain image processing and into the more domain specific area of Medical Image Analysis.

Features and Functionalities

Software Engineering

Explicit Instantiations

This will improve compile time and will reduce the size of libraries and binaries

Typeless layer

A C++ typeless layer will provide users with a toolkit where the pixel type and dimension of the image do not have to be decided in advance. This layer will look very close to what VTK uses for representing images and imaging filters.

Adopting Dart2

Dart2 will bring a large number of interesting features that will improve the quality control system for ITK. We should plan the transition from Dart to Dart2 in a progressive and organized way.

Increasing Code Coverage

Code coverage is low for a significan number of files. It is usually correlated with the less used classes in the toolkit

Add support to CTest for reducing redundant tests

CTest could provide coverage of classes per test in order to determine if a particular class is being covered in multiple tests. From this analysis, developers could remove redundant testing in the toolkit.

Explore other tools for code coverage in CTest

Currently we use gcov but this tool have some limitations. It will be desirable to have an alternative way of measuring code coverage.

CTest will have to parse the output of these alternative tools and format it according to what Dart/Dart2 expects as input.

Potential Candidates

• Valgrind

 Valgrind is a good candidate since it has many different tools/skins. 
 It supports the profiling of memory usage and code.
 (see addrcheck, cachegrind, corecheck, lackey, massif, memcheck )

• Visual Studio
• http://c2.com/cgi/wiki?CodeCoverageTools
• http://www.testing.com/writings/coverage-terminology.html

Increasing Valgrind/Purify Testing

Testing for run-time problems has received reduced attention. As ITK move more and more into applications, we should pay closer attention to the run-time correctness of the code. We should setup more machines for running Valgrind/Purify testing

Since these builds take longer, we could take advantage here of Dart2 new alternative periodicity. For example, run these tests on a weekly or montly basis and then enter in sessions for cleaning up the code.

Add support to CTest for runtime profiling

Improving run-timeperformance of ITK algorithem requires to analyze the amount of time that each piece of code take for execution.

Potential candidate tools

• Visual Studio
• gprof
• Valgrind/Calltree

Add Support for parallel testing in CTest

This will make possible to launch multiple test simultaneously from CTest. In this way, users with multiprocessor machines will be able to reduce the time needed for running the test suite.

Add Support to CTest for generating reports locally

This functionality will make possible to analyze the results of the tests suite in a local machine, before it is submitted to the Dart/Dart2 server.

Deprecation Policy

A committe has beed named for designing and enforcing the implementation of a compasionate and user-friendly deprecation policy.

Factorizing IO Classes with VTK

The ITK IO classes in ITK are very similar to the VTK ones. In many cases they were simply copy/pasted from VTK

• A particular attention should be done for jpeg since ITK and VTK does not use exactly the same one.
• Question: Does this mean that VTK/ITK should be synchronized on let say the exact same png library ?

ITK Complete

Package a release of ITK that contains

• Insight
• InsightApplications
• VTK
• FLTK

Along with a commong CMakeLists.txt file that will allow to configure the entire package with consistent options and minimal effort.

• These distribution will factorize common resources such as KWSYS/PNG/JPEG/TIFF/expat/gdcm/

Experimental repository with Subversion (SVN)

Subversion is an alternative to CVS and provides functionalites that are superior to CVS. Since this is quite a significant change, what we probably want to do is to start maintaining a parallel Subversion repository in order to gain experience with it for a project of the size of ITK.

Image Modalities

A variety of image modalities are now well supported in ITK at this point. New image modalities may become important for the medical research community and later on for the clinical users. This section list the upcoming image modalities, their relevance and what would it take to support them in ITK. Note that the categories are not independent and that some of them will overlap.

Color Images

Basic support exists now in ITK for dealing with RGB images. A reduced number of filters and segmenation techniques are ready-to-use in the toolkit. No support for registration of color images is currently available. Although several users have explored this domains, have written ITK classes for it and are willing to contribute their code to the toolkit.

The challenges are

• Color model conversion (HSV, RGB)
• Image IO
• Segmentation
  • Statistical classifications
  • Region growing
  • Levelsets

Multi-Spectral Images

Multi-spectral images are used for modalities such as Microscopy, MRI and optical tomograpy. Not to mention that ITK has been used for astronomy and for satellite imagery. The issues here are very similar to the ones of color images, with the distinction that here we are not subject to the particularities of the human visual system.

The challenges are

• Enhancement and Noise reduction
• Principla components analysis
• Registration for multiple-component images
• Callibration
• Segmentation
  • Statistical classification
  • Region growing
  • Multicomponents level sets
• Pattern recognition
• Image registration
• File formats

4D Images (time series)

Time series are becoming more and more commong in clinical and medical research applications. ITK has the advantage of providing support for N-D images and algorithms. However few of it has been exercised in real cases, and few 4D specific studies have been performed. This issue is relevant for

• Functional MRI
• Perfusion images
• Cardiac 4D
  • MRI
  • CT
• 3D Ultrasound + time
• PET

Tensor Images

Diffusion tensor images are probably the best known variant of tensor images. However, a large number of other physical properties of tissue are represented with Tensors, for example

• Electrical conductivity
• Refraction index
• Mechanical resistance

It is conceivable that images measuring such parameters will become availabe in the near future.

ITK currently has expanded its support for DTI (Diffusion Tensor Images), but much remains to be done in order to provide all the functionalities needed for helping users to develop medical applications.

The challenges are

• File formats
• Image size
• Tensor operations
• Tractography
• Tensor based Segmentation
• Tensor based Registration
  • Tensor to Tensor images
  • Tensor to Vector images
  • Tensor to Scalar images
• Feature detection

Microscopy Images

The specific features of microscopy images are

• 2D images
• Large images (1K x 1K) up to (4K to 4K)
• Mosaics
• Need for annotations
• Movement and tracking of particles
• Multi-spectral images
• Polarization
• File formats (the microscopy community uses a large set of fileformats)

Confocal Microscopy Images

The specific features of Confocal Microscopy images are

• 3D images
• Strong anisotropy (Z spacing >> X,Y spacing)
• Attenuation in Z
• Large images
• Multi-spectral images
• Need for registration
• 3D Mosaics
• File formats (the microscopy community uses a large set of fileformats)

Transmision Electron Microscopy

The specific features of Transmission Electron Microscopy images are

• 3D images
• Small signal to noise ratio
• Large images
• Need for model based segmentation techniques

Endoscopic Images

The specific features of Endoscopic Microscopy images are

• Video images
• Color
• Reflections in tissue
• Continues movement
• Shape from shading

Endoscopic images are more on the domain of computer vision but they are quite important in clinical applications.

Fluoroscopy Images

Fluoroscopic images are very important for image guided surgery, and for computer assisted diagnosis. Since fluoroscopy images are 2D projections of a 3D structure, they have specific challenges.

• Time varying images (video)
• Geometric distorsion
• Need for detection and tracking of instruments
• 3D / 2D registration with 3D preoperative images

Mamography Images

Very few is availble in ITK in order to support Mamograms. This is an area of significant public health relevance and deserves more attention.

Particular issues are

• Large images
• 12 bits / 16 bits
• Contrast enhancement
• Calcification detection
• Significance for Biopsies (assist localization)

Perfusion Images

Perfusion images are quite important for evaluating the vascularization of tissue and for diagnosis and treatement of cardiovascular disorders.

The challenges with this images are

• Intensity normalization
• Time series
• Vascular modeling

SpatialObjects

Spatial objects are being used by a variety of projects in foreseen and unforeseen ways, and their API must be reviewed/expanded to support the unforeseen applications. Additionally, SpatialObjects remain relatively unconnected to the rest of ITK - they are easy to use as a product of ITK processing, but there is limited support for having them drive ITK processing. Specifically,

1) Enable SpatialObjects to be passed along ITK's pipeline. This requires the definition of an itkSpatialObjectSource class as well as derived classes such as itkImageToSpatialObjectFilter

2) Implement SpatialObject traits. SpatialObjects use the itkTree data structure; however, methods for querying that tree are very limited. Currently spatialObject type definition is maintained by a class-specific string accessed via GetTypeName(). A standard naming structure or other form of type designation (e.g., traits) is needed so that filters can be tuned by spatialObject type. For example, certain filters can work on Tube, DTITube, and VesselTube spatialObjects (these classes have a common derivation), but the current GetTypeName designations do not expose their common derivation.

Neural Networks

Neural network methods will be added to ITK. These include multilayered neural networks with linear, sigmoid, and radial basis transfer functions. Training strategies such as backpropagation and quickprop. Also Kohonen self-organizing feature maps will be added. Helper-classes will simplify the implementation and use of traditional 3-layer backprop, non-linear component analysis, radial basis function, and feature map networks.

DICOM Query and Retrieve

A file-dialog-style FLTK-based tool will be provided for dicom query and retrieve. This tool will be require the use of DCMTK. Using this tool, studies on a remote dicom server can be searched, retrieved, and loaded by an ITK application with minimal programming.

What to include in ITK 2.2 (Scheduled for September 2005)

Tensor Images

• Symmetric Second Rank Pixel
• Diffusion Tensor Pixel
• Eigen Values Calculator
• Image IO for Tensor Images
• Support for registration of Tensor Images

FITS File Format

This file format is mostly used in Astronomy, but has the among others, the following interesting features

• Supporting streaming. That is, we can load block of data from the files.
• Supports N-Dimensional data blocks
• Supports Multiple components per pixel

This file format seems to be suitable for managing Tensor images, Microscopy images, and very large datasets.