Multi-modal 3D image registration using mutual information maximization

This application illustrates the use of the ITK registration framework, specifically, the mutual information maximization algorithm to register multi-modality three-dimensional medical images.

What is Mutual Information?
Mutual information is a measure of how much information one random variable tells about another. The use of mutual information for medical image registration applications was independently introduced in 1995 by both Viola and Wells [1] and Collignon [2]. For two images, the mutual information is computed from the joint probability distribution of the images’ intensity or gray-values. When two images are aligned, the joint probability distribution is “peaky” resulting in a high mutual information value. Misregistration causes the distribution to disperse resulting in a low mutual information value.

One of the main advantages of using mutual information is than it can be used to align images of different modalities (e.g. CT to MR-T1, MR-T1 to PET etc).

More information on mutual information based registration can be found at the following web sites:
  • Paul Viola’s homepage contains links to his Ph.D. thesis, papers and pointers to other mutual information resources

  • Sebastien Gilles’s web tutorial

  • This page from the Image Sciences Institute of Utrecht University has an animation of the effect of misregistration on the joint probability distribution.
Application Overview
The ITK toolkit provides an application called MultiResMIRegistration which is command line application demonstrating the use of ITK components for performing rigid multi-modal registration using mutual information.

This application reads in two 3D raw image volumes: the target volume and the reference volume. The application then iteratively estimates the rigid transform that will align the reference onto the target volume.

The application terminates after a completing a user-defined number of iterations. The estimated rigid transform is applied to the reference, the resulting registered image is then written out to file as a raw 3D image volume.

Optionally, each 2D slice form the target, this and registered image volume can be written out as PGM files – facilitating viewing with simple 2D image viewers.

What Components of ITK does this Application?
This application makes use of the ITK registration framework and ITK multi-resolution framework.

ITK Registration Framework
The ITK registration framework is a generic framework for registering images and point sets. A registration algorithm is built from three components: a Transform, a Metric and an Optimizer. Standard component API’s allows mixing and matching of the components.

The part of the registration framework used by this application are the itk::QuaternionRigidTransform, the itk::MutualInformationImageToImageMetric and the itk::GradientDescentOptimizer.

ITK Multi-Resolution Registration Framework
Performing image registration using a multi-resolution strategy has been widely shown to improve speed, accuracy and robustness. The ITK Multi-resolution registration framework is a generic framework for defining a multi-resolution registration scheme.

There are two major components in the framework: the underlying registration method and the down-sampling strategy. Any RegistrationMethod that meets the minimal API requirements can plug into the multi-resolution framework. The down-sampling strategy is encapsulated in a MultiResolutionPyramidImageFilter object. A flexible schedule-based scheme allows the user to define the shrink factors at each level of the pyramid.

The multi-resolution scheme used in this application is defined in itk::MultiResolutionImageRegistrationMethod.

Application Experiments
The Vanderbilt Retrospective Registration Evaluation Project contains multi-modal images (CT, MR-T1, MR-T2, MR-PD, PET) of ten patients. The images used in this experiment are a part of that study.

The first example shows the registration of a CT series to a T1 weighted MR image series from the practice patient of the Vanderbilt dataset. The registration was done in three dimensions. The following are sample slices from the 3D volumes.

CT Image MR Image Registered Image
Slice 7 from CT
Slice 7 from T1-MR
Slice 7 from registered CT

This second example shows the registration of a MR Proton Density (PD) image series to a PET image series. Again, the registration was done in three dimensions and the following are sample slices from the 3D volume.

PET Image MR-PD Image Registered-PET Image
Slice 7 from PET
Slice 7 from PD-MR
Slice 7 from registered PET

[1] P. Viola and W.M. Wells III, "Alignment by maximization of mutual information", International Conference on Computer Vision (E. Grimson, S. Shafer, A. Blake and K. Sugihara, eds.), IEEE Computer Society Press, Los Alamitos, CA, pp. 16-23, 1995.

[2] A. Collignon, F. Maes, D. Delaere, D. Vandermeulen, P. Suetens and G. Marchal, "Automated multimodality image registration based on information theory", Information Processing in Medical Imaging (Y. Bizais, C. Barillot and R. Di Paola, eds.), Kluwer Academic Publishers, Dordrecht, pp. 263-274, 1995.

Go to next application.