TubeTK/Intra-operative Ultrasound Registration

Intra-operative Ultrasound Registration

The Problem

During preparation for a surgical operation such as liver tumor radiofrequecy abation or surgical resection of a brain tumor, pre-operative medical images are acquired to help inform both the radiologist and also the surgeon involved in the treatment. The radiologist can use this diagnostic image to identify and evaluate the extent of a lesion. A surgeon will use both lesion segmentation information provided by the radiologist along with patient-specific anatomical information found in the pre-operative image to plan her operation.

While the surgeon tries to hold a mental picture of the internal structures she will be encountering as she penetrates the body, but it would be very helpful if the surgeon could see where the structures examined in the pre-operative image lie with respect to knives, needles, caughterizing agents or other tools intended to deliver damage. In this way, damage is delivered precisely and completely to the problematic tissues with minimal collatoral damage to healthy tissue.

In a pre-operative situation, it is possible to collect well-sampled 3D volumes with modalities like X-ray computed tomography (CT) or magnetic resonance images (MRI) that can be critically examined. During an operation, this is not the case. It is essential that the patient spends as little time as possible under anesthesia and with the skin opened. Also, the imaging device cannot occlude surgical instrumentation and the actions of the surgical team. Ultrasound imaging is well-suited for this situation because of its real-time performance and small spatial footprint. Yet, the ultrasound images are usually only 2D, have a limited field of view, noisy, and do not have the annotated segmentations created in the pre-operative image. The grand challenge addressed here is the identification of the spatial transformation that maps the pre-operative image to the intra-operative real-time ultrasound image, i.e. the spatial registration problem. After this mapping is known, the pre-operative image can be overlayed on the intra-operative ultrasound to give the surgeon a more complete picture of where their instruments are located.

The confounding demands in this problem include:

Real-time performance

Even though a computationally complex analysis is required, real-time performance is essential so feedback is presented to the surgeon as she is manipulating her tools and probing their location relative to the insight provided by the images.

Sparse and incomplete datasets

As previously mentioned, while the pre-operative image may be a well-sampled volume, the intra-operative ultrasound is typically 2D or a limited 3D view of the area of interest. The ultrasound imaging plane cannot be directly in line with the plane of interest because the surgical tool is in this plane. Incomplete or imperfect coupling of the transducer with the skin results in signal dropout as does acoustic shadowing from intervening structures or surgical instruments.

Multi-modality registration

We are often posed with the registration of a CT or MRI image against an ultrasound (US) image. Tissue's presentation in these modalities drastically vary, which makes it difficult to compare them.

Related Work

Noise reduction

• Noise reduction presentations
  • http://www.2dix.com/ppt2011/speckle-noise-reduction-in-ultrasound-images-ppt.php
• A View on Despeckling in Ultrasound Imaging
  • http://www.doaj.org/doaj?func=abstract&id=590052
  • S.Kalaivani Narayanan ; R.S.D.Wahidabanu
  • Ultrasound imaging is a widely used and safe medical diagnostic technique, due to its noninvasive nature, low cost and capability of forming real time imaging. However the usefulness of ultrasound imaging is degraded by the presence of signal dependant noise knownas speckle. The speckle pattern depends on the structure of the image tissue and various imaging parameters. There are two main purposes for speckle reduction in medical ultrasound imaging (1) to improve the human interpretation of ultrasound images (2) despeckling is the preprocessing step for many ultrasound image processing tasks such as segmentation and registration. A number of methods have been proposed for speckle reduction in ultrasoundimaging. While incorporating speckle reduction techniques as an aid for visual diagnosis, it has to keep in mind that certain speckle contains diagnostic information and should be retained. The objective of this paper is to give an overview about types of speckle reduction techniques in ultrasound imaging.
• Noise suppression and motion estimation in medical ultrasound imaging
  • https://dspacedev.rice.edu/handle/1911/20673
  • Echocardiographic imaging is a primary modality in the diagnosis of heart disease. Compared to other imaging techniques, such as X-Ray, MRI, and PET, ultrasound imaging owes its great popularity to the fact that it is a safe and non-invasive procedure for visualizing the heart and vasculature. The ultrasound image however is corrupted by speckle, which is distinguished from Gaussian noise by its signal-dependent nature. This dissertation focuses on two important issues for the clinical applications of medical ultrasound images: speckle suppression and motion estimation. The dissertation first describes the statistics of speckle and ultrasound image models, which are important for performance evaluation and further algorithm development. Secondly, a novel speckle suppression approach is developed for the purpose of visualization enhancement and auto-segmentation improvement. This method is designed to utilize the favorable denoising properties of two frequently used techniques: wavelet and nonlinear diffusion. Speckle is iteratively reduced by the multiscale nonlinear diffusion via the framework of dyadic wavelet transform. With a noise adaptive feature, our algorithm is versatile for both envelop-detected and log-compressed ultrasound images. We validate our method using synthetic speckle images and real ultrasonic images. Performance improvement over other despeckling filters is quantified in terms of the quality indices. In summary, our algorithm provides very significant speckle suppression and edge enhancement for the purposes of visualization and automatic structure detection. We further extend the ultrasound statistical knowledge into the motion estimation, and develop a speckle tracking algorithm for myocardial wall motion estimation in intracardiac echocardiographic images. To achieve robust noise resistance, we employ maximum likelihood estimation while fully exploiting ultrasound speckle statistics, and treat the maximization of motion probability as the minimization of an energy function. Non-rigid myocardial deformation is estimated by optimizing this energy function within a framework of elastic registration. Accuracy of the method is evaluated by using a computer model and an animal model, which provides continuous intracardiac echocardiographic images as well as reference measurements for myocardial deformation. As a result, our approach achieves an accurate estimation of regional myocardial deformation from intracardiac echocardiography. This approach has important clinical implications for multimodal imaging during catheterization.

Surface-to-ultrasound registration

• Evaluation on Similarity Measures of a Surface-to-Image Registration Technique for Ultrasound Images.
  • Wei Shao, Ruoyun Wu, Keck Voon Ling, Choon Hua Thng, Henry Sun Sien Ho, Christopher Wai Sam Cheng, Wan Sing Ng
  • MICCAI (2) 2006: 742-749
• Medical image registration based on a creaseness measure
  • Lloret i Vilallonga, David
  • Thesis
  • http://www.tesisenxarxa.net/TDX/TDX_UAB/TESIS/AVAILABLE/TDX-0301103-120226/
  • http://www.tdx.cat/TDX-0301103-120226

Ultrasound simulation

• http://campar.in.tum.de/Chair/ProjectSimulatedUltrasound
• Real-Time Simulation of Medical Ultrasound from CT Images
  • Lecture Notes In Computer Science archive. Proceedings of the 11th International Conference on Medical Image Computing and Computer-Assisted Intervention, Part II, New York, New York, Pages: 734 - 741
  • Ramtin Shams, Richard Hartley
    • RSISE, The Australian National University, Canberra, and NICTA, Canberra, Australia
  • Nassir Navab, Computer Aided Medical Procedures (CAMP), TU München, Germany
  • Medical ultrasound interpretation requires a great deal of experience. Real-time simulation of medical ultrasound provides a cost-effective tool for training and easy access to a variety of cases and exercises. However, fully synthetic and realistic simulation of ultrasound is complex and extremely time-consuming. In this paper, we present a novel method for simulation of ultrasound images from 3D CT scans by breaking down the computations into a preprocessing and a run-time phase. The preprocessing phase produces detailed fixed-view 3D scattering images and the run-time phase generates view-dependent ultrasonic artifacts for a given aperture geometry and position within a volume of interest. We develop a simple acoustic model of the ultrasound for the run-time phase, which produces realistic ultrasound images in real-time when combined with the previously computed scattering image.
  • http://portal.acm.org/citation.cfm?id=1483392
• Advanced training methods using an Augmented Reality ultrasound simulator
  • Blum, T.; Heining, S.M.; Kutter, O.; Navab, N.; Comput. Aided Med. Procedures & Augmented Reality (CAMP), Tech. Univ. Munchen, Munich, Germany
  • This paper appears in: Mixed and Augmented Reality, 2009. ISMAR 2009. 8th IEEE International Symposium on: 177 - 178
  • Ultrasound (US) is a medical imaging modality which is extremely difficult to learn as it is user-dependent, has low image quality and requires much knowledge about US physics and human anatomy. For training US we propose an Augmented Reality (AR) ultrasound simulator where the US slice is simulated from a CT volume. The location of the US slice inside the body is visualized using contextual in-situ techniques. We also propose advanced methods how to use an AR simulator for training.
  • http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5336476
• Registration of 3D ultrasound to computed tomography images of the kidney
  • http://circle.ubc.ca/handle/2429/27817?show=full
  • The integration of 3D computed tomography (CT) and ultrasound (US) is of considerable interest because it can potentially improve many minimally invasive procedures such as robot-assisted laparoscopic partial nephrectomy. Partial nephrectomy patients often receive preoperative CT angiography for diagnosis. The 3D CT image is of high quality and has a large field of view. Intraoperatively, dynamic real-time images are acquired using ultrasound. While US is real-time and safe for frequent imaging, the images captured are noisy and only provide a limited perspective. Providing accurate registration between the two modalities would enhance navigation and image guidance for the surgeon because it can bring the pre-operative CT into a current view of the patient provided by US.
  • The challenging aspect of this registration problem is that US and CT produce very different images. Thus, a recurring strategy is to use preprocessing techniques to highlight the similar elements between the images. The registration technique presented here goes further by dynamically simulating an US image from the CT, and registering the simulated image to the actual US. This is validated on US and CT volumes of porcine phantom data. Validation on realistic phantoms remains an ongoing problem in the development of registration methods. A detailed protocol is presented here for constructing tissue phantoms that incorporate contrast agent into the tissue such that the kidneys appear representative of in vivo human CT angiography. Registration with 3D CT is performed successfully on the reconstructed 3D US volumes, and the mean TREs ranged from 1.8 to 3.5 mm. In addition, the simulation-based algorithm was revised to consider the shape of the US beam by using pre-scan converted US data. The corresponding CT image is iteratively interpolated along the direction of the US beam during simulation. The mean TREs resulting from registering the pre-scan US data and CT data were between 1.4 to 2.6 mm. The results show that both methods yield similar results and are promising for clinical application. Finally, the method is tested on a set of in vivo CT and US images of a partial nephrectomy patient, and the registration results are discussed.
• http://portal.acm.org/citation.cfm?id=1844544&CFID=102173375&CFTOKEN=74693374
  • Rigid registration of segmented volumes in frequency domain using spherical correlation
  • Proceedings of the 12th WSEAS international conference on Mathematical methods, computational techniques and intelligent systems table of contents, Pages: 234-238
  • An algorithm for the rigid registration of binary volumes is described in this paper. Binary volumes result from a segmentation of ovarian ultrasound volumes. Rigid registration is preformed in frequency domain, where the rotation and translation can be calculated separately. The calculation of rotation is done using the amplitude spectrum and with the help of sphere correlation. The method was tested on 100 synthetic ultrasonic volume pairs. Registration accuracy was estimated by a ratio ρ that compares the intersection volume of the two registered volumes to the final volume. The average ratio ρ between registered volumes was 0.50 (std 0.09) when final result of registration was used. For comparison we tested transformation, used in synthetic volumes creation. The average ratio ρ was 0.53 (std. 0.08) in that case.

Registration

• http://wwwx.cs.unc.edu/~mn/sites/default/files/lee2010_physically-based-deformable-image-registration.pdf
  • Physically-based deformable image registration with material properties and boundary conditions
  • We propose a new deformable medical image registration method that uses a physically-based simulator and an iterative optimizer to estimate the simulation parameters determining the deformation field between the two images. Although a simulation-based registration method can enforce physical constraints exactly and considers different material properties, it requires hand adjustment of material properties, and boundary conditions cannot be acquired directly from the images. We treat the material properties and boundary conditions as parameters for the optimizer, and integrate the physically-based simulation into the optimization loop to generate a physically accurate deformation automatically.
• http://ukpmc.ac.uk/abstract/MED/18975707;jsessionid=5437290B6533DA7FFD4DBA261D257325.jvm4
  • Mutual-information-based image to patient re-registration using intraoperative ultrasound in image-guided neurosurgery.
  • An image-based re-registration scheme has been developed and evaluated that uses fiducial registration as a starting point to maximize the normalized mutual information (nMI) between intraoperative ultrasound (iUS) and preoperative magnetic resonance images (pMR). We show that this scheme significantly (p<0.001) reduces tumor boundary misalignment between iUS pre-durotomy and pMR from an average of 2.5 mm to 1.0 mm in six resection surgeries. The corrected tumor alignment before dural opening provides a more accurate reference for assessing subsequent intraoperative tumor displacement, which is important for brain shift compensation as surgery progresses. In addition, we report the translational and rotational capture ranges necessary for successful convergence of the nMI registration technique (5.9 mm and 5.2 deg, respectively). The proposed scheme is automatic, sufficiently robust, and computationally efficient (<2 min), and holds promise for routine clinical use in the operating room during image-guided neurosurgical procedures.
• http://ieeexplore.ieee.org/search/searchresult.jsp?newsearch=partialPref&queryText=ultrasound&x=0&y=0&filter=OR%28Publication+Number%3A5162127%29
  • Bioinformatics and Biomedical Engineering , 2009. ICBBE 2009. 3rd International Conference on
    1. Medical Ultrasound Image Segmentation Based on Improved Watershed Scheme
    2. Dynamic Persistence of Ultrasound Images After Local Tissue Motion Tracking
    3. Improved T-Snake Model Based Edge Detection of the Coronary Arterial Walls in Intravascular Ultrasound Images
    4. Clutter Removal of Doppler Ultrasound Signal Using Double Density Discrete Wavelet Transform
• http://portal.acm.org/citation.cfm?id=1487504&dl=ACM&coll=portal
  • Non-Rigid Ultrasound Image Registration Based on Intensity and Local Phase Information
  • Jonghye Woo Electrical Engineering, University of Southern California, Los Angeles, USA 90089-2564
  • Byung-Woo Hong School of Computer Science and Engineering, Chung-Ang University, Seoul, Korea 156-756
  • Chang-Hong Hu Biomedical Engineering, University of Southern California, Los Angeles, USA 90089-1451
  • K. Kirk Shung Biomedical Engineering, University of Southern California, Los Angeles, USA 90089-1451
  • C. -C. Kuo Electrical Engineering, University of Southern California, Los Angeles, USA 90089-2564
  • Piotr J. Slomka Department of Imaging, Cedars-Sinai Medical Center, Los Angeles, USA
  • A non-rigid ultrasound image registration method is proposed in this work using the intensity as well as the local phase information under a variational framework. One application of this technique is to register two consecutive images in an ultrasound image sequence. Although intensity is the most widely used feature in traditional ultrasound image registration algorithms, speckle noise and lower image resolution make the registration process difficult. By integrating the intensity and the local phase information, we can find and track the non-rigid transformation of each pixel under diffeomorphism between the source and target images. Experiments using synthetic and cardiac images of in vivo mice and human subjects are conducted to demonstrate the advantages of the proposed method.

RFA Monitoring

• http://onlinelibrary.wiley.com/doi/10.1111/j.1477-2574.2010.00247.x/full
  • Intra-operative ultrasound elasticity imaging for monitoring of hepatic tumour thermal ablation
  • Mark G. Van Vledder, Emad M. Boctor, Lia R. Assumpcao, Hassan Rivaz, Pezhman Foroughi, Gregory D. Hager, Ulrike M. Hamper, Timothy M. Pawlik, Michael A. Choti
  • HPB (Hepato-Pancreato-Biliary ) Volume 12, Issue 10, pages 717–723, December 2010
  • Abstract
    • Background:  Thermal ablation is an accepted therapy for selected hepatic malignancies. However, the reliability of thermal ablation is limited by the inability to accurately monitor and confirm completeness of tumour destruction in real time. We investigated the ability of ultrasound elasticity imaging (USEI) to monitor thermal ablation.
    • Objectives:  Capitalizing on the known increased stiffness that occurs with protein denaturation and dehydration during thermal therapy, we sought to investigate the feasibility and accuracy of USEI for monitoring of liver tumour ablation.
    • Methods:  A model for hepatic tumours was developed and elasticity images of liver ablation were acquired in in vivo animal studies, comparing the elasticity images to gross specimens. A clinical pilot study was conducted using USEI in nine patients undergoing open radiofrequency ablation for hepatic malignancies. The size and shape of thermal lesions on USEI were compared to B-mode ultrasound and post-ablation computed tomography (CT).
    • Results:  In both in vivo animal studies and in the clinical trial, the boundary of thermal lesions was significantly more conspicuous on USEI when compared with B-mode imaging. Animal studies demonstrated good correlation between the diameter of ablated lesions on USEI and the gross specimen (r = 0.81). Moreover, high-quality strain images were generated in real time during therapy. In patients undergoing tumour ablation, a good size correlation was observed between USEI and post-operative CT (r = 0.80).
    • Conclusion:  USEI can be a valuable tool for the accurate monitoring and real-time verification of successful thermal ablation of liver tumours.