[Insight-users] How do I validate a 3D segmentation process ?

[Gavin Baker]

Salut Mathieu,

On Tue, Mar 11, 2003 at 06:56:48PM +0100, Mathieu Malaterre wrote:

> 	I have a general question about validation in segmentation,
> and I would like you, people, to comment on this.
>
> 	I have done filters that segmented my images. I am then using a 
> software to see (at the same time) the model and image. And everything 
> 'looks' ok. But now if I play little with parameters I can find other 
> results that 'looks' good.

Visual inspection is too subjective and imprecise to use for
evaluation.  As Luis mentioned, you really need qualitative results
and compare your segmented data with a gold standard.  Either use a
phantom (an artificial subject with known shape/parameters) or get an
expert (eg. radiologist) to do it by hand.  Then you have a base with
which to compare.

> 	So my question is what is the method to validate a 'good'
> segmentation (just looking at images is too biased).

There are several metrics for comparing segmentation results.
Volumetric overlap is the simplest; given two registered objects, uou
can calculate the ratio of the intersection to the union.  Closest to
1 is better :).  There are some others described in the literature
(see below for more).

> - As I am doing 3D segmentation I don't want to do any 2D related stuff.
> - Has anyone tried registration between Mesh and Image ?
> - I have thought about using Mesh (mesh.epfl.ch) but I can't mesure 
> error to an isovalue image (I am using a dynamic isovalue to find my model).

Have you seen VALMET?  It may do what you need.  It is a free tool
produced by the good people at UNC for performing validation and
evaluating segmentation results.  The homepage is:

    http://www.ia.unc.edu/public/valmet

There is a paper describing it:

@InProceedings{gerig:valmet,
  author = 	 {Guido Gerig and Matthieu Jomier and Miranda Chakos},
  title = 	 {Valmet: A new Validation Tool for Assessing and Improving 3D Object Segmentation},
  booktitle =	 {Medical Image Computing and Computer-Assisted Intervention},
  pages =	 {516--523},
  year =	 2001,
  editor =	 {Wiro Niessen and Max Viergever},
  series =	 {LNCS 2208},
  address =	 {Utrecht, The Netherlands},
  month =	 {October},
  publisher =	 {Springer}
}

ciao,

  :: Gavin

-- 
Gavin Baker                                Computer Vision Lab (CVMIL)
http://www.cs.mu.oz.au/~gavinb                 University of Melbourne