[Insight-users] Evaluation of Segmentation quality

[Nicholas Tustison]

Hi Robert and Arturo,

I just wanted to point out that some of the methods described
are available in ITK.  The relevant IJ article is available at

http://www.insight-journal.org/browse/publication/707

and the class Doxygen documentation is here:

http://www.itk.org/Doxygen/html/classitk_1_1LabelOverlapMeasuresImageFilter.html

Nick



On Aug 25, 2012, at 8:19 AM, Robert Haase wrote:

> Dear Arturo,
> 
> quality determination of segmentation results in medical images is a hot topic. From an methological point of view, you should compare the delineations of an algorithm to those from a reliable reference standard. Just looking at the images to determine segmentation quality is absolutely NOT enough.
> 
> Reliable reference standards can be based on histological specimen, other imaging modalities, where the object boundary is better visible, or [my favorite:] reference delineations from experienced physicians (radiologist, nuclear medicine physician,...). The latter allows to state that an algorithm can generate clinically reliable delineations. Of course, only if the algorithms contours fit those from the physicians.
> 
> The actual comparison between your segmentation results and those from the reference can be based on 
> * Jaccard Index or Dice Index as measures of object overlap,
> * Volume / area error measures (relative or absolute),
> * Symmetrical mean contours distance or maximum Haussdorf distance as measures of contour distance, e.g. given in millimeters,
> * Voxel-wise sensitivity and specificity for determination of well known statistical measures
> * Resulting from sensitvity and specificity, Receiver-Operating-Characteristics-Curves may be available. These may show you impressively how reliable your algorithm really is.
> 
> Depending on which field you are working in, there may be more measures. For example in the field of segmentation of PET images, the mean activity concentration of corresponding contours may be relevant. For lung tumors in gated CT-data, a shift of the geometric center of the contour may describe therapeutical decisive parameters of an object. At the end, it really depends on the field where you are working in, which method fits your needs best. A detailed look in the scientific literature should answer this question.
> 
> However, I'm afraid that these methods are not available in ITK. But implementation is simple. If you loop over all voxels/pixels and the corresponding segmentation results, it should be easy to count true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN). Afterwards, almost all measures I stated above can be calculated. Jaccard Index, for example is:
> 
> JI = TP / (TP + FN + FP)
> 
> For the implementation of contour/Haussdorf distances, a little more experience on efficient computing is needed.
> 
> Best regards,
> Robert
> 
> 
> 
> 
> 
> -------- Original-Nachricht --------
>> Datum: Fri, 24 Aug 2012 19:06:03 -0500
>> Von: "Ing. Arturo Vargas Olivares" <ing.arturo.vargas at gmail.com>
>> An: insight-users at itk.org
>> Betreff: [Insight-users] Evaluation of Segmentation quality
> 
>> Hi community.
>> 
>> In segmentation of medical images, is there any recommended method to
>> evaluate the quality of the segmentation applied? Do you know if there is
>> a
>> method or technique included in 'ITK' or 'SimpleITK'?
>> 
>> Comments are appreciated.
>> 
>> Arturo.
> _____________________________________
> Powered by www.kitware.com
> 
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
> 
> Kitware offers ITK Training Courses, for more information visit:
> http://www.kitware.com/products/protraining.php
> 
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
> 
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-users