ITK/Release 5/DICOM/Color: Difference between revisions
| (21 intermediate revisions by 2 users not shown) | |||
| Line 6: | Line 6: | ||
By design the itk::GDCMImageIO was designed and implemented so that an input YBR_FULL image would be loaded as such (no implicit conversion to RGB was done). The main reason for that is that ITK is a processing toolkit, so quantitative analysis is supposed to be done on the best possible pixel representation. Since conversion from integer YBR_FULL to integer RGB colorspace is a lossy operation (truncation in floating point representation), the conversion has never been implemented. | By design the itk::GDCMImageIO was designed and implemented so that an input YBR_FULL image would be loaded as such (no implicit conversion to RGB was done). The main reason for that is that ITK is a processing toolkit, so quantitative analysis is supposed to be done on the best possible pixel representation. Since conversion from integer YBR_FULL to integer RGB colorspace is a lossy operation (truncation in floating point representation), the conversion has never been implemented. | ||
As side note MONOCHROME1 color space is not converted to MONOCHROME2 (itk::GDCMImageIO / ITK 5.0) | |||
ITK currently does not have any standard way of noting the color space of an Image. Originally, there was a direct mapping between the `ImageIOBase::PixelType` and the intended pixel type used for the template parameter in the `Image` class. Modern usage of ITK frequently uses the generalized `VectorImage` to hold multi-component images so there may even be less knowledge of the content in the Image buffer. Additionally, we need to consider the default behavior of `ConverPixeltBuffer` (https://itk.org/Doxygen/html/classitk_1_1ConvertPixelBuffer.html), which may be applied when an image is loaded by the `ImageFileReader`. For example if the ImageIO loads a 3 component pixel buffer and the ImageFileReader is templated over a scalar image default a RGB to Gray scale conversion occurs based on the CIE luminance for the 3-components. | |||
== Issue == | == Issue == | ||
This behavior was considered ok, as long as the image was not directly loaded in viz application such as Slicer, where | This behavior was considered ok, as long as the image was not directly loaded in viz application such as Slicer, where unexpectedly the image would appears with a weird color scheme. (original bug report from Mihail Isakov) | ||
BCL: After hearing the descriptions of the issue, I considered loading YBR directly as a bug because GDCMImageIO was reporting the pixel type as `IOPixelType::RGB` and the buffer was in YBR color space. If the image is loaded in YBR color space is should have a different `IOPixelType` may be `VECTOR` or `FIXEDARRAY` for lack of a more descriptive `IOPixelType`. | |||
== Notation == | == Notation == | ||
| Line 30: | Line 36: | ||
! MONOCHROME2 || style="background:lime" | MONOCHROME2 || style="background:lime" |MONOCHROME2 (aka min-is-black) || style="background:lime" |MONOCHROME2 || style="background:lime" |MONOCHROME2 | ! MONOCHROME2 || style="background:lime" | MONOCHROME2 || style="background:lime" |MONOCHROME2 (aka min-is-black) || style="background:lime" |MONOCHROME2 || style="background:lime" |MONOCHROME2 | ||
|- | |- | ||
! | ! PALETTE COLOR || style="background:silver" |X || style="background:orange" | RGB (??) || style="background:orange" | Converted to RGB || style="background:orange" | ?Converted to RGB? (need to check compressed LUT) | ||
|- | |- | ||
! RGB || style="background:orange" |RGB || RGB|| RGB || ?RGB? | ! RGB || style="background:orange" |RGB || RGB|| RGB || ?RGB? | ||
| Line 48: | Line 54: | ||
! YBR_PARTIAL_420 || || || || | ! YBR_PARTIAL_420 || || || || | ||
|- | |- | ||
! YBR_ICT || style="background:silver" |ITU 81 does not support this colorspace || ?? || RGB || style="background:red" |no J2K plugin | ! YBR_ICT || style="background:silver" |ITU 81 does not support this colorspace || ?? || style="background:orange" |RGB || style="background:red" |no J2K plugin | ||
|- | |- | ||
! YBR_RCT || style="background:silver" |ITU 81 does not support this colorspace || ?? || RGB || style="background:red" |no J2K plugin | ! YBR_RCT || style="background:silver" |ITU 81 does not support this colorspace || ?? || RGB || style="background:red" |no J2K plugin | ||
| Line 61: | Line 67: | ||
! Photometric Interpretation !! JPEG !! TIFF !! GDCM !! DCMTK | ! Photometric Interpretation !! JPEG !! TIFF !! GDCM !! DCMTK | ||
|- | |- | ||
! MONOCHROME1 || style="background:silver" | X || style="background:orange" |MONOCHROME2 (aka min-is-black)|| style="background: | ! MONOCHROME1 || style="background:silver" | X || style="background:orange" |MONOCHROME2 (aka min-is-black)|| style="background:orange" |MONOCHROME2 || style="background:orange" |MONOCHROME2 | ||
|- | |- | ||
! MONOCHROME2 || style="background:lime" | MONOCHROME2 || style="background:lime" |MONOCHROME2 (aka min-is-black)|| style="background:lime" |MONOCHROME2 || style="background:lime" |MONOCHROME2 | ! MONOCHROME2 || style="background:lime" | MONOCHROME2 || style="background:lime" |MONOCHROME2 (aka min-is-black)|| style="background:lime" |MONOCHROME2 || style="background:lime" |MONOCHROME2 | ||
|- | |- | ||
! | ! PALETTE COLOR || style="background:silver" | X || style="background:orange" | RGB || style="background:orange" | Converted to RGB || style="background:orange" | RGB | ||
|- | |- | ||
! RGB || style="background:orange" |RGB || RGB|| RGB || ?RGB? | ! RGB || style="background:orange" |RGB || RGB|| RGB || ?RGB? | ||
| Line 75: | Line 81: | ||
! CMYK || || || || | ! CMYK || || || || | ||
|- | |- | ||
! YBR_FULL || style="background:silver" |ITU 81 enforce sub-sampling|| ?? || RGB || | ! YBR_FULL || style="background:silver" |ITU 81 enforce sub-sampling|| ?? || style="background:orange" |RGB || style="background:orange" |RGB | ||
|- | |- | ||
! YBR_FULL_422 || style="background:orange" |RGB || style="background:orange" |RGB || RGB || | ! YBR_FULL_422 || style="background:orange" |RGB || style="background:orange" |RGB || style="background:orange" |RGB || style="background:orange" |RGB | ||
|- style="background:silver" | |- style="background:silver" | ||
! YBR_PARTIAL_422 || || || || | ! YBR_PARTIAL_422 || || || || | ||
| Line 83: | Line 89: | ||
! YBR_PARTIAL_420 || || || || | ! YBR_PARTIAL_420 || || || || | ||
|- | |- | ||
! YBR_ICT || || || RGB || style="background:red" |no J2K plugin | ! YBR_ICT || || || style="background:orange" |RGB || style="background:red" |no J2K plugin | ||
|- | |- | ||
! YBR_RCT || || || RGB || style="background:red" |no J2K plugin | ! YBR_RCT || || || RGB || style="background:red" |no J2K plugin | ||
| Line 89: | Line 95: | ||
|} | |} | ||
When a Photometric Interpretation is specified in red, it means this is not supposed to happen (does not make any sense). | |||
When a Photometric Interpretation is specified in silver background, it means the Photometric Interpretation is retired. | |||
== Discussion == | == Discussion == | ||
Participants: | |||
# Mathieu Malaterre | |||
# Mihail Isakov | |||
# Bradley Lowekamp | |||
For visualization people it is important that: | For visualization people it is important that: | ||
| Line 102: | Line 115: | ||
# ? Need MONOCHROME1 native support, because <fill reason ??> | # ? Need MONOCHROME1 native support, because <fill reason ??> | ||
# ? Is there a need to manipulated native palette data ? I doubt so | # ? Is there a need to manipulated native palette data ? I doubt so | ||
## BL: The `ExpandRGBPalette` attribute was just added to `ImageIOBase`. TIFFImageIO support this option. For feature consistency it would be nice for GDCMImageIO to support this feature. https://itk.org/Doxygen/html/classitk_1_1ImageIOBase.html#a04804c60d755ae102e104261536a0f46 https://github.com/InsightSoftwareConsortium/ITK/pull/1219/files | |||
For ITK consistency it is important that the `ImageIOBase::PixelType` is set correctly so that users of the `ITK::ImageIOBase` API know generally what the image buffer is and how to load and use the image. We likely should improve the documentation in the ImageIOBase class about the expectations for how these are used. Here is a list of type pertinent to this discussion: | |||
# RGB | |||
## BL: should only be used for general RGB buffers. This is not very descriptive of the color space sRGB, AdobeRGB, Linear RGB etc., but using this as a type for general "color" buffers should be avoided IMHO. | |||
# SCALAR | |||
## BL: Images of unapplied palettes, when ExpandRGBPalette is false, are currently labeled as scalars | |||
# VECTOR | |||
## BL: Currently this is frequently used for displacement fields. | |||
# FIXEDARRAY | |||
## BL: Not regularly used to my knowledge, and may be the most generic multi-component `PixelType`, so this may be the base label to used for YBR, CMYK etc. | |||
== Data == | |||
# JPEG 8 bits mono2 | |||
# JPEG 8 bits YBR_422 | |||
# JPEG 8 bits RGB | |||
# TIFF mono1 min-is-white | |||
# TIFF mono2 min-is-black | |||
# TIFF YBR_422 | |||
# TIFF RGB | |||
# DICOM mono1 min-is-white | |||
# DICOM mono2 min-is-black | |||
# DICOM YBR | |||
# [https://sourceforge.net/p/gdcm/gdcmdata/ci/master/tree/US-YBR_FULL_422-EVRLE.dcm| DICOM YBR_422] | |||
# [https://sourceforge.net/p/gdcm/gdcmdata/ci/master/tree/US-GE-4AICL142.dcm| DICOM JPEG/YBR_422] | |||
# DICOM RGB | |||
# DICOM YBR_RCT | |||
# DICOM YBR_ICT | |||
== References == | == References == | ||
* http://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.3.html#sect_C.7.6.3.1.2 | |||
* http://gdcm.sourceforge.net/wiki/index.php/Color_Space_Transformations | * http://gdcm.sourceforge.net/wiki/index.php/Color_Space_Transformations | ||
Latest revision as of 14:37, 11 October 2019
Multiple Components in DICOM
ITK 5.0
DICOM standard allow storing of image with more than one components. As of today, only 1 or 3 components are considered valid DICOM SOP Class instances (for all IODs).
By design the itk::GDCMImageIO was designed and implemented so that an input YBR_FULL image would be loaded as such (no implicit conversion to RGB was done). The main reason for that is that ITK is a processing toolkit, so quantitative analysis is supposed to be done on the best possible pixel representation. Since conversion from integer YBR_FULL to integer RGB colorspace is a lossy operation (truncation in floating point representation), the conversion has never been implemented.
As side note MONOCHROME1 color space is not converted to MONOCHROME2 (itk::GDCMImageIO / ITK 5.0)
ITK currently does not have any standard way of noting the color space of an Image. Originally, there was a direct mapping between the `ImageIOBase::PixelType` and the intended pixel type used for the template parameter in the `Image` class. Modern usage of ITK frequently uses the generalized `VectorImage` to hold multi-component images so there may even be less knowledge of the content in the Image buffer. Additionally, we need to consider the default behavior of `ConverPixeltBuffer` (https://itk.org/Doxygen/html/classitk_1_1ConvertPixelBuffer.html), which may be applied when an image is loaded by the `ImageFileReader`. For example if the ImageIO loads a 3 component pixel buffer and the ImageFileReader is templated over a scalar image default a RGB to Gray scale conversion occurs based on the CIE luminance for the 3-components.
Issue
This behavior was considered ok, as long as the image was not directly loaded in viz application such as Slicer, where unexpectedly the image would appears with a weird color scheme. (original bug report from Mihail Isakov)
BCL: After hearing the descriptions of the issue, I considered loading YBR directly as a bug because GDCMImageIO was reporting the pixel type as `IOPixelType::RGB` and the buffer was in YBR color space. If the image is loaded in YBR color space is should have a different `IOPixelType` may be `VECTOR` or `FIXEDARRAY` for lack of a more descriptive `IOPixelType`.
Notation
- JPEG refers to the subset of ITU-T T.81, ISO/IEC IS 10918-1 (lossy 8bits)
- JPEG 2000 refers to ITU-T T.800, ISO/IEC IS 15444-1
ITK 5.x
The behavior for YBR_FULL vs RGB color model has been discussed and it seems consensus would be to convert to RGB always. Since DICOM, JPEG and TIFF can be somewhat relates for this matter, we should strive to keep the behavior consistent.
Current behavior of ITK 5.0:
| Photometric Interpretation | JPEG | TIFF | GDCM | DCMTK |
|---|---|---|---|---|
| MONOCHROME1 | X | MONOCHROME2 (aka min-is-black) | MONOCHROME1 | ?MONOCHROME1? |
| MONOCHROME2 | MONOCHROME2 | MONOCHROME2 (aka min-is-black) | MONOCHROME2 | MONOCHROME2 |
| PALETTE COLOR | X | RGB (??) | Converted to RGB | ?Converted to RGB? (need to check compressed LUT) |
| RGB | RGB | RGB | RGB | ?RGB? |
| HSV | ||||
| ARGB | ||||
| CMYK | ||||
| YBR_FULL | ITU 81 enforce sub-sampling | ?? | YBR_FULL | ?? |
| YBR_FULL_422 | RGB | RGB | YBR_FULL | ??Maybe RGB? |
| YBR_PARTIAL_422 | ||||
| YBR_PARTIAL_420 | ||||
| YBR_ICT | ITU 81 does not support this colorspace | ?? | RGB | no J2K plugin |
| YBR_RCT | ITU 81 does not support this colorspace | ?? | RGB | no J2K plugin |
Desired behavior of ITK 5.0:
| Photometric Interpretation | JPEG | TIFF | GDCM | DCMTK |
|---|---|---|---|---|
| MONOCHROME1 | X | MONOCHROME2 (aka min-is-black) | MONOCHROME2 | MONOCHROME2 |
| MONOCHROME2 | MONOCHROME2 | MONOCHROME2 (aka min-is-black) | MONOCHROME2 | MONOCHROME2 |
| PALETTE COLOR | X | RGB | Converted to RGB | RGB |
| RGB | RGB | RGB | RGB | ?RGB? |
| HSV | ||||
| ARGB | ||||
| CMYK | ||||
| YBR_FULL | ITU 81 enforce sub-sampling | ?? | RGB | RGB |
| YBR_FULL_422 | RGB | RGB | RGB | RGB |
| YBR_PARTIAL_422 | ||||
| YBR_PARTIAL_420 | ||||
| YBR_ICT | RGB | no J2K plugin | ||
| YBR_RCT | RGB | no J2K plugin |
When a Photometric Interpretation is specified in red, it means this is not supposed to happen (does not make any sense). When a Photometric Interpretation is specified in silver background, it means the Photometric Interpretation is retired.
Discussion
Participants:
- Mathieu Malaterre
- Mihail Isakov
- Bradley Lowekamp
For visualization people it is important that:
- MONOCHROME1 (min-is-white) is converted to MONOCHROME2 (min-is-black)
- YBR_FULL / YBR_FULL_422 are converted to RGB
The requirements for quantitative analysis may be different, please list reqs here:
- ? Extract Luminance out of YBR ?
- ? Need MONOCHROME1 native support, because <fill reason ??>
- ? Is there a need to manipulated native palette data ? I doubt so
- BL: The `ExpandRGBPalette` attribute was just added to `ImageIOBase`. TIFFImageIO support this option. For feature consistency it would be nice for GDCMImageIO to support this feature. https://itk.org/Doxygen/html/classitk_1_1ImageIOBase.html#a04804c60d755ae102e104261536a0f46 https://github.com/InsightSoftwareConsortium/ITK/pull/1219/files
For ITK consistency it is important that the `ImageIOBase::PixelType` is set correctly so that users of the `ITK::ImageIOBase` API know generally what the image buffer is and how to load and use the image. We likely should improve the documentation in the ImageIOBase class about the expectations for how these are used. Here is a list of type pertinent to this discussion:
- RGB
- BL: should only be used for general RGB buffers. This is not very descriptive of the color space sRGB, AdobeRGB, Linear RGB etc., but using this as a type for general "color" buffers should be avoided IMHO.
- SCALAR
- BL: Images of unapplied palettes, when ExpandRGBPalette is false, are currently labeled as scalars
- VECTOR
- BL: Currently this is frequently used for displacement fields.
- FIXEDARRAY
- BL: Not regularly used to my knowledge, and may be the most generic multi-component `PixelType`, so this may be the base label to used for YBR, CMYK etc.
Data
- JPEG 8 bits mono2
- JPEG 8 bits YBR_422
- JPEG 8 bits RGB
- TIFF mono1 min-is-white
- TIFF mono2 min-is-black
- TIFF YBR_422
- TIFF RGB
- DICOM mono1 min-is-white
- DICOM mono2 min-is-black
- DICOM YBR
- DICOM YBR_422
- DICOM JPEG/YBR_422
- DICOM RGB
- DICOM YBR_RCT
- DICOM YBR_ICT
