[ITK-users] TIFFImageIO refactoring, please test

Fri Nov 14 09:53:06 EST 2014

Greetings,

I have recently refactored ITK's TIFFImageIO to remove a significant amount of dead code, and improve performance up to 3-5X. A number of bug related to reading BigTIFF files have been addressed. Additionally, Tiff tags are now place into the Meta-DataDictionary to allow access to additional TIFF fields such as OME-XML tags ( Originally contributed by Richard Beare http://www.insight-journal.org/browse/publication/728 ).

Significant performance improvements were made to both scalar images and palette images. Handling of certain orientations are now "better" handled, while not changing how the default TOPLEFT orientation is loaded or written. There were a number of cases which appear unreachable and illogical such as support from Zeiss two-componet types, and support for some type of SGI tiled tiff as a 3D image. These were removed.

If you are an active user of TIFF images with ITK, please checkout the latest ITK master and test the refactored TIFFImageIO.

Thanks,
Brad

p.s. Here is the git log for the recent changes.

$git log --no-merges 6791b4a3613aade07e3f11f765f1316faf03b113..

commit 46f4125df47aeff9c42c9f2b3699c06e67ffaca9
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Fri Oct 31 14:25:13 2014 -0400

    ENH: adding TIFFImageIO test for RGB palette images

    Change-Id: I320acc8beac8225616451d11b291b1d7e12a17d0

commit 9adc54c7f53b4da84a41f500d561ae494cb85158
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Fri Oct 24 13:42:02 2014 -0400

    BUG: Adding missing parentheses around boolean expression

    Change-Id: I669d160ff3ba830f3c33ac87de434db41472eb79

commit 40afd95161727be3274f8094ce19fe45e2c3aa50
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Wed Oct 22 15:05:39 2014 -0400

    PERF: Refactor color table lookup

    Change to use a index lookup for the color and gray scale
    palettes. The remove excessive exception checking done on a per pixel
    basis with the GetColor method. Additionally the InitializeColor
    method was change do initialize the color palette variables.

    An exception no longer occurs when an index exceeds the number of
    entries in a palette. Instead a module operator is used to prevent
    out of bounds memory access.

    Change-Id: Ia62c65cd8b1be0d66c488540709a8fb961190cc9

commit d98cbeb7206f6007f7d0c25fd4a4becad8973053
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Fri Oct 3 09:01:28 2014 -0400

    ENH: Refactor per pixel conversion function to per scan-line method

    This patch changes the implementation of conversion from the TIFF
    scanline buffer from a function call on each pixel to a function call
    per scanline. The enable efficient copying of similar scanline and
    better reuse of variables for palette look ups.

    This can increase the performance of reading grayscale images by up to
    3X for cached files.

    Change-Id: I24bb744fda35629f1c1ed7154e26e5708e912059

commit 8e488f40e8cb7defa046700798b77e3944ac33c9
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Tue Sep 30 08:56:58 2014 -0400

    ENH: RGBA read images should stay unchanged.

    The image that are read with the TIFFReadRGBAImage should not have
    their pixel types converted. This libtiff method always read images
    into an ABGR uint8 format. When the tiff file is read by this method
    it should report that uint8 and RGBA is the component an pixel type
    for the image. This allows the standard ITK buffer conversion to be
    performed.

    This change remove extra cases and template instantiations.

    Additionally the RGBAImageToBuffer method was flipping the image along
    the y-axis, this has been more explicitly done with the call to
    TIFFReadRGBAImageOriented.

    Change-Id: I022b12921d406edb09730f905213b0db811f1bd4

commit e5834d5b00e4e3fab3255d88e7d8fd917ee722e9
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Fri Sep 26 14:04:09 2014 -0400

    ENH: Remove support for TIFF tile as 3D and dead code

    This TIFF file format is a file which contains a list of directories,
    if there are no directories in the file it's not a valid tiff. Code
    which supported 3D tiles was conditioned on a file with no directories
    and no data. Additional code was removed related to a private SGI tag
    TIFFTAG_TILEDEPTH, ImageJ specific tag also pre-conditioned on this
    invalid case.

    Change-Id: Id4ef171c5dbe0c797be206e0059c7488719ef6e6

commit 3ac19a9ffd94e02254900dd0e5e2661c329e3183
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Mon Oct 6 10:01:50 2014 -0400

    BUG: Address Coverity warning about null pointer dereferences

    The patch addressed the following Coverity warnings:

    ** CID 1243373:  Explicit null dereferenced  (FORWARD_NULL)
    itkTIFFImageIO.cxx:
    1192 in itk::TIFFImageIO::ReadTIFFTags()()

    ** CID 1243372:  Dereference null return value  (NULL_RETURNS)
    itkTIFFImageIO.cxx:
    1096 in itk::TIFFImageIO::ReadTIFFTags()()

    ** CID 1243371:  Dereference before null check  (REVERSE_INULL)
    itkTIFFImageIO.cxx:
    1100 in itk::TIFFImageIO::ReadTIFFTags()()
    Last login: Fri Oct  3 19:47:29 on ttys004

    Change-Id: I02c8435628c860ff03889b9b57e9dd1543bf82da

commit 52ce992f3a57ef9f187f9c4b4ebd7ce20db365bd
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Fri Sep 26 11:04:38 2014 -0400

    ENH: Reduce code duplicate in TIFFImageIO::ReadCurrentPage

    Refactoring of the pixelOffset to include the number of components
    remove the need for a separate block for RGB multi-component files.

commit 97b33787ec17b76a61fdb43b5f1e1ea708bbac2c
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Wed Sep 24 15:00:54 2014 -0400

    ENH: add arbitrary TIFF TAGs to meta-data dictionary

    A more general approach to adding custom tiff tags to the meta-data
    dictionary has been used based on the implementation of libtiff's
    PrintDirectory method and the libtiff "Defining New TIFF Tags"
    documentation.

    Additionally a new test image was added generated in Photoshop, which
    has a novel pixel layout, and also contains additional tag types such
    as byte, and size specified ascii.

    Change-Id: Idb2e669e540cfbe27a932444011f4b9896ca403d

commit 795c71abb8b5c3a09d461ec7b34eefbfdcb460e9
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Mon Sep 22 15:08:36 2014 -0400

    BUG: Use array delete operator for array new allocations

    Change-Id: Ic9bc3c3792083a1a1b3fd842486a7436a4086150

commit 83e8b6e699b3d9e2da8533dd1ea36dc0692bc9eb
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Tue Sep 23 11:14:15 2014 -0400

    STYLE: Fix minor kwstyle defects in test and test results

    Change-Id: I4b2641d31acb5c764de4041bb7d21c08b3c99846

commit 869f3275120646829b27d5cd75c23800d9df01cd
Author: Richard Beare <Richard.Beare at ieee.org>
Date:   Tue Sep 23 11:11:34 2014 -0400

    ENH: Include TIFF tags in the MetaDataDictionary

    Add contribution from the Insight Journal:
    http://hdl.handle.net/10380/3170

    Change-Id: I20b02394d92c421bad75c5d0af496a6e80089ac0

commit c7a76c9acb319862647a651a2060dd3fb16560d7
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Mon Sep 22 14:46:44 2014 -0400

    BUG: Remove dead separated plannar code, add test

    The PLANARCONFIG_SEPARATE block in TIFFImageIO::ReadGenericImage, was
    preceded by a unreachable due to a conditional and exception
    check. Support for this configuration remains in the TIFFReadRGBAImage
    method. A test for the configuration has been added.

    A new input image is added from the from the libTiff test images:

    PlanarConfiguration = 2 (separated samples)
    -------------------------------------------
    oxford.tif      601x81 8-bit RGB (lzw) screendump off oxford

    Change-Id: Ibb874f80809b70b1aee7a2c9ff1fc8544ed56c04

commit 1b57d75964496dd2cc9e050a4a4425f1163d4b22
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Mon Sep 22 13:32:23 2014 -0400

    BUG: Fix right oriented tiff images

    The TIFFImageIO::ReadGenericImage method correctly handled top-left
    oriented images, and bottom-right image, however images oriented with
    right were non-correctly handled. These cases are now handled
    correctly by the TIFFRGBAImage generic path.

    New test images of the cthead1 image were added which have been
    flipped the to the different orientation, and the matching orientation
    tag specified so that they should all be read as equivalent
    images. These have been added as a test case.

    Change-Id: I058382d6d2e18341ab87edd165490be2187c1d15

commit 6263f4389026f1bd770054e86c428b56629d705b
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Thu Sep 18 16:20:41 2014 -0400

    ENH: Refactor duplicated code to read a page

    Created a method to read a single page or sub-image in a tiff
    file. Simular code was both in the ReadVolume method for 3-d and the
    Read method for 2-d. The code block from ReadVolume was used because
    it contained more typed cases. There for this patch does improve the
    2d reading capabilities for tiffio, so that they are consistent with
    the 3d.

    Change-Id: I0fbfb19f04435a703bb3fa09ea4e79179592e443

commit 233571f1e626917bce872e450907bc8e8521e371
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Thu Sep 18 14:59:58 2014 -0400

    ENH: Refactor method to convert RGBA image to output buffer

    BUG: Address overflow issue when computing page offset with
    TIFFReadRGBAImage method.

    Extract repeated code to convert from ReadRGBAImage to output image
    type.

    Change-Id: Ia90b5e104f22c77493b8a4cd5bf640fd793b8b4b

commit 1cd59825fa65f1123726a32f67c1a443015b40ec
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Thu Sep 18 11:30:41 2014 -0400

    BUG: Remove Zeiss 2-channel support code in TIFFImageIO ( and LSMImageIO )

    There are no tests for this branch of code. It only works in 3D under
    certain cases. There are apparent bugs in the code such as not setting
    all of the input image, and logically dead branches. Some downloaded
    test images for similar described format do not load. This may effect
    files with the extensions tif, tiff, or lsm.

    Removal of this code will enable expansion of the current code to more
    generically support multi-sample per pixel images.

    For more robust reading of this type of image the SCIFIO remote
    module should be used.

    Change-Id: Idb98d69538db3bc8520881ea4a5e22476238c41e

commit 690cfdc89343fcc8c7d3cc266d652cfb14de9d41
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Tue Sep 16 15:31:52 2014 -0400

    ENH: Refactor GenericReadImage into template function

    Reduce code duplication by using template method.

    Change-Id: I79411ad6595258d54cbfa5a4152cc55994acd8e0

commit 5f73f24bfb668fa9f33804218c0db05cf5cadb3f
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Tue Sep 16 12:09:59 2014 -0400

    ENH: Refactor ReadTwoSamplePerPixelImage into template function

    Reduce code duplication by using template method.

    Change-Id: I45f417486e60a4d3e67ff31fa967dac069b3ce3e

commit 0b573136d636613217df9683ea846db8039f9312
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Tue Sep 16 11:12:45 2014 -0400

    ENH: Extract TiffReaderInternal to separate file

    Refactored private class to be in separate private header.

    Change-Id: Icb4008c1605a3a0b62c563350cf2e3bbe948ec6d

commit 6791b4a3613aade07e3f11f765f1316faf03b113
Author: Bradley Lowekamp <blowekamp at mail.nih.gov>
Date:   Mon Sep 15 16:46:26 2014 -0400

    BUG: Fix overflows computing size of read tiff image

    When multiplying the height and width of an tiff image (32-bits),
    at least one element should be converted to size_t to help prevent
    numeric overflow.

    This change enables reading of larger RGB and large 2D image, that
    were previously truncated when reading. This does not address
    potential buffer overflow or resource limits which could occur.

    Change-Id: I192370606677c8dc1a903c67c2a0701026256d5a

commit b6fabfeb2a5b8d339b2d459edd3e3c50232b55ff
Author: Matt McCormick <matt.mccormick at kitware.com>
Date:   Mon Aug 25 00:35:48 2014 -0400

    COMP: Do not use _stat64 with MinGW-32.

    Unlike MinGW-w64, MinGW-32 aka MinGW, does not define as much of the API
    including _stat64.  Do not use use _stat64 in TIFFImageIO when building with
    MinGW-32.

    Change-Id: I8f13a4500cdadf4f4adae7b5cf8893fe8b8cad55

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-users/attachments/20141114/5772acb5/attachment.html>