Proposals:Updating Nrrd library 2009

From KitwarePublic
Jump to navigationJump to search

Nrrd Library

This page describes the procedure for synchronizing the NrrdIO parts of Teem, with the one in Insight/Utilities/NrrdIO

Note that patches may have to be moved in both directions.

Background

  • Nrrd started as one of the libraries in Teem <http://teem.sf.net>
  • NrrdIO is a subset of Teem containing only the IO functionalities (basically, parts of the air, biff, and nrrd libraries).
  • The only part of Teem that's in ITK is in NrrdIO. Also, while Teem has a weakened LGPL license, NrrdIO is licensed identically to ITK.
  • Creating NrrdIO from Teem is done by a GNUMakefile (NrrdIO/pre-GNUmakefile) and a perl script (NrrdIO/unteem.pl)
  • The back-propagation that should happen is from ITK's NrrdIO back to Teem, because the idea is that NrrdIO should be periodically regenerated from Teem, and ITK's NrrdIO should be an up-to-date copy of the NrrdIO automatically regenerated from Teem
  • You should anticipate that the process for updating and synchronizing the libraries will take several days.

Update Life Cycle

The "SF_" prefix refers to the SVN servers at SourceForge

Generating a Fresh NrrdIO

This is a graph with borders and nodes. Maybe there is an Imagemap used so the nodes may be linking to some Pages.

Moving changes from ITK to Teem

This is a graph with borders and nodes. Maybe there is an Imagemap used so the nodes may be linking to some Pages.

This is a cycle to be run until all worth-while modifications from ITK-NrrdIO have been moved into Teem

(LUIS to fix: the arrow between ITK_NrrdIO and Generated NrrdIO should look different: it represents comparison (not moving files))

Move Updates in Teem into into ITK's NrrdIO

This is a graph with borders and nodes. Maybe there is an Imagemap used so the nodes may be linking to some Pages.

Simply copy the files from the most recently generated NrrdIO directory into the ITK_NrrdIO directory, and make Experimental builds of ITK in multiple platforms.

Step by Step Process

Recipe for generating a fresh NrrdIO

General Overview)

  • Checkout NrrdIO from SourceForge (this will only be used as a source of the files listed in YYY)
  • Checkout Teem from SourceForge
  • Create a new empty directory( that will contain the new generated NrrdIO)
  • Copy files (YYY) from the NrrdIO SourceForge checkout into the newly created directory
  • Run the 9 step script process (detailed in XXX)

Checking out Teem and NrrdIO from sourceforge

You can check-out Teem by using the SVN command: 

  svn co http://teem.svn.sourceforge.net/svnroot/teem/teem/trunk teem

Use the following SVN command to checkout NrrdIO: 

  svn co http://teem.svn.sourceforge.net/svnroot/teem/NrrdIO/trunk NrrdIO

This is Sourceforge's NrrdIO, which has fallen out of sync with ITK's NrrdIO. After the fixes that have accumulated in ITK's NrrdIO have been back-propagated to Sourceforge's Teem, Gordon (or others) can do whatever is needed to get them back into NrrdIO.

Files that must be taken from SourceForge NrrdIO (and not from Teem)

The official source of these files is the NrrdIO project in SourceForge.

First, get the scripts

  • pre-GNUMakefile
  • unteem.pl

add to them the list of files can be seen in the pre-GNUMakefile under the variables "NEED_NOT" and "TEEM_HDRS"

  • NEED_NOT
    • encodingBzip2.c
    • formatEPS.c
    • formatPNG.c
    • formatPNM.c
    • formatText.c
    • formatVTK.c


  • TEEM_HDRS
    • teem32bit.h
    • teemDio.h
    • teemEndian.h
    • teemPng.h
    • teemQnanhibit.h


and add the Licensing Header

  • preamble.c

and the following files needed for regenerating the mangling of the library

  • sample-GNUmakefile
  • sampleIO.c
  • mangle.pl

Code Generation

These commands are to be executed in the freshly generated NrrdIO directory.

  1. cd to the Freshly generated NrrdIO directory
  2. set the TEEM_SRC_ROOT environment variable to the directory where Teem wash checked out.
    • This is the directory that contains the "src" and "include" directories, as well as files like "CMakeLists.txt" and "README.txt"
    • no trailing "/" required
  3. rm -f itk_NrrdIO_mangle.h
  4. touch itk_NrrdIO_mangle.h
  5. make -f pre-GNUmakefile clean
  6. setenv ITK_NRRDIO
  7. make -f pre-GNUmakefile
  8. unsetenv ITK_NRRDIO
  9. make -f sample-GNUmakefile
  10. perl mangle.pl itk >! itk_NrrdIO_mangle.h
  11. make -f sample-GNUmakefile clean

Update Procedure

Perform the following two tasks:

From ITK to Teem

What code base to use

In this step we will compare the files in 

       Insight/Utilities/NrrdIO

with the file in the Freshly generated NrrdIO directory.

What to do

For all the differences between files in 

             Insight/Utilities/NrrdIO

and files in 

             Freshly generated NrrdIO directory

Do the following:

  1. Understand what the change was
  2. Identify the file in Teem that is the source of the file in question in the freshly generated NrrdIO directory.
  3. Apply the same change (or something equivalent) to the corresponding file in Teem (not to the NrrdIO SVN checkout)
  4. Verify the Teem Dashboard, to make sure that it is green.
  5. Re-run the process for generating a fresh NrrdIO from Teem

From Nrrd to ITK

  1. Make a working copy of NrrdIO by running the pre-GNUmakefile to build a NrrdIO from Teem
  2. Gradually or all at once, depending on our bravery, we copy files from this NrrdIO to Insight/Utilities/NrrdIO
  3. Run an Experimental ITK build and see what breaks.
    1. Problems are likely related to updates to Teem itself (including the components of NrrdIO) since 2005
    2. but might also be related to errors in how the updates to ITK's NrrdIO were back-propagated to Teem.

Progress

Table

The table below lists all the files in NrrdIO which are generated from a Teem checkout via NrrdIO/pre-GNUmakefile; these are marked with "*", and the originating directory in Teem is also indicated (the air, biff, and nrrd directories inside teem/src). Changes in these files in ITK's NrrdIO since GLK's last commits in 2005 should be back-propagated to the corresponding files in Teem, so that when NrrdIO is next regenerated from Teem, none of the improvements to ITK's NrrdIO are lost. Handling updates to these files is the hard/interesting part of this project.

The files not marked with a "*" are not generated from Teem via NrrdIO/pre-GNUmakefile; they exist only within NrrdIO. GLK doesn't see why there would have been any changes to these in files in ITK's NrrdIO, because it wouldn't effect how ITK uses Nrrd, but if there are changes they should also be reviewed and possibly back-propagated to Sourceforge's NrrdIO.

File Check ITK for Bug Fix Move from NrrdIO to ITK
000-README.txt Not present in current Sourceforge Teem. Keep?
* air/754.c Change of ITK rev 1.8 present in NrrrdIO (l. 132); NrrdIO version updated (l. 358: mant0 >> 19). Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/accessors.c ITK version 4 years old; NrrdIO version updated (static functions). Keep NrrdIO version. Moved to ITK. 1/27/10
* air/array.c ITK version 4 years old; NrrdIO version updated (AIR_CALLOC, check on a->data, ...). Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/arraysNrrd.c ITK version 4 years old; NrrdIO version updated (AIR_CALLOC, check on a->data, ...). Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/axis.c Casts from size_t to unsigned int, as in rev 1.13, no longer needed: everything is size_t now. Keep NrrdIO version. Moved to ITK. 1/27/10
* biff/biffbiff.c Casts from size_t to int, as in revs 1.8, 1.9, 1.10, no longer needed. Code, e.g biffAddErr() , superceded. Keep NrrdIO version. Moved to ITK. Committed biff.h as well. 1/27/10
* biff/biffmsg.c Does not exist in ITK version. Keep NrrdIO version. Added and committed on 1/27/10.
CMakeLists.txt Updated to account for new source files and reconcile flags. 1/27/10
* nrrd/comment.c ITK version 4 years old; not much difference. Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/defaultsNrrd.c ITK version 4 years old; not much difference. Keep NrrdIO version. Moved to ITK. 1/27/10
* air/dio.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/encodingAscii.c Casts from size_t to unsigned int, as in rev 1.6, no longer needed. Code, e.g sprintf()/biffAdd(), superceded. Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/encodingBzip2.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/encoding.c ITK version 4 years old; NrrdIO version updated(static consts, biffAddf). Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/encodingGzip.c Casts from size_t to unsigned int, as in rev 1.8, no longer needed. Code, e.g. _nrrdGzRead(),superceded. Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/encodingHex.c Casts as in rev 1.8 in ITK now handled by AIR_CAST() in NrrdIO, + other NrrdIO version updates. Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/encodingRaw.c ITK version 4 years old; NrrdIO version updated (temporary bug in gcc 4.2). Keep NrrdIO version (for now). Moved to ITK. 1/28/10
* air/endianAir.c ITK version 4 years old; NrrdIO version updated (const correctness). Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/endianNrrd.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 1/28/10
* air/enum.c ITK version 4 years old; NrrdIO version updated (const correctness). Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/enumsNrrd.c ITK version 4 years old; NrrdIO version updated (const correctness). Keep NrrdIO version. Non-NrrdIO taken out. Moved to ITK. 1/27/10
* nrrd/format.c ITK version 4 years old; NrrdIO version updated (const correctness, biffAddf). Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/formatEPS.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/formatNRRD.c ITK version 4 years old; NrrdIO version updated (const correctness, biffAddf). Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/formatPNG.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
* nrrd/formatPNM.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
* nrrd/formatText.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
* nrrd/formatVTK.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/19/10 along with randMT.c.
* nrrd/gzio.c Changes in ITK also found in NrrdIO, + latter version updated (const correctness, biffAddf). Keep NrrdIO version. Moved to ITK. 1/28/10
* nrrd/keyvalue.c ITK version 4 years old; NrrdIO version updated (parameters added _nrrdWriteEscaped(), _nrrdKeyValueWrite()). Keep NrrdIO version. Moved to ITK. 1/28/10
mangle.pl Not present in current Sourceforge Teem. Keep?
* nrrd/methodsNrrd.c Most changes in ITK last 4 years superceded by NrrdIO version (e.g. biffAddf() ). Rollback in rev 1.10 mysterious. Keep NrrdIO version (for now). Moved to ITK. 1/27/10
* air/miscAir.c ITK version 4 years old; minor changes (float to double, ...) in NrrdIO version. Keep NrrdIO version. Non-NrrdIO taken out. Moved to ITK. 1/27/10
* air/mop.c ITK version 4 years old; minor change (airMopAdd now returns int) in NrrdIO version. Keep NrrdIO version. Moved to ITK. 1/27/10
* NrrdIO.h Latest ITK changes, plus recent changes (#ifdef __APPLE__ l 696) seem to appear in NrrdIO version. Keep NrrdIO version. Use NrrdIO version, and modify it to account for #include "itk_NrrdIO_mangle.h" and TEEM_API as in ITK. Updated in ITK. 1/28/10
NrrdIO_Srcs.txt Not found in NrrdIO repository. Not applicable.
* air/parseAir.c ITK version 4 years old; recent changes (const correctness, AIR_CAST...) in NrrdIO version. Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/parseNrrd.c ITK casts to unsigned int, rev 1.11, superceded in NrrdIO by dealing with size_t + other changes. Rollback in rev 1.10 mysterious. Keep NrrdIO version (for now). Moved to ITK. 1/27/10
preamble.c ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
pre-GNUmakefile ITK version 4 years old; minor changes (addprefix l 83, ...) in NrrdIO version. Keep NrrdIO version, but ... ... ITK currently uses cvs not svn.
* air/privateAir.h ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/privateNrrd.h Rollback in rev 1.12 mysterious. Keep NrrdIO version (for now). ITK version should use #include "itk_zlib.h", as currently in ITK. Moved to ITK. 1/28/10
qnanhibit.c Not present in current Sourceforge Teem. Keep?
* nrrd/read.c Rollback in rev 1.9 mysterious. Keep NrrdIO version (for now). Add logic for separators in 1.11. Moved to ITK. 1/27/10
* nrrd/reorder.c Cast in rev. 1.10 missing (via AIR_CAST), look at pragma (array bound warning) in rev 1.11. Moved to ITK. 1/27/10
sample-GNUmakefile Not present in current Sourceforge Teem. Keep?
sampleIO.c Not present in current Sourceforge Teem. Keep?
* air/sane.c ITK version 4 years old; recent changes in NrrdIO version (airInsane_AIR_NAN). Keep NrrdIO version. Causes failing tests in cross-compiled Mac.
* nrrd/simple.c ITK casts to int, rev 1.13, superceded in NrrdIO by dealing with size_t + other changes (const). Keep NrrdIO version. Moved to ITK. 1/27/10
* air/string.c ITK casts either superceded in NrrdIO by dealing with size_t or handled by AIR_CAST. Keep NrrdIO version. Moved to ITK. 1/27/10
* nrrd/subset.c Rollback in rev 1.7 mysterious. Recent changes in NrrdIO version. Start with NrrdIO version (for now), but look at pragma (array bound warning) in rev 1.8. Moved to ITK. 1/28/10
* include/teem32bit.h ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
* include/teemDio.h ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
* include/teemEndian.h ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
* include/teemPng.h ITK version 4 years old; NrrdIO has "vtk_png_mangle.h". Keep NrrdIO version. Moved to ITK. 2/08/10
* include/teemQnanhibit.h ITK version 4 years old; no difference except copyright. Keep NrrdIO version. Moved to ITK. 2/08/10
unteem.pl Not present in current Sourceforge Teem. Keep?
* nrrd/write.c Reconciliation of NrrdIO & ITK versions req'd. !!airStrlen vs (int)airStrlen; rollback in rev 1.10 mysterious; airSinglePrintf(NULL, buff, "%.17lg", val[dd])-type fixes as in ITK rev 1.12. Moved to ITK. 1/27/10


Reference Material

NrrdIO / Teem Dashboard

For the Curious Minds

What pre-GNUmakefile does

The "pre-GNUmakefile" makefile

  • Copies files from a Teem checkout
  • Modifies them slightly (through the action of unteem.pl). Specifically, it excises lines of code that are delimited by "/* ---- BEGIN non-NrrdIO */" and "/* ---- END non-NrrdIO */"; this is code that Teem needs but NrrdIO does not.

What sample-GNUmakefile does

Then "sample-GNUmakefile"

  • Builds a NrrdIO library
  • Builds a small test executable

What mangle.pl does

The script "mangle.pl"

  • Looks at all the symbols in the library in order to generate the name-mangling header file required for ITK.

Extra Information (Not part of the process)

If we weren't making NrrdIO for ITK, the name mangling wouldn't be required, and the process would just be:

  • make -f pre-GNUmakefile clean
  • make -f pre-GNUmakefile
  • make -f sample-GNUmakefile
  • make -f sample-GNUmakefile clean

where we run make with "sample-GNUmakefile" just to make sure that everything compiles ok.

Retrieved from "https://public.kitware.com/Wiki/index.php?title=Proposals:Updating_Nrrd_library_2009&oldid=20771"