Proposals:Updating Nrrd library 2009: Difference between revisions
(37 intermediate revisions by 3 users not shown) | |||
Line 13: | Line 13: | ||
* The back-propagation that should happen is from ITK's NrrdIO back to Teem, because the idea is that NrrdIO should be periodically regenerated from Teem, and ITK's NrrdIO should be an up-to-date copy of the NrrdIO automatically regenerated from Teem | * The back-propagation that should happen is from ITK's NrrdIO back to Teem, because the idea is that NrrdIO should be periodically regenerated from Teem, and ITK's NrrdIO should be an up-to-date copy of the NrrdIO automatically regenerated from Teem | ||
* You should anticipate that the process for updating and synchronizing the libraries will take several days. | * You should anticipate that the process for updating and synchronizing the libraries will take several days. | ||
* Write access to the Teem project in SourceForge is administered by Gordon Kindlmann (ask the ITK Developers list for contact information). | |||
* The overall process should be coordinated in the ITK Developers list, to make sure that it doesn't interfere with the release schedule | |||
= Update Life Cycle = | |||
The "SF_" prefix refers to the SVN servers at SourceForge | The "SF_" prefix refers to the SVN servers at SourceForge | ||
== Generating a Fresh NrrdIO == | |||
<graphviz> | <graphviz> | ||
Line 23: | Line 27: | ||
Generated_NrrdIO; | Generated_NrrdIO; | ||
SF_NrrdIO [URL="http://teem.svn.sourceforge.net/svnroot/teem/NrrdIO/trunk"]; | SF_NrrdIO [URL="http://teem.svn.sourceforge.net/svnroot/teem/NrrdIO/trunk"]; | ||
SF_Teem -> Generated_NrrdIO; | |||
SF_NrrdIO -> Generated_NrrdIO; | |||
} | |||
</graphviz> | |||
== Synchronizing changes between ITK and Teem == | |||
<graphviz> | |||
digraph G { | |||
SF_Teem [URL="http://teem.svn.sourceforge.net/svnroot/teem/teem/trunk"]; | |||
Generated_NrrdIO; | |||
ITK_NrrdIO[URL="http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/NrrdIO/?root=Insight"]; | ITK_NrrdIO[URL="http://www.itk.org/cgi-bin/viewcvs.cgi/Utilities/NrrdIO/?root=Insight"]; | ||
ITK_NrrdIO -> SF_Teem; | ITK_NrrdIO -> SF_Teem; | ||
SF_Teem -> Generated_NrrdIO; | SF_Teem -> Generated_NrrdIO; | ||
Generated_NrrdIO -> ITK_NrrdIO; | Generated_NrrdIO -> ITK_NrrdIO; | ||
ITK_NrrdIO -> Generated_NrrdIO; | |||
} | } | ||
</graphviz> | </graphviz> | ||
This is a cycle to be run until all worth-while modifications from ITK-NrrdIO have been moved into Teem | |||
When moving changes into ITK, they can simply be copied from the generated NrrdIO into ITK. | |||
When moving changes from ITK, they must be committed as patches to Teem SVN, and then the Generated NrrdIO should be regenerated. | |||
= Step by Step Process = | |||
== Recipe for generating a fresh NrrdIO == | |||
=== General Overview === | |||
* Checkout NrrdIO from SourceForge (this will only be used as a source of the files listed in YYY) | * Checkout NrrdIO from SourceForge (this will only be used as a source of the files listed in YYY) | ||
Line 38: | Line 62: | ||
* Create a new empty directory( that will contain the new generated NrrdIO) | * Create a new empty directory( that will contain the new generated NrrdIO) | ||
* Copy files (YYY) from the NrrdIO SourceForge checkout into the newly created directory | * Copy files (YYY) from the NrrdIO SourceForge checkout into the newly created directory | ||
* Run the | * Run the 11 steps script process (detailed in XXX) | ||
=== Checking out Teem and NrrdIO from sourceforge === | === Checking out Teem and NrrdIO from sourceforge === | ||
Line 101: | Line 125: | ||
# touch itk_NrrdIO_mangle.h | # touch itk_NrrdIO_mangle.h | ||
# make -f pre-GNUmakefile clean | # make -f pre-GNUmakefile clean | ||
# setenv ITK_NRRDIO | # setenv ITK_NRRDIO (in csh), or export ITK_NRRDIO="" (in bash) | ||
# make -f pre-GNUmakefile | # make -f pre-GNUmakefile | ||
# unsetenv ITK_NRRDIO | # unsetenv ITK_NRRDIO (in csh), or unset ITK_NRRDIO (in bash) | ||
# make -f sample-GNUmakefile | # make -f sample-GNUmakefile | ||
# perl mangle.pl itk > | # perl mangle.pl itk >> itk_NrrdIO_mangle.h | ||
# make -f sample-GNUmakefile clean | # make -f sample-GNUmakefile clean | ||
# files are ready to copy to ITK, except privateNrrd.h, where one must replace #include <zlib.h> by #include "itk_zlib.h" | |||
= | == Synchronizing ITK with Teem == | ||
=== What code base to use === | === What code base to use === | ||
Line 135: | Line 156: | ||
# Understand what the change was | # Understand what the change was | ||
## Was this a change made in ITK CVS, that must be moved to Teem, or | |||
## Was this a change made in Teem, that must be moved to ITK | |||
In order to figure out the case, you will have to look at the SVN logs of Teem, and at the CVS logs of ITK. To identify the source of the difference. | |||
=== If it is a change in ITK to be moved to Teem === | |||
# Identify the file in Teem that is the source of the file in question in the freshly generated NrrdIO directory. | # Identify the file in Teem that is the source of the file in question in the freshly generated NrrdIO directory. | ||
# Apply the same change (or something equivalent) to the corresponding file in Teem (not to the NrrdIO SVN checkout) | # Apply the same change (or something equivalent) to the corresponding file in Teem (not to the NrrdIO SVN checkout) | ||
Line 140: | Line 168: | ||
# Re-run the process for generating a fresh NrrdIO from Teem | # Re-run the process for generating a fresh NrrdIO from Teem | ||
== | === If it is a change in Teem to be moved in ITK === | ||
* Patch the file in ITK | |||
= Progress = | = Progress (for the Update made in 2010) = | ||
== Table == | == Table == | ||
Line 278: | Line 302: | ||
| * nrrd/write.c || Reconciliation of NrrdIO & ITK versions req'd. !!airStrlen vs (int)airStrlen; rollback in rev 1.10 mysterious; airSinglePrintf(NULL, buff, "%.17lg", val[dd])-type fixes as in ITK rev 1.12. || Moved to ITK. 1/27/10 | | * nrrd/write.c || Reconciliation of NrrdIO & ITK versions req'd. !!airStrlen vs (int)airStrlen; rollback in rev 1.10 mysterious; airSinglePrintf(NULL, buff, "%.17lg", val[dd])-type fixes as in ITK rev 1.12. || Moved to ITK. 1/27/10 | ||
|} | |} | ||
= Reference Material = | = Reference Material = | ||
Line 286: | Line 309: | ||
* http://my.cdash.org/index.php?project=Teem | * http://my.cdash.org/index.php?project=Teem | ||
== For the Curious Minds == | |||
==== What pre-GNUmakefile does ==== | ==== What pre-GNUmakefile does ==== |
Latest revision as of 20:29, 1 March 2010
Nrrd Library
This page describes the procedure for synchronizing the NrrdIO parts of Teem, with the one in Insight/Utilities/NrrdIO
Note that patches may have to be moved in both directions.
Background
- Nrrd started as one of the libraries in Teem <http://teem.sf.net>
- NrrdIO is a subset of Teem containing only the IO functionalities (basically, parts of the air, biff, and nrrd libraries).
- The only part of Teem that's in ITK is in NrrdIO. Also, while Teem has a weakened LGPL license, NrrdIO is licensed identically to ITK.
- Creating NrrdIO from Teem is done by a GNUMakefile (NrrdIO/pre-GNUmakefile) and a perl script (NrrdIO/unteem.pl)
- The back-propagation that should happen is from ITK's NrrdIO back to Teem, because the idea is that NrrdIO should be periodically regenerated from Teem, and ITK's NrrdIO should be an up-to-date copy of the NrrdIO automatically regenerated from Teem
- You should anticipate that the process for updating and synchronizing the libraries will take several days.
- Write access to the Teem project in SourceForge is administered by Gordon Kindlmann (ask the ITK Developers list for contact information).
- The overall process should be coordinated in the ITK Developers list, to make sure that it doesn't interfere with the release schedule
Update Life Cycle
The "SF_" prefix refers to the SVN servers at SourceForge
Generating a Fresh NrrdIO
Synchronizing changes between ITK and Teem
This is a cycle to be run until all worth-while modifications from ITK-NrrdIO have been moved into Teem
When moving changes into ITK, they can simply be copied from the generated NrrdIO into ITK.
When moving changes from ITK, they must be committed as patches to Teem SVN, and then the Generated NrrdIO should be regenerated.
Step by Step Process
Recipe for generating a fresh NrrdIO
General Overview
- Checkout NrrdIO from SourceForge (this will only be used as a source of the files listed in YYY)
- Checkout Teem from SourceForge
- Create a new empty directory( that will contain the new generated NrrdIO)
- Copy files (YYY) from the NrrdIO SourceForge checkout into the newly created directory
- Run the 11 steps script process (detailed in XXX)
Checking out Teem and NrrdIO from sourceforge
You can check-out Teem by using the SVN command:
svn co http://teem.svn.sourceforge.net/svnroot/teem/teem/trunk teem
Use the following SVN command to checkout NrrdIO:
svn co http://teem.svn.sourceforge.net/svnroot/teem/NrrdIO/trunk NrrdIO
This is Sourceforge's NrrdIO, which has fallen out of sync with ITK's NrrdIO. After the fixes that have accumulated in ITK's NrrdIO have been back-propagated to Sourceforge's Teem, Gordon (or others) can do whatever is needed to get them back into NrrdIO.
Files that must be taken from SourceForge NrrdIO (and not from Teem)
The official source of these files is the NrrdIO project in SourceForge.
First, get the scripts
- pre-GNUMakefile
- unteem.pl
add to them the list of files can be seen in the pre-GNUMakefile under the variables "NEED_NOT" and "TEEM_HDRS"
- NEED_NOT
- encodingBzip2.c
- formatEPS.c
- formatPNG.c
- formatPNM.c
- formatText.c
- formatVTK.c
- TEEM_HDRS
- teem32bit.h
- teemDio.h
- teemEndian.h
- teemPng.h
- teemQnanhibit.h
and add the Licensing Header
- preamble.c
and the following files needed for regenerating the mangling of the library
- sample-GNUmakefile
- sampleIO.c
- mangle.pl
Code Generation
These commands are to be executed in the freshly generated NrrdIO directory.
- cd to the Freshly generated NrrdIO directory
- set the TEEM_SRC_ROOT environment variable to the directory where Teem wash checked out.
- This is the directory that contains the "src" and "include" directories, as well as files like "CMakeLists.txt" and "README.txt"
- no trailing "/" required
- rm -f itk_NrrdIO_mangle.h
- touch itk_NrrdIO_mangle.h
- make -f pre-GNUmakefile clean
- setenv ITK_NRRDIO (in csh), or export ITK_NRRDIO="" (in bash)
- make -f pre-GNUmakefile
- unsetenv ITK_NRRDIO (in csh), or unset ITK_NRRDIO (in bash)
- make -f sample-GNUmakefile
- perl mangle.pl itk >> itk_NrrdIO_mangle.h
- make -f sample-GNUmakefile clean
- files are ready to copy to ITK, except privateNrrd.h, where one must replace #include <zlib.h> by #include "itk_zlib.h"
Synchronizing ITK with Teem
What code base to use
In this step we will compare the files in
Insight/Utilities/NrrdIO
with the file in the Freshly generated NrrdIO directory.
What to do
For all the differences between files in
Insight/Utilities/NrrdIO
and files in
Freshly generated NrrdIO directory
Do the following:
- Understand what the change was
- Was this a change made in ITK CVS, that must be moved to Teem, or
- Was this a change made in Teem, that must be moved to ITK
In order to figure out the case, you will have to look at the SVN logs of Teem, and at the CVS logs of ITK. To identify the source of the difference.
If it is a change in ITK to be moved to Teem
- Identify the file in Teem that is the source of the file in question in the freshly generated NrrdIO directory.
- Apply the same change (or something equivalent) to the corresponding file in Teem (not to the NrrdIO SVN checkout)
- Verify the Teem Dashboard, to make sure that it is green.
- Re-run the process for generating a fresh NrrdIO from Teem
If it is a change in Teem to be moved in ITK
- Patch the file in ITK
Progress (for the Update made in 2010)
Table
The table below lists all the files in NrrdIO which are generated from a Teem checkout via NrrdIO/pre-GNUmakefile; these are marked with "*", and the originating directory in Teem is also indicated (the air, biff, and nrrd directories inside teem/src). Changes in these files in ITK's NrrdIO since GLK's last commits in 2005 should be back-propagated to the corresponding files in Teem, so that when NrrdIO is next regenerated from Teem, none of the improvements to ITK's NrrdIO are lost. Handling updates to these files is the hard/interesting part of this project.
The files not marked with a "*" are not generated from Teem via NrrdIO/pre-GNUmakefile; they exist only within NrrdIO. GLK doesn't see why there would have been any changes to these in files in ITK's NrrdIO, because it wouldn't effect how ITK uses Nrrd, but if there are changes they should also be reviewed and possibly back-propagated to Sourceforge's NrrdIO.
File | Check ITK for Bug Fix | Move from NrrdIO to ITK |
---|---|---|
000-README.txt | Not present in current Sourceforge Teem. | Keep? |
* air/754.c | Change of ITK rev 1.8 present in NrrrdIO (l. 132); NrrdIO version updated (l. 358: mant0 >> 19). Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/accessors.c | ITK version 4 years old; NrrdIO version updated (static functions). Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* air/array.c | ITK version 4 years old; NrrdIO version updated (AIR_CALLOC, check on a->data, ...). Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/arraysNrrd.c | ITK version 4 years old; NrrdIO version updated (AIR_CALLOC, check on a->data, ...). Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/axis.c | Casts from size_t to unsigned int, as in rev 1.13, no longer needed: everything is size_t now. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* biff/biffbiff.c | Casts from size_t to int, as in revs 1.8, 1.9, 1.10, no longer needed. Code, e.g biffAddErr() , superceded. Keep NrrdIO version. | Moved to ITK. Committed biff.h as well. 1/27/10 |
* biff/biffmsg.c | Does not exist in ITK version. Keep NrrdIO version. | Added and committed on 1/27/10. |
CMakeLists.txt | Updated to account for new source files and reconcile flags. 1/27/10 | |
* nrrd/comment.c | ITK version 4 years old; not much difference. Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/defaultsNrrd.c | ITK version 4 years old; not much difference. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* air/dio.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/encodingAscii.c | Casts from size_t to unsigned int, as in rev 1.6, no longer needed. Code, e.g sprintf()/biffAdd(), superceded. Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/encodingBzip2.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/encoding.c | ITK version 4 years old; NrrdIO version updated(static consts, biffAddf). Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/encodingGzip.c | Casts from size_t to unsigned int, as in rev 1.8, no longer needed. Code, e.g. _nrrdGzRead(),superceded. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/encodingHex.c | Casts as in rev 1.8 in ITK now handled by AIR_CAST() in NrrdIO, + other NrrdIO version updates. Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/encodingRaw.c | ITK version 4 years old; NrrdIO version updated (temporary bug in gcc 4.2). Keep NrrdIO version (for now). | Moved to ITK. 1/28/10 |
* air/endianAir.c | ITK version 4 years old; NrrdIO version updated (const correctness). Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/endianNrrd.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* air/enum.c | ITK version 4 years old; NrrdIO version updated (const correctness). Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/enumsNrrd.c | ITK version 4 years old; NrrdIO version updated (const correctness). Keep NrrdIO version. | Non-NrrdIO taken out. Moved to ITK. 1/27/10 |
* nrrd/format.c | ITK version 4 years old; NrrdIO version updated (const correctness, biffAddf). Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/formatEPS.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/formatNRRD.c | ITK version 4 years old; NrrdIO version updated (const correctness, biffAddf). Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/formatPNG.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
* nrrd/formatPNM.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
* nrrd/formatText.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
* nrrd/formatVTK.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/19/10 along with randMT.c. |
* nrrd/gzio.c | Changes in ITK also found in NrrdIO, + latter version updated (const correctness, biffAddf). Keep NrrdIO version. | Moved to ITK. 1/28/10 |
* nrrd/keyvalue.c | ITK version 4 years old; NrrdIO version updated (parameters added _nrrdWriteEscaped(), _nrrdKeyValueWrite()). Keep NrrdIO version. | Moved to ITK. 1/28/10 |
mangle.pl | Not present in current Sourceforge Teem. | Keep? |
* nrrd/methodsNrrd.c | Most changes in ITK last 4 years superceded by NrrdIO version (e.g. biffAddf() ). Rollback in rev 1.10 mysterious. Keep NrrdIO version (for now). | Moved to ITK. 1/27/10 |
* air/miscAir.c | ITK version 4 years old; minor changes (float to double, ...) in NrrdIO version. Keep NrrdIO version. | Non-NrrdIO taken out. Moved to ITK. 1/27/10 |
* air/mop.c | ITK version 4 years old; minor change (airMopAdd now returns int) in NrrdIO version. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* NrrdIO.h | Latest ITK changes, plus recent changes (#ifdef __APPLE__ l 696) seem to appear in NrrdIO version. Keep NrrdIO version. | Use NrrdIO version, and modify it to account for #include "itk_NrrdIO_mangle.h" and TEEM_API as in ITK. Updated in ITK. 1/28/10 |
NrrdIO_Srcs.txt | Not found in NrrdIO repository. | Not applicable. |
* air/parseAir.c | ITK version 4 years old; recent changes (const correctness, AIR_CAST...) in NrrdIO version. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/parseNrrd.c | ITK casts to unsigned int, rev 1.11, superceded in NrrdIO by dealing with size_t + other changes. Rollback in rev 1.10 mysterious. Keep NrrdIO version (for now). | Moved to ITK. 1/27/10 |
preamble.c | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
pre-GNUmakefile | ITK version 4 years old; minor changes (addprefix l 83, ...) in NrrdIO version. Keep NrrdIO version, but ... | ... ITK currently uses cvs not svn. |
* air/privateAir.h | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/privateNrrd.h | Rollback in rev 1.12 mysterious. Keep NrrdIO version (for now). | ITK version should use #include "itk_zlib.h", as currently in ITK. Moved to ITK. 1/28/10 |
qnanhibit.c | Not present in current Sourceforge Teem. | Keep? |
* nrrd/read.c | Rollback in rev 1.9 mysterious. Keep NrrdIO version (for now). Add logic for separators in 1.11. | Moved to ITK. 1/27/10 |
* nrrd/reorder.c | Cast in rev. 1.10 missing (via AIR_CAST), look at pragma (array bound warning) in rev 1.11. | Moved to ITK. 1/27/10 |
sample-GNUmakefile | Not present in current Sourceforge Teem. | Keep? |
sampleIO.c | Not present in current Sourceforge Teem. | Keep? |
* air/sane.c | ITK version 4 years old; recent changes in NrrdIO version (airInsane_AIR_NAN). Keep NrrdIO version. | Causes failing tests in cross-compiled Mac. |
* nrrd/simple.c | ITK casts to int, rev 1.13, superceded in NrrdIO by dealing with size_t + other changes (const). Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* air/string.c | ITK casts either superceded in NrrdIO by dealing with size_t or handled by AIR_CAST. Keep NrrdIO version. | Moved to ITK. 1/27/10 |
* nrrd/subset.c | Rollback in rev 1.7 mysterious. Recent changes in NrrdIO version. Start with NrrdIO version (for now), but look at pragma (array bound warning) in rev 1.8. | Moved to ITK. 1/28/10 |
* include/teem32bit.h | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
* include/teemDio.h | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
* include/teemEndian.h | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
* include/teemPng.h | ITK version 4 years old; NrrdIO has "vtk_png_mangle.h". Keep NrrdIO version. | Moved to ITK. 2/08/10 |
* include/teemQnanhibit.h | ITK version 4 years old; no difference except copyright. Keep NrrdIO version. | Moved to ITK. 2/08/10 |
unteem.pl | Not present in current Sourceforge Teem. | Keep? |
* nrrd/write.c | Reconciliation of NrrdIO & ITK versions req'd. !!airStrlen vs (int)airStrlen; rollback in rev 1.10 mysterious; airSinglePrintf(NULL, buff, "%.17lg", val[dd])-type fixes as in ITK rev 1.12. | Moved to ITK. 1/27/10 |
Reference Material
NrrdIO / Teem Dashboard
For the Curious Minds
What pre-GNUmakefile does
The "pre-GNUmakefile" makefile
- Copies files from a Teem checkout
- Modifies them slightly (through the action of unteem.pl). Specifically, it excises lines of code that are delimited by "/* ---- BEGIN non-NrrdIO */" and "/* ---- END non-NrrdIO */"; this is code that Teem needs but NrrdIO does not.
What sample-GNUmakefile does
Then "sample-GNUmakefile"
- Builds a NrrdIO library
- Builds a small test executable
What mangle.pl does
The script "mangle.pl"
- Looks at all the symbols in the library in order to generate the name-mangling header file required for ITK.
Extra Information (Not part of the process)
If we weren't making NrrdIO for ITK, the name mangling wouldn't be required, and the process would just be:
- make -f pre-GNUmakefile clean
- make -f pre-GNUmakefile
- make -f sample-GNUmakefile
- make -f sample-GNUmakefile clean
where we run make with "sample-GNUmakefile" just to make sure that everything compiles ok.