View source for Proposals:Compiling on Bluegene Supercomputer

This page describes the changes that are needed in ITK in order to make possible to build it in the [http://en.wikipedia.org/wiki/Bluegene Bluegene/L] supercomputer.

The tests have been developed at the Bluegene/L machine available at the [http://en.wikipedia.org/wiki/Rensselaer_Polytechnic_Institute Rensselaer Polytechnic Institute][http://www.rpi.edu RPI]. This computer, located at the  [http://en.wikipedia.org/wiki/Computational_Center_for_Nanotechnology_Innovations RPI Computational Center for Nanotechnology Innovations] is the seventh worlds largest supercomputer, according to the
[http://www.top500.org/list/2007/06/100 Top500 list released on June 2007].

== Developers ==

* Arunachalam Narayanaswamy
* Pat Marion
* Luis Ibanez 

==List of common changes==

*itkmkg3states(mkg3states in itk 3.4) has to be copied from the native build's "bin" directory and replaced in "bin" directory of  cross-compilation build directory. This further generates files like tif_fax3sm.c during compilation.

==List of Changes using the mpixlC compiler==

* Remove optimization (use -O0) from the following files
** Insight/Utilities/vxl/v3p/netlib/linpack
*** csvdc.c
*** zsvdc.c
** Done by adding a SET_SOURCE_FILES_PROPERTIES() command in the CMakeLists.txt file of "netlib".

*Change Insight/Code/Common/itkMersenneTwisterRandomVariateGenerator.h
** change os<<indent<< "State vector:" << state <<std::endl; to os<<indent<< "State vector:" << &state[0] <<std::endl;

*Change Insight/Code/IO/itkVTKImageIO.cxx
** add defined(__IBMCPP__) to lines 71 and 112 to create OpenMode to be of type std::ios_base::openmode
** It is currently created with const int type which throws up error in xlC compiler

*Change Insight/Code/Common/itkNeighborhood.h
** Change lines defining SizeType & RadiusType (lines 76,80 in mine) as "typedef itk::Size<VDimension> SizeType"(similarly for RadiusType). This is because Size is also a function inside the "Neighborhood" class. 
** It might be better to avoid such overlapping names for classes and functions in the same namespace.

*Change InsightCode/Common/itkFiniteDifferenceFunction.txx
** Change m_ScaleCoefficients to &m_ScaleCoefficients[0]

*Change Insight/Code/Common/itkGaussianDerivativeImageFunction.txx
** Change << m_Sigma to << &m_Sigma[0]
** Change << m_Extent to << &m_Extent[0]

*Change Insight/Code/Common/itkDiffisionTensor3D.h
** add || defined(__IBMCPP__) to line 89 to reach "if" and not "else" part of the directive

* Insight/Testing/Code/Common/itkSTLThreadTest.cxx
** Change line 144 to "return ;" instead of "return EXIT_SUCCESS;"
** Note: This error may not reflect in all compilations. It might be due to my CMake configuration of ITK being bad.

* Change Insight/Code/Common/itkCannyEdgeDetectionImageFilter.txx
** Change line 715 from "<< m_Stride" to "<< &m_Stide[0]"

* Removed itkParallelLevelSetImageFilter from the BasicFiltersTests1
** comment the ADD_TEST() line in CMakeLists.txt and remove the .cxx file from the SRCs variable
** Was giving weird errors in the itkParallelLevelSetImageFilter.txx file that "template argument 1 does not match .... in NumericTraits<>::One". I didnt find anything wrong with NumericTraits<>, since it had explicit instantiation for "float" ( which is used in the test.

* Removed itkSparseFieldFourthFourthOrderLevelSetImageFilterTest similar to the above one



You can find the TryRunResults.cmake file [[Media:TryRunResults_xlc.cmake|here]]

Too many changes to be made along the way. The maximum I got to was 60%. I've deferred this job until I can solve the current compile error.  

==Notes for compiling using mpicc==

This is a MPI wrapper written over the gnu compiler for PowerPC. 

There are two ways to do this

*(1)Compile using the MPI wrapper

The problem with this is that the MPI wrapper itself is not small, and there is no need to set the optimization flags (which makes compilation slow usually) used by the wrapper. More often than not it will interfere adversely with your CMake setup. Nevertheless, this is one option.

*(2)Compile using the gnu compiler used by the MPI wrapper

The problem with this is that CMake will force the C/C++ compiler for all projects that use ITK later, to be the one used for compiling ITK. This means, you cannot just use the MPI Wrapper for all future compilation of the other projects(that use ITK) by setting CMAKE_C(XX)_COMPILER. So, the way I did it is to have a local copy of the gnu compiler (used inside the wrapper) and name it as mpicc/mpicxx. Later, I'll replace it with the original MPI wrappers for all my future projects. I know, this is a dirty hack for now. CMake is happy, and so are we!


From here on, I'll talk about the problems that you *might* encounter in the method 2. Feel free to try 1


You can find the TryRunResults.cmake file [[Media:TryRunResults_gnu.cmake|here]] . You would need this after configuring the build for the first time. These are necessary for the build system to know more about the target system because we are cross compiling.


* Remove -fp_port from compiler options for compiling Utilities/vxl/v3p/netlib/blas/slamch.c and Utilities/vxl/v3p/netlib/blas/dlamch.c
** -fp_port option belongs to icc(?), I dont know how it crept into this. May be the compiler identification step was wrong?
** For the time being, one could do 'make -n|grep fp_port'. Run the commands manually after removing -fp_port. Then continue with 'make'.

Compilation successful by method (2)! Hurray!