[Insight-users] Error when writing a large number of files

[Kedar Grama]

Hi,

I have an executable that is writing 140,000 image files to a folder
and the files are being written in 16 openmp threads. The same code
runs fine on a Linux machine with similar hardware with upto 80
threads but when I run it on windows(the machine has about 30GB of
free RAM and the hard disk still has about 0.5TB space free) I get the
following error.
HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: ..\..\..\..\..\..\..\ITK_V\Modules\ThirdParty\HDF5\src\itkhdf5\src\H5I.c
 line 1131 in H5Iget_type(): library initialization failed
    major: Function entry/exit
    minor: Unable to initialize object
  #001: ..\..\..\..\..\..\..\ITK_V\Modules\ThirdParty\HDF5\src\itkhdf5\src\H5.c
line 181 in H5_init_library(): unable to initialize link interface
    major: Function entry/exit
    minor: Unable to initialize object
  #002: ..\..\..\..\..\..\..\ITK_V\Modules\ThirdParty\HDF5\src\itkhdf5\src\H5L.c
 line 231 in H5L_init(): interface initialization failed
    major: Function entry/exit
    minor: Unable to initialize object
  #003: ..\..\..\..\..\..\..\ITK_V\Modules\ThirdParty\HDF5\src\itkhdf5\src\H5L.c
 line 260 in H5L_init_interface(): unable to register external link class
    major: Links
    minor: Link class not registered
  #004: ..\..\..\..\..\..\..\ITK_V\Modules\ThirdParty\HDF5\src\itkhdf5\src\H5Lex
ternal.c line 614 in H5L_register_external(): unable to register external link c
lass
    major: Links
    minor: Link class not registered
  #005: ..\..\..\..\..\..\..\ITK_V\Modules\ThirdParty\HDF5\src\itkhdf5\src\H5L.c
 line 1499 in H5L_register(): unable to extend link type table
    major: Resource unavailable
    minor: No space available for allocation


The skeleton of the code looks like this:
Primary_function( )
{
  #pragma omp parallel for num_threads(n_thr)
  for( itk::IndexValueType i=0; i<labelsList.size(); ++i )
  {
    //some processing
    SegmentNucleiInBox( arguments );
    //more processing
  }
}

SegmentNucleiInBox( arguments  )
{
  typedef itk::ImageFileWriter< IntermediateLabelType > WriterType;
  //Some processing
  WriterType::Pointer writer = WriterType::New();
  writer->SetInput( PointerImageToBeWritten );
  writer->SetFileName( Filename );
  try{ writer->Update(); }
  catch( itk::ExceptionObject & excp )
  {
    std::cerr <<  "Write for intermediate labels failed" << excp << std::endl;
  }
}

How do I debug the problem?

Thanks,

-Kedar