[Insight-users] [ome-users] bioformats and ITK

Mark Hiner hiner at wisc.edu
Fri May 17 16:39:50 EDT 2013 

Joe,

 That's great! It's almost certainly an issue of the jars being put in the
wrong place because of the Debug/Release addition on Windows.

 I confirmed that it doesn't work on Windows :-p I think it's in the
DownloadSCIFIO.cmake.in which downloads to ${jarsDownloadDirectory}... but
the jars are installed from
${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/${jarsDownloadDirectory} which I'm
assuming points to lib/Debug/jars, which doesn't exist.

Unfortunately I don't have time to verify that fixes things today. I'll try
to test it on the weekend or Monday.


On Fri, May 17, 2013 at 2:29 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:

> Mark,
>
> Just want to let you know that I repeated the same steps on my macbook
> with Mac OSX 10.8.3, and program works fine.
>
> Joe
>
>
> On Fri, May 17, 2013 at 10:45 AM, Mark Hiner <hinerm at gmail.com> wrote:
>
>> Joe,
>>
>>  That's odd.. not sure why manually adding them to the classpath didn't
>> work. I will test locally on a Windows 7 box and let you know if I can
>> reproduce and/or resolve the issue.
>>
>> Thanks for reporting it, hopefully we can get it working for you soon.
>>
>> - Mark
>>
>>
>> On Thu, May 16, 2013 at 4:30 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:
>>
>>> Mark,
>>>
>>> Yes, I was using the command "ITKIOSCIFIOTestDriver
>>> itkSCIFIOImageIOTest inputfile outputfile".
>>>
>>> I am on Windows 7. The two jars (loci_tools.jar and
>>> scifio-itk-bridge-1.0.0.jar) are in ITK/lib/jars.
>>>
>>> I manually added the path to classpath, and my classpath is now:
>>> CLASSPATH=.;C:\Program Files
>>> (x86)\Java\jre6\lib\ext\QTJava.zip;D:\Joe\dev\ITK\lib\jars
>>>
>>> But I still have no luck.
>>>
>>> After running the command, I first get a dialog of
>>>  "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>> Program will exit."
>>> Please see attached image.
>>>
>>> And there are also error messages in the console:
>>>
>>> D:\Joe\dev\ITK\bin\Release>ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest
>>> C:\Users\phsiao\Desktop\itk-scifio-build\Release\
>>> chZT.lsm C:\Users\phsiao\Desktop\itk-scifio-build\Release\abc.tif
>>> reader->GetUseStreaming(): 1
>>> done checking streaming usage
>>> itk::ExceptionObject (0000000000A4EF10)
>>> Location: "unknown"
>>> File: .\itkSCIFIOImageIO.cxx
>>> Line: 172
>>> Description: itk::ERROR: SCIFIOImageIO(0000000002A5C080): SCIFIOImageIO
>>> exited abnormally. java.lang.NoClassDefFoundErr
>>> r: loci/scifio/itk/SCIFIOITKBridge
>>> Caused by: java.lang.ClassNotFoundException:
>>> loci.scifio.itk.SCIFIOITKBridge
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>> Exception in thread "main"
>>>
>>>
>>>
>>>
>>> On Thu, May 16, 2013 at 4:46 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>
>>>> Hi Joe,
>>>>
>>>>  Hmm.. well that is the correct package specification. What was the
>>>> exact command you used? I'm assuming it was something like
>>>>
>>>> "ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest inputfile outputfile"
>>>>
>>>> ?
>>>>
>>>> Anyway it sounds like the scifio-itk-bridge jar isn't on your
>>>> classpath. I just tested a clean build off ITK.git master (commit
>>>> 2e056e0cee1271d3080a9bfc4f7353c1d9165ef9) and it worked for me... but
>>>> that's on OSX.
>>>>
>>>> What's in your build/lib/jars directory? It should have 2 things:
>>>> loci_tools.jar
>>>> scifio-itk-bridge-1.0.0.jar
>>>>
>>>> It's possible that they aren't being added to the classpath correctly
>>>> though, if on Windows there's another directory level (e.g. is it
>>>> build/Release/lib/jars ?). So if they are in a different directory, let me
>>>> know.
>>>>
>>>> If the jars are there, try manually adding them to your classpath for
>>>> now and let me know how that goes. If they're not then it's a problem with
>>>> the download script.
>>>>
>>>> You can download the jars manually from:
>>>>
>>>> http://cvs.openmicroscopy.org.uk/snapshots/bioformats/4.4.5/loci_tools.jar
>>>>
>>>> http://jenkins.imagej.net/view/SCIFIO/job/SCIFIOITKBridge/lastSuccessfulBuild/artifact/target/scifio-itk-bridge-1.0.0.jar
>>>>
>>>> That's all I can think of at the moment...
>>>>
>>>> - Mark
>>>>
>>>>
>>>> On Thu, May 16, 2013 at 1:02 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:
>>>>
>>>>> Hi Mark,
>>>>>
>>>>> Glad to see SCIFIO is now becoming ITK's remote module. Thanks for
>>>>> your continuous working.
>>>>> I did a clean git clone of the latest ITK, and configure it with
>>>>>
>>>>>    Fetch_SCIFIO
>>>>>    BUILD_TESTING
>>>>>
>>>>> set to ON. Then I run ITKIOSCIFIOTestDriver.exe on an image file, but
>>>>> I got the error message
>>>>>
>>>>> "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>>>> Program will exit."
>>>>>
>>>>> Do I miss setting some path, or it still requires Bio-Formats?
>>>>>
>>>>> Thanks,
>>>>> Joe
>>>>>
>>>>>
>>>>> On Fri, Apr 26, 2013 at 2:43 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>
>>>>>> Hi again Joe,
>>>>>>
>>>>>>  I noticed a couple things that may help you: in the
>>>>>> itkVectorImageSCIFIOImageIOTest you sent me (and in all the test classes I
>>>>>> mentioned in my last e-mail) the PixelType and Dimension variables must be
>>>>>> set carefully.
>>>>>>
>>>>>>  For example, with the code in the state you sent me, PixelType is an
>>>>>> unsigned short (which is 16 bits) and the dimension is 3.
>>>>>>
>>>>>>  These settings would be incorrect for the 2chZT.lsm sample data, as
>>>>>> it has an 8 bit unsigned pixel type (so PixelType should be defined as
>>>>>> unsigned char). Also if you're using the SCIFIOImageIO you can typically
>>>>>> set dimensions to 5.
>>>>>>
>>>>>>  I don't know the specifications of your .oif dataset, but the
>>>>>> PixelType and dimensions may need to be adjusted for it.
>>>>>>
>>>>>>   Sorry if you already knew this and were updating these values!
>>>>>>
>>>>>>   Anyway, I've been working on improving the stability of the
>>>>>> SCIFIOImageIO plugin. Using the itkSCIFIOImageIOTest I mentioned
>>>>>> previously, I confirmed that I was able to read and write the 2chsZT.lsm
>>>>>> file and the .oif test images I have access to.
>>>>>>
>>>>>>   These changes aren't in an ITK patch set yet because they include
>>>>>> changes to both the c++ and java code, which will may require some
>>>>>> restructuring of SCIFIOImageIO.
>>>>>>
>>>>>> However, if you're eager to test these changes, you could build off
>>>>>> my development branches:
>>>>>> 1. Clone my ITK repo. <https://github.com/hinerm/ITK> The default
>>>>>> branch is what you want here.
>>>>>> 2. Build ITK as normal (with tests enabled)
>>>>>> 3. Clone my Bio-Formats repo <https://github.com/hinerm/bioformats>.
>>>>>> 4. Check out the itk-bridge-fixes branch in Bio-Formats
>>>>>> 5. From the top level of Bio-Formats, run "ant tools"
>>>>>> 6. Copy the loci_tools.jar from ${bio-formats}/artifacts to
>>>>>> ${ITK_build}/lib/jars/
>>>>>>    (this will overwrite the loci_tools.jar that was downloaded
>>>>>> automatically when ITK built)
>>>>>>
>>>>>>  At this point you can link in the ImageToVTKFilter you were using or
>>>>>> run the SCIFIO ImageIO test programs included in ITK.
>>>>>>
>>>>>>  Whether you build these manually or not, I'll try to get a new
>>>>>> official patch set released soon.
>>>>>>
>>>>>>  Let me know if you run into any problems.
>>>>>>
>>>>>> Thanks,
>>>>>> Mark
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 25, 2013 at 4:12 PM, Mark Hiner <hinerm at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Joe,
>>>>>>>
>>>>>>>
>>>>>>> Here are my steps to compile the program, in case I missed something:
>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>
>>>>>>>
>>>>>>> This looks good. Note that a Patch set 4 is now out, and I'm working
>>>>>>> on Patch 5. I'm hoping to include any necessary fixes for your issues in
>>>>>>> patch 5.
>>>>>>>
>>>>>>> So, to clarify here - you're end goal is to convert your .oif
>>>>>>> dataset to .ome.tiff? Is that right? Are you specifically wanting to
>>>>>>> extract a subset of the image or anything? If you're just trying to do a
>>>>>>> simple conversion it may be better to use the tests that are part of the
>>>>>>> ITK SCIFIOImageIO module. You can take a look at them at:
>>>>>>> $ITK/Modules/IO/SCIFIO/test
>>>>>>>
>>>>>>> To build them, just turn the BUILD_TESTING flag on when configuring
>>>>>>> cmake. The ITKIOSCIFIOTestDriver is built to $build/bin
>>>>>>>
>>>>>>> I've been using the itkSCIFIOImageIOTest to test image conversion,
>>>>>>> for example.
>>>>>>>
>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>>
>>>>>>>
>>>>>>> Looking at the script that suggests it read your dataset
>>>>>>> successfully but failed to write.
>>>>>>>
>>>>>>> Using the ImageIOTest mentioned above I tried converting the
>>>>>>> 2chsZT.lsm to an .ome.tiff. It got through half the planes and then crashed
>>>>>>> (which sounds like an improvement over your report, so I'm taking that as a
>>>>>>> small victory).
>>>>>>>
>>>>>>> There are some oddities in the ITK <-> Bio-Formats axis conversion..
>>>>>>> it looks like only 1 of the 2 channels was detected for some reason. So
>>>>>>> I'll fix that in the next patch set.
>>>>>>>
>>>>>>> I got similar results with an .oif file locally.
>>>>>>>
>>>>>>> So I apologize, as I don't feel I've helped you much yet. I just
>>>>>>> wanted you to know that I'm still looking at these issues. Once I'm to the
>>>>>>> point that I can convert that 2chsZT.lsm and local .oif files I'll try to
>>>>>>> finalize another patch and see if that fixes things for you.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mark
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 24, 2013 at 2:26 PM, Joe Ping-Lin Hsiao <
>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>
>>>>>>>> Hi Mark,
>>>>>>>>
>>>>>>>>     Thanks for the explanation. I have tried switching to
>>>>>>>> SCIFIOImageIO in the cpp code, but unfortunately the program still hangs
>>>>>>>> during writing files out (with a .oif input). I have attached the source
>>>>>>>> code. What I did was just replacing itk::BioFormatsImageIO with
>>>>>>>> itk::SCIFIOImageIO and a change of the included header file.
>>>>>>>>
>>>>>>>> Here are my steps to compile the program, in case I missed
>>>>>>>> something:
>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>
>>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>> By the way, I am using VS 2008 sp1 on Windows 7 64-bit.
>>>>>>>>
>>>>>>>> I also tested the latest bf-itk-pipe code. The hanging happens
>>>>>>>> after the line printing "pre-update" in this case.
>>>>>>>> Note that I had to link bf-itk-pipe against the SCIFIOImageIO ITK
>>>>>>>> otherwise there'd be a link error if linked against the master ITK.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Joe
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 23, 2013 at 4:11 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>>>>
>>>>>>>>> Hi Joe,
>>>>>>>>>
>>>>>>>>>  First of all, in case you are using the bf-itk-pipes plugin, you
>>>>>>>>> should know that we currently have a patch under review<http://review.source.kitware.com/9450>by the ITK developers. This patch adds a true Bio-Formats (SCIFIO) ITK
>>>>>>>>> ImageIO, which will be available by default in ITK.
>>>>>>>>>
>>>>>>>>>  This means that the old bf-itk-pipes plugin won't be supported,
>>>>>>>>> as all development efforts are going towards the SCIFIOImageIO.
>>>>>>>>>
>>>>>>>>>  As this code is still under review, there are issues that I'm
>>>>>>>>> currently working on resolving.
>>>>>>>>>
>>>>>>>>>  So first I'd ask that you check out the latest patch code using
>>>>>>>>> the links on that review site and rework the splitting code to use
>>>>>>>>> SCIFIOImageIO<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/include/itkSCIFIOImageIO.h>instead of the BioFormatsImageIO. You can also look at the test
>>>>>>>>> directory<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/test/itkSCIFIOImageIOTest.cxx>for more sample code.
>>>>>>>>>
>>>>>>>>>  If you have any trouble updating your code, please let me know
>>>>>>>>> and I'll be happy to help.
>>>>>>>>>
>>>>>>>>>  As to your actual issue of the .oif image hanging, I actually
>>>>>>>>> just fixed a bug in the last week regarding hanging when plane size is
>>>>>>>>> miscalculated by the plugin, so with any luck that was your issue. :)
>>>>>>>>>
>>>>>>>>>  If the latest SCIFIOImageIO still gives you problems with your
>>>>>>>>> .oif dataset though let me know and I'll investigate.
>>>>>>>>>
>>>>>>>>>  As for the 2chZT data, that sounds unrelated to the hanging so
>>>>>>>>> I'll take a look at it locally when I get a chance.
>>>>>>>>>
>>>>>>>>>  Thanks for using our software and reporting these issues!
>>>>>>>>>
>>>>>>>>> - Mark
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 23, 2013 at 2:40 PM, Joe Ping-Lin Hsiao <
>>>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>>>
>>>>>>>>>> Mark,
>>>>>>>>>>      I tested your code with the .lsm download from
>>>>>>>>>> http://www.loci.wisc.edu/files/software/data/2chZT.zip. The
>>>>>>>>>> program just crashed with the error message
>>>>>>>>>>
>>>>>>>>>> "This application has requested the Runtime to terminate it in an
>>>>>>>>>> unusual way.
>>>>>>>>>> Please contact the application's support team for more
>>>>>>>>>> information."
>>>>>>>>>>
>>>>>>>>>> If I tested with my own 5D .oif image collection, the execution
>>>>>>>>>> hangs at 'extractor->Update()' and it does not even use any CPU. Any idea
>>>>>>>>>> how to fix it?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Oct 19, 2012 at 12:41 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>
>>>>>>>>>>>  I updated your test<https://gist.github.com/3900888/81064cf96fb06b4575a64e81481fda0936cd9056>to use an ExtractImageFilter in the IO pipeline. When a Reader's output is
>>>>>>>>>>> piped to this Filter, the reader->Update() call only reads the extracted
>>>>>>>>>>> region.
>>>>>>>>>>>
>>>>>>>>>>>  When I tested on the
>>>>>>>>>>> ConfocalMemCitrine516LeftEar18hpfAug17_2012_Fish1.lsm test image you
>>>>>>>>>>> uploaded, it correctly splits out the stacks for each time point. And when
>>>>>>>>>>> I tested on a 9GB image it processed each time point without reading the
>>>>>>>>>>> entire file.
>>>>>>>>>>>
>>>>>>>>>>>  So I'm hoping this solves your issue. If not, just let me know
>>>>>>>>>>> how it's failing for you and I'll try again!
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Mark
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Oct 17, 2012 at 2:34 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> One thought is that, as the itkBioFormatsImageIO is a
>>>>>>>>>>>> StreamingImageIOBase<http://www.itk.org/Doxygen320/html/classitk_1_1StreamingImageIOBase.html>,
>>>>>>>>>>>> you could just set the desired region (as you were) and use *
>>>>>>>>>>>> its* Read and Write methods instead of wrapping it in a Reader
>>>>>>>>>>>> or a Writer.
>>>>>>>>>>>>
>>>>>>>>>>>> I'll try to put a code example up using this approach. It would
>>>>>>>>>>>> be a good test case for the bf-itk-pipe plug-in anyway.
>>>>>>>>>>>>
>>>>>>>>>>>> - Mark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Oct 17, 2012 at 1:56 PM, Mark Hiner <hinerm at gmail.com>wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>
>>>>>>>>>>>>>  Just wanted to let you know that I'm still looking at your
>>>>>>>>>>>>> issue today. I updated the gist<https://gist.github.com/3900888/7aab8fddb38af87ea98cd49dd921524ace6430c8>to reflect the topics discussed in this e-mail, using the region setting
>>>>>>>>>>>>> method calls instead of pipelining, and added some (hopefully) helpful
>>>>>>>>>>>>> print statements to see what's going on with the regions.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  From what I can tell there are two separate issues preventing
>>>>>>>>>>>>> your code from writing the desired regions.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  First of all, the IORegion needs to be set on the Writer, for
>>>>>>>>>>>>> example:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>     writer->SetImageIO(io);
>>>>>>>>>>>>>     writer->SetInput(reader->GetOutput());
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>     writer->SetIORegion(region);
>>>>>>>>>>>>>     writer->Write();
>>>>>>>>>>>>>
>>>>>>>>>>>>>  In the original code you sent to me, that wasn't happening
>>>>>>>>>>>>> and it was defaulting to the first 3D region and thus the first time point.
>>>>>>>>>>>>> I mentioned this, without providing a code example, in a past e-mail - so
>>>>>>>>>>>>> apologies if you've already updated your code!
>>>>>>>>>>>>>
>>>>>>>>>>>>>  Note that according to the itkImageFileWriter API<http://www.itk.org/Doxygen/html/classitk_1_1ImageFileWriter.html#ab2cba65f716aa75ec962171ee0b46fe3>Write() is a special form of Update() that respects the IORegion. However,
>>>>>>>>>>>>> in practice, for me it still just writes the largest possible region
>>>>>>>>>>>>> instead of what was specified. So that could suggest a bug in the
>>>>>>>>>>>>> bf-itk-pipes plug-in.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  The second issue is that even after manually specifying the
>>>>>>>>>>>>> IO regions, the Reader output still has the largest possible dimensions
>>>>>>>>>>>>> (T=2) so the current gist test ends up creating 2 full copies of the image.
>>>>>>>>>>>>> I think this was always happening at the Reader, but then it was truncated
>>>>>>>>>>>>> to the first time point you saw by the Writer. Anyway, this implies that
>>>>>>>>>>>>> the full image is still being read, despite the region specifications.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  I think this is a side effect of setting using the
>>>>>>>>>>>>> LargestPossibleRegion in the Update() method, as mentioned in the itkProcessObject
>>>>>>>>>>>>> specification<http://www.itk.org/Doxygen/html/classitk_1_1ProcessObject.html#a4041fb21e9105500eee311e265691bd5>
>>>>>>>>>>>>> .
>>>>>>>>>>>>>
>>>>>>>>>>>>>  Right now I'm going through the examples from this ITK Users
>>>>>>>>>>>>> thread<http://itk-insight-users.2283740.n2.nabble.com/Memory-limits-of-ITKImage-object-td5670422.html>about streaming for a better pipelining solution. I'm also testing to see
>>>>>>>>>>>>> what can be done at the ImageIO level, as when Read and Write are hit in
>>>>>>>>>>>>> the itkBioFormatsImageIO, GetIORegion() is already returning the largest
>>>>>>>>>>>>> possible region, instead of the specified region.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  I'll keep you updated.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 1:37 PM, Kishore Mosaliganti <
>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>  Dear Mark,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the clarification. My original problem remains
>>>>>>>>>>>>>> unresolved. For example, in the LSM gist on lines 104-107:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Here, calling reader->Update() loads the entire image into
>>>>>>>>>>>>>> memory. If you have a very large LSM file (several gigabytes large), then
>>>>>>>>>>>>>> this code will fail since such an image cannot get into the memory.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hence, I am trying to send the read region directly into the
>>>>>>>>>>>>>> reader using
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> io->SetRegion(region);
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and then
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> reader->SetImageIO(io);
>>>>>>>>>>>>>> reader->Update();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> But this step is not giving me two different timepoints. It
>>>>>>>>>>>>>> is writing out the same timepoint.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 2:13 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello again,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  I created a gist for your LSM test and updated it with my
>>>>>>>>>>>>>>> most recent code:
>>>>>>>>>>>>>>> https://gist.github.com/3900888
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  That example is currently working for me, in that it writes
>>>>>>>>>>>>>>> out 2 images of 72 planes - one for T=0 and one for T=1. Since everything
>>>>>>>>>>>>>>> in ITK seems to be pipeline-oriented, I am not sure there is a solution to
>>>>>>>>>>>>>>> splitting out channels just using Setters like we were both trying.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  Instead, I followed this ROI Filter example<http://www.vtk.org/Wiki/ITK/Examples/ImageProcessing/RegionOfInterestImageFilter>and used the logic from my last e-mail.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  I do need to correct one thing I said in my last e-mail:
>>>>>>>>>>>>>>> SizeT should always be 1, as it is a relative offset to the Index value.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  When I ran that test, both images went to grayscale.. I'm
>>>>>>>>>>>>>>> going to see if that's an error in the bf-itk-pipes logic, or just
>>>>>>>>>>>>>>> something that needs to be set up differently in using the ITK API.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  Let me know if you run into any more problems with this
>>>>>>>>>>>>>>> code, or if you just want to discuss any part of it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks again for using our software :)
>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 11:00 AM, Mark Hiner <hiner at wisc.edu
>>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  I am sorry I wasn't able to get to your issue yesterday.
>>>>>>>>>>>>>>>> Looking at it today, I think there are a couple of things going on here.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  First of all: in your test class
>>>>>>>>>>>>>>>> - I consolidated a BioFormatsImageIO and the ImageIOBase.
>>>>>>>>>>>>>>>> - In the region updating loop, the wrong region (dim-3
>>>>>>>>>>>>>>>> instead of dim-2) was being set, and only the index is set. I suspect both
>>>>>>>>>>>>>>>> the index and size need to be set.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  I attached an updated version of your test, where in the
>>>>>>>>>>>>>>>> first iteration it sets IndexT = 0, sizeT = 1. The second iteration sets
>>>>>>>>>>>>>>>> IndexT = 1, sizeT = 2.  I believe those regions will split the time points
>>>>>>>>>>>>>>>> as you wanted.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But there is a second issue: although the Reader IORegion
>>>>>>>>>>>>>>>> is correct (I think) the Writer is not. It is defaulting to the ITK base 3D
>>>>>>>>>>>>>>>> region. When I tried setting the Writer's region to our desired region, it
>>>>>>>>>>>>>>>> throws an exception because the 5D region can't be contained by what it
>>>>>>>>>>>>>>>> thinks is the largest possible (3D) region. I am refreshing myself on the
>>>>>>>>>>>>>>>> ITK API and once we figure out how to set that region, I think it will
>>>>>>>>>>>>>>>> write the images correctly (or we can move on to the next problem).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> In your test I changed the output format to ome.tiff
>>>>>>>>>>>>>>>> because I know that will use the itkBioFormatsImageIO writer code, which I
>>>>>>>>>>>>>>>> know is capable of handling 5D writing.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, in testing I found a couple of bugs in the
>>>>>>>>>>>>>>>> ITKPipesBridge code. I started a new branch to fix these and any other
>>>>>>>>>>>>>>>> issues that may be discovered. If you want to use the itkBioFormatsImageIO
>>>>>>>>>>>>>>>> writer code you'll have to use this new branch for now. You can find it
>>>>>>>>>>>>>>>> here:
>>>>>>>>>>>>>>>> https://github.com/hinerm/bioformats/commits/bf-itk-fixes
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As always, let me know if this isn't fully addressing your
>>>>>>>>>>>>>>>> issue. Otherwise, I'll let you know as soon as I have the Writer region
>>>>>>>>>>>>>>>> setting properly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 11:28 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thank you for the email and invaluable help. The
>>>>>>>>>>>>>>>>> bioformats-itk plugin is wonderful. I am attaching my script here to help
>>>>>>>>>>>>>>>>> you debug. Let me know if you have any questions on my script. I am
>>>>>>>>>>>>>>>>> starting with a 5D (XYZTC) image and trying to write out all the 3D
>>>>>>>>>>>>>>>>> timepoints for the first channel.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 12:19 PM, Mark Hiner <
>>>>>>>>>>>>>>>>> hinerm at gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  I helped develop the bf-itk-pipe plugin and am
>>>>>>>>>>>>>>>>>> investigating your issue right now.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  What you're doing seems reasonable, so I'm trying to
>>>>>>>>>>>>>>>>>> determine if there's a bug in the plugin. My goal is to have some working
>>>>>>>>>>>>>>>>>> example code for you (and a fix if necessary) by the end of the day.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank you for using our plugin! I hope we can get it
>>>>>>>>>>>>>>>>>> working for you soon.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 10:54 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Dear ITK and OME users,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To further elaborate, I figured that this has nothing to
>>>>>>>>>>>>>>>>>>> do with the itkStreamingImageFilter.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Instead, the itkImageFileReader seems to be extracting
>>>>>>>>>>>>>>>>>>> the same image region although I update the requested region in the
>>>>>>>>>>>>>>>>>>> BioformatsImageIO class :;
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>     itk::BioFormatsImageIO::Pointer io =
>>>>>>>>>>>>>>>>>>> itk::BioFormatsImageIO::New();
>>>>>>>>>>>>>>>>>>>     io->SetIORegion( region ); // doesn't make a
>>>>>>>>>>>>>>>>>>> difference in terms of image data extracted. Its always from the beginning.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So, my general question is whether the BioformatsImageIO
>>>>>>>>>>>>>>>>>>> expects the whole LSM image to be loaded into memory before writing it out?
>>>>>>>>>>>>>>>>>>> I would like to specify small image regions since my LSM is too large to be
>>>>>>>>>>>>>>>>>>> fully loaded into memory. How else can I stream data from large LSM?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 8:53 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I am trying to use bioformats and itk. I downloaded and
>>>>>>>>>>>>>>>>>>>> compiled bf-itk-pipe using cmake by linking against ITK 3.2:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I tested out itkBFImageInfo and
>>>>>>>>>>>>>>>>>>>> ./itkRGBBioFormatsImageIOTest on a few simple LSM images. It works great
>>>>>>>>>>>>>>>>>>>> and writes out all the associated metadata and pixel data for the first
>>>>>>>>>>>>>>>>>>>> timepoint.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> My LSM microscopy image is 5D. It is
>>>>>>>>>>>>>>>>>>>> X-Y-Z-Time-Channel. I am interested in extracting individual timepoints
>>>>>>>>>>>>>>>>>>>> from the LSM file for processing. The filter seems to be using
>>>>>>>>>>>>>>>>>>>> itkStreamingImageFilter. I set the NumberOfStreams to 10 but that still
>>>>>>>>>>>>>>>>>>>> seems to still write out only the first image.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> How do I set the output requestion region for an
>>>>>>>>>>>>>>>>>>>> individual timepoint?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20130517/43d2f1de/attachment-0001.htm>

More information about the Insight-users mailing list