[ITK] convert .lif file with bfconvert

Wed Jun 4 10:52:33 EDT 2014

Hi Mark

I thinks ExtractImageFilter is what I was looking for and there are
also some nice examples.
I'll try...

Thanks again
Michael

On 4 June 2014 13:59, Mark Hiner <hiner at wisc.edu> wrote:
> Hi Michael,
>
>
>> This worked for me as well. Thanks a lot.
>
>
> I'm glad we got it working too!
>
>
>> Could you give me a hint on how to get the single channells out of
>> "itk::Image< unsigned short, 5 >"?
>
>
> I spend most of my time in Java-land, so I'm not as familiar with the ITK
> API as I'd like to be. It does sound like the ExtractImageFilter could do
> what you want by collapsing the image along the channel axis. Maybe someone
> else on this list can provide a code example, or a better solution?
>
> Thanks,
> Mark
>
>
> On Tue, Jun 3, 2014 at 1:56 AM, Michael Meuli <michael.meuli at gmail.com>
> wrote:
>>
>> Hi Mark
>>
>> > export JAVA_FLAGS=-Xmx3400m
>> > /SCIFIOTestDriver itkSCIFIOImageIOTest
>> > bfconvert_output/output_series_0.ome.tiff test.ome.tiff -w -d 5
>>
>> This worked for me as well. Thanks a lot.
>> So I can convert my multi-channell-z-series images contained in my
>> .lif files with bfconvert into ome.tiff and read these with itk thanks
>> to the SCIFIO module.
>> If I see this correctly this reads the image into an object of type
>> "itk::Image< unsigned short, 5 >".
>> I was trying to read it into "itk::Image< <itk::Vector<unsigned short,
>> 3>>, 3 >" and to use VectorIndexSelectionCastImageFilter
>> to get to the single channells.
>> Could you give me a hint on how to get the single channells out of
>> "itk::Image< unsigned short, 5 >"?
>>
>> Thanks,
>> Michael
>>
>>
>>
>> On 2 June 2014 15:27, Mark Hiner <hiner at wisc.edu> wrote:
>> > Hi Michael,
>> >
>> >
>> >> So far I've used "File > Open". Sorry.
>> >
>> >
>> > No need to apologize! This is a common misunderstanding. When you use
>> > "File
>> >> Open", your .ome.tiff gets picked up by the ImageJ TIFF reader instead
>> >> of
>> > Bio-Formats. So that's why metadata is missing. This is actually
>> > something
>> > we're improving with the ImageJ2 release this week: "File > Open" will
>> > be
>> > able to work with Bio-Formats automatically.
>> >
>> >
>> >> I just tried "File > Import > Bio-Formats" and it works as well.
>> >
>> >
>> > This made me suspect that there was something going wrong on my end. I
>> > should have asked which version of Bio-Formats you were using bfconvert
>> > from.. I was using it from v5.0.1. I updated to v5.0.2 this morning and
>> > tried bfconvert, and now my output .ome.tiffs open just fine in Fiji's
>> > Bio-Formats Importer. So I'm sorry for any misguidance there.
>> >
>> > Anyway, as for the itkSCIFIOImageIO use, I was still getting
>> > OutOfMemoryErrors. So you'll want to increase the amount of memory Java
>> > can
>> > use. By default, the ImageIO creates a JVM with 256m. You can override
>> > this
>> > by setting a "JAVA_FLAGS" environment variable. Once I went over 3g, I
>> > was
>> > actually able to convert your data appropriately, finally! To increase
>> > available memory, I used the command:
>> >
>> > export JAVA_FLAGS=-Xmx3400m
>> >
>> > Also! I forgot there were some important considerations with other flags
>> > to
>> > use with the itkSCIFIOImageIO:
>> >
>> > use "-d 5". I know your data is 3D + channels, but as an .ome.tiff it
>> > will
>> > be considered 5D by the imageio and the wrong dimensions could get
>> > truncated
>> > if you set -d less than 5.
>> > If necessary, set pixel type with "-t". The default is unsigned short
>> > (an
>> > ITK pixel type, which is equivalent to Bio-Formats's uint16). Your data
>> > is
>> > uint16 so there's no need to use this flag, just wanted you to be aware
>> > of
>> > it.
>> >
>> > So, the command that worked for me was:
>> >
>> > /SCIFIOTestDriver itkSCIFIOImageIOTest
>> > bfconvert_output/output_series_0.ome.tiff test.ome.tiff -w -d 5
>> >
>> > Let me know if you have any more problems,
>> >
>> > Mark
>> >
>> >
>> >
>> > On Sun, Jun 1, 2014 at 1:19 AM, Michael Meuli <michael.meuli at gmail.com>
>> > wrote:
>> >>
>> >> Hi Mark,
>> >>
>> >> I just tried "File > Import > Bio-Formats" and it works as well.
>> >> If I select one of the files produced with ...%s.ome.tiff in the first
>> >> open dialog box and then Hyperstack in the "Bio-Formats Import
>> >> Options", I can open any of the series files.
>> >> If I separate on of these files into another directory and try to open
>> >> it the same way, Fiji crashes.
>> >> I can also open the file containing all the series, but made some
>> >> coffee in between.
>> >> So far I've used "File > Open". Sorry. "File > Open" also works for
>> >> the separated file, in case this matters, but shows them in gray and
>> >> plane by plane (no channell selection).
>> >>
>> >> Thanks,
>> >> Michael
>> >>
>> >>
>> >> On 1 June 2014 03:08, Mark Hiner <hiner at wisc.edu> wrote:
>> >> > Hi Michael,
>> >> >
>> >> >
>> >> >> I can import the bfconvert output into Fiji with no problem.
>> >> >
>> >> >
>> >> > I just wanted to confirm, you're using "File > Import > Bio-Formats"
>> >> > or
>> >> > the
>> >> > Bio-Formats importer in Fiji, correct? It definitely didn't work for
>> >> > me
>> >> > when
>> >> > I tried yesterday. I can check next week to see if I was using an
>> >> > outdated
>> >> > Bio-Formats or something..
>> >> >
>> >> > Thanks,
>> >> > Mark
>> >> >
>> >> >
>> >> > On Sat, May 31, 2014 at 7:52 AM, Michael Meuli
>> >> > <michael.meuli at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi Mark
>> >> >>
>> >> >> I can import the bfconvert output into Fiji with no problem.
>> >> >> (If I do "./bfconvert /path/to/wt-dead-A.lif
>> >> >> /path/to/output.ome.tiff"
>> >> >> or
>> >> >> "./bfconvert /path/to/wt-dead-A.lif
>> >> >> /path/to/output_series_%s.ome.tiff"
>> >> >> I can open the file(s) with Fiji.)
>> >> >> If I open the .lif file with Fiji and export one series to ome.tiff
>> >> >> I
>> >> >> can read the output with ./SCIFIOTestDriver itkSCIFIOImageIOTest ...
>> >> >> whereas I'm not able to read the output of bfconvert.
>> >> >>
>> >> >> Thanks a lot and best regards
>> >> >> Michael
>> >> >>
>> >> >>
>> >> >>
>> >> >> On 30 May 2014 16:18, Mark Hiner <hinerm at gmail.com> wrote:
>> >> >> > Hi Michael,
>> >> >> >
>> >> >> >> "./SCIFIOTestDriver
>> >> >> >> itkSCIFIOImageIOTest/path/to/output_series_0.ome.tiff
>> >> >> >> /path/to/scifio-test.ome.tiff"
>> >> >> >
>> >> >> > Just a reminder, since you're writing ome.tiff you should have a
>> >> >> > "-w"
>> >> >> > flag
>> >> >> > at the end of this command. It sounds like you were using -w, just
>> >> >> > wanted to
>> >> >> > clarify for anyone else reading this thread.
>> >> >> >
>> >> >> >> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>> >> >> >> space
>> >> >> >
>> >> >> >  There are two problems here. First, it seems that in
>> >> >> > SCIFIOImageIO
>> >> >> > the
>> >> >> > use
>> >> >> > of Bio-Formats is overly aggressive in pulling in all the series,
>> >> >> > thus
>> >> >> > the
>> >> >> > OutOfMemoryError. If you physically isolate a single series (e.g.
>> >> >> > delete
>> >> >> > or
>> >> >> > move to another directory series 1-19) the application will run,
>> >> >> > but
>> >> >> > we
>> >> >> > get
>> >> >> > another exception (at least on my end):
>> >> >> >
>> >> >> > ./SCIFIOTestDriver itkSCIFIOImageIOTest output_series_0.ome.tiff
>> >> >> > scifio-test.ome.tiff -w
>> >> >> >
>> >> >> > reader->GetUseStreaming(): 1
>> >> >> > done checking streaming usage
>> >> >> > ITK test driver caught an ITK exception:
>> >> >> >
>> >> >> > itk::ExceptionObject (0x7ff8838b6ae8)
>> >> >> > Location: "unknown"
>> >> >> > File:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > /Users/mhiner/loci/ITK/build/Modules/Remote/SCIFIO/src/itkSCIFIOImageIO.cxx
>> >> >> > Line: 180
>> >> >> > Description: itk::ERROR: SCIFIOImageIO(0x7ff8838b4f00):
>> >> >> > SCIFIOImageIO
>> >> >> > exited
>> >> >> > abnormally. Creating new reader for output_series_0.ome.tiff
>> >> >> >
>> >> >> > log4j:WARN No appenders could be found for logger
>> >> >> > (loci.common.services.ServiceFactory).
>> >> >> > log4j:WARN Please initialize the log4j system properly.
>> >> >> > log4j:WARN See
>> >> >> > http://logging.apache.org/log4j/1.2/faq.html#noconfig
>> >> >> > for
>> >> >> > more info.
>> >> >> > Exception in thread "main" loci.formats.FormatException: Unmatched
>> >> >> > UUID:
>> >> >> > urn:uuid:b72fe97b-02c9-4029-a876-ca2bae46cbac
>> >> >> >     at
>> >> >> > loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:456)
>> >> >> >
>> >> >> >     at loci.formats.FormatReader.setId(FormatReader.java:1244)
>> >> >> >     at loci.formats.ImageReader.setId(ImageReader.java:727)
>> >> >> >     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
>> >> >> >     at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
>> >> >> >     at
>> >> >> >
>> >> >> >
>> >> >> > loci.scifio.itk.SCIFIOITKBridge.createReader(SCIFIOITKBridge.java:617)
>> >> >> >     at
>> >> >> >
>> >> >> >
>> >> >> > loci.scifio.itk.SCIFIOITKBridge.readImageInfo(SCIFIOITKBridge.java:268)
>> >> >> >     at
>> >> >> >
>> >> >> >
>> >> >> > loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:133)
>> >> >> >
>> >> >> >     at
>> >> >> >
>> >> >> >
>> >> >> > loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:108)
>> >> >> >     at
>> >> >> >
>> >> >> > loci.scifio.itk.SCIFIOITKBridge.waitForInput(SCIFIOITKBridge.java:83)
>> >> >> >     at
>> >> >> >
>> >> >> >
>> >> >> > loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:163)
>> >> >> >     at
>> >> >> > loci.scifio.itk.SCIFIOITKBridge.main(SCIFIOITKBridge.java:688)
>> >> >> >
>> >> >> > I also get this exception when importing with Bio-Formats into
>> >> >> > Fiji.
>> >> >> >
>> >> >> > As a side note, there are flags for itkSCIFIOImageIOTest that, in
>> >> >> > theory,
>> >> >> > should help the out of memory side:
>> >> >> >
>> >> >> > "-s <n1, n2>" should limit the series read.
>> >> >> > "-v <n>" should divide the data into divisions, which might
>> >> >> > mitigate
>> >> >> > the
>> >> >> > risk of OutOfMemoryErrors.
>> >> >> >
>> >> >> > But neither of these will help the UUID exception. Also, I still
>> >> >> > got
>> >> >> > OutOfMemoryErrors with these, so it was still trying to initialize
>> >> >> > using
>> >> >> > all
>> >> >> > the series. So I created a new issue in the SCIFIO-ImageIO
>> >> >> > tracker.
>> >> >> >
>> >> >> > Anyway, try importing the bfconvert output into Fiji with
>> >> >> > Bio-Formats
>> >> >> > and
>> >> >> > let me know if you don't get the UUID exception. If you do, then I
>> >> >> > think
>> >> >> > the
>> >> >> > bfconvert output is just invalid..
>> >> >> >
>> >> >> > Regards,
>> >> >> > Mark
>> >> >> >
>> >> >> >
>> >> >> > On Thu, May 29, 2014 at 12:17 PM, Michael Meuli
>> >> >> > <michael.meuli at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Hi Mark,
>> >> >> >>
>> >> >> >> Thanks for the quick response. I can see I was using
>> >> >> >> itkSCIFIOImageIOTest the wrong way and it doesn't make sense to
>> >> >> >> use
>> >> >> >> it
>> >> >> >> like this.
>> >> >> >> If I use the -w flag I get an error in all cases but at the
>> >> >> >> moment I
>> >> >> >> don't care as my aim now is to read ome.tiff files with itk.
>> >> >> >> I've written a small program (mainly by copy paste from examples
>> >> >> >> /
>> >> >> >> https://github.com/michaelmeuli/bactelize) which so far reads the
>> >> >> >> multi-channel-z-series.ome.tif file, does a maximum intensity
>> >> >> >> projection of one channel and writes a 2D image and it seems to
>> >> >> >> work
>> >> >> >> with the multi-channel-z-series.ome.tif but not with the files
>> >> >> >> converted with bfconvert.
>> >> >> >>
>> >> >> >> Expressed with itkSCIFIOImageIOTest:
>> >> >> >> "./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> /path/to/multi-channel-z-series.ome.tif
>> >> >> >> /path/to/scifio-test.ome.tiff"
>> >> >> >> doesn't give an error and seems to produce a tiff file of the
>> >> >> >> first
>> >> >> >> z-plane, but
>> >> >> >> "./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> /path/to/output_series_0.ome.tiff /path/to/scifio-test.ome.tiff"
>> >> >> >> gives
>> >> >> >> an error:
>> >> >> >>
>> >> >> >> reader->GetUseStreaming(): 1
>> >> >> >> done checking streaming usage
>> >> >> >> ITK test driver caught an ITK exception:
>> >> >> >>
>> >> >> >> itk::ExceptionObject (0x2f87680)
>> >> >> >> Location: "unknown"
>> >> >> >> File:
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> /home/michael/Colocalization/ITK-Release-build/Modules/Remote/SCIFIO/src/itkSCIFIOImageIO.cxx
>> >> >> >> Line: 180
>> >> >> >> Description: itk::ERROR: SCIFIOImageIO(0x2f78970): SCIFIOImageIO
>> >> >> >> exited abnormally. Creating new reader for
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> /home/michael/bioimage/ome/data/wt-dead-A-ome-tiff/output_series_0.ome.tiff
>> >> >> >> log4j:WARN No appenders could be found for logger
>> >> >> >> (loci.common.services.ServiceFactory).
>> >> >> >> log4j:WARN Please initialize the log4j system properly.
>> >> >> >> log4j:WARN See
>> >> >> >> http://logging.apache.org/log4j/1.2/faq.html#noconfig
>> >> >> >> for more info.
>> >> >> >> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>> >> >> >> space
>> >> >> >> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
>> >> >> >> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.common.NIOByteBufferProvider.allocateDirect(NIOByteBufferProvider.java:132)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.common.NIOByteBufferProvider.allocate(NIOByteBufferProvider.java:118)
>> >> >> >> at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:532)
>> >> >> >> at loci.common.NIOFileHandle.seek(NIOFileHandle.java:254)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:140)
>> >> >> >> at
>> >> >> >> loci.formats.tiff.TiffParser.getIFDOffsets(TiffParser.java:308)
>> >> >> >> at loci.formats.tiff.TiffParser.getIFDs(TiffParser.java:226)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.formats.in.MinimalTiffReader.initFile(MinimalTiffReader.java:426)
>> >> >> >> at loci.formats.FormatReader.setId(FormatReader.java:1244)
>> >> >> >> at loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:704)
>> >> >> >> at loci.formats.FormatReader.setId(FormatReader.java:1244)
>> >> >> >> at loci.formats.ImageReader.setId(ImageReader.java:727)
>> >> >> >> at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
>> >> >> >> at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.scifio.itk.SCIFIOITKBridge.createReader(SCIFIOITKBridge.java:622)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.scifio.itk.SCIFIOITKBridge.readImageInfo(SCIFIOITKBridge.java:266)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:131)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:108)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.scifio.itk.SCIFIOITKBridge.waitForInput(SCIFIOITKBridge.java:83)
>> >> >> >> at
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:161)
>> >> >> >> at loci.scifio.itk.SCIFIOITKBridge.main(SCIFIOITKBridge.java:693)
>> >> >> >>
>> >> >> >> Best regards
>> >> >> >> Michael Meuli
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> On 29 May 2014 17:41, Mark Hiner <hiner at wisc.edu> wrote:
>> >> >> >> > Hi Michael,
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >> The problem is that ./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> >> /path/to/output_series_0.ome.tiff /path/to/scifiotest.ome.tiff
>> >> >> >> >> (and my little program) throws an exception.
>> >> >> >> > ...
>> >> >> >> >
>> >> >> >> >> How do I use bfconvert the correct way for my case?
>> >> >> >> >
>> >> >> >> > You used bfconvert correctly to convert your dataset. Just so
>> >> >> >> > it's
>> >> >> >> > clear, if
>> >> >> >> > your goal was to produce an OME-TIFF:
>> >> >> >> >
>> >> >> >> >> /SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> >> /path/to/output_series_0.ome.tiff
>> >> >> >> >> /path/to/scifiotest.ome.tiff
>> >> >> >> >
>> >> >> >> > would be unnecessary, as you already have the OME-TIFF output
>> >> >> >> > from
>> >> >> >> > bfconvert. The itkSCIFIOImageIOTest is an analogous tool to
>> >> >> >> > bfconvert
>> >> >> >> > written in the Bio-Formats C++ framework. (although comparing
>> >> >> >> > the
>> >> >> >> > outputs of
>> >> >> >> > the itkSCIFIOImageIOTest conversion would be a good way to
>> >> >> >> > check
>> >> >> >> > for
>> >> >> >> > errors
>> >> >> >> > in the conversion process)
>> >> >> >> >
>> >> >> >> > That said, I believe your conversion failed because the output
>> >> >> >> > was
>> >> >> >> > picked up
>> >> >> >> > by the ITK TIFF ImageIO, which I believe only supports 3D data.
>> >> >> >> > The
>> >> >> >> > itkSCIFIOImageIOTest utility does basically no automation - you
>> >> >> >> > have
>> >> >> >> > to
>> >> >> >> > tell
>> >> >> >> > it exactly how to run, and because the SCIFIOImageIO is an
>> >> >> >> > external
>> >> >> >> > module
>> >> >> >> > it is not prioritized by the ITK format detection framework
>> >> >> >> > over
>> >> >> >> > internal
>> >> >> >> > modules (I'm not sure it's part of the automatic detection
>> >> >> >> > framework
>> >> >> >> > at
>> >> >> >> > all,
>> >> >> >> > actually). So you need to force the use of the SCIFIOImageIO
>> >> >> >> > for
>> >> >> >> > writing.
>> >> >> >> >
>> >> >> >> > If you run:
>> >> >> >> >
>> >> >> >> >  /SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> >
>> >> >> >> > with no arguments from the command line, it will print the
>> >> >> >> > usage +
>> >> >> >> > available
>> >> >> >> > flags. The flag you wanted here is "-w"  to write with the
>> >> >> >> > SCIFIOImageIO
>> >> >> >> > (which will use Bio-Formats, and thus the OME-TIFF writer).
>> >> >> >> >
>> >> >> >> > So the final syntax is:
>> >> >> >> >
>> >> >> >> > /SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> > /path/to/output_series_0.ome.tiff
>> >> >> >> > /path/to/scifiotest.ome.tiff -w
>> >> >> >> >
>> >> >> >> > which worked for me locally.
>> >> >> >> >
>> >> >> >> > Let me know if you have any other questions or issues!
>> >> >> >> >
>> >> >> >> > Regards,
>> >> >> >> > Mark
>> >> >> >> >
>> >> >> >> > On Thu, May 29, 2014 at 4:40 AM, Michael Meuli
>> >> >> >> > <michael.meuli at gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> ./SCIFIOTestDriver itkSCIFIOImageIOTest
>> >> >> >> >> /path/to/output_series_0.ome.tiff /path/to/scifiotest.ome.tiff
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>