Load virtual stack

[Ricard Delgado Gonzalo]

Hi,

I was wondering if there is a way to open a large dataset without loading it on memory. Like the "virtual stack" in ImageJ.

Thanks

Ricard 

[Stephane]

Hi Ricard,

"Programmatically" yes but not (yet) from the GUI. I guess you are looking for the second case unfortunately.

[Ricard Delgado Gonzalo]

Indeed. I have a user with a very large time-lapse sequence, and he cannot load it all in memory.

[Stephane]

Of course this is a planned feature in Icy, I can't give any date though :-/

[Ricard Delgado Gonzalo]

Good. I'll make my guy wait for the feature :)

[Ricard Delgado Gonzalo]

On ImageJ I used to load virtual stacks with File>Import>Image sequence. However, the virtual stack option in the ImageJ that you have within Icy does not support that feature either, right?

[Curtis Rueden]

Hi guys,

FWIW, ImageJ2 has much improved support for virtual stacks. Using the SCIFIO library (which uses ImgLib2 data structures), you can now open really big multidimensional data which is paged in and out of memory on demand, including writing altered pixels to a disk cache so changes are transparently preserved.

For details, see: http://developer.imagej.net/2013/06/12/imagej-v200-beta-7#Virtual_stacks

It would be awesome if in the future, Icy supported ImgLib2 data structures so that it could transparently work with huge data using SCIFIO.

Regards,

Curtis

--
You received this message because you are subscribed to the Google Groups "Icy imaging" group.
To unsubscribe from this group and stop receiving emails from it, send an email to icy-software...@googlegroups.com.
To post to this group, send email to icy-so...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/icy-software/9e7eefde-6223-4d5f-a636-0f5ad63ac8f1%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[Stephane]

Hi,

Ricard>
Very kind from you to wait for it but honestly i am not sure he will be that patient ;) Still i am really surprised the embedded ImageJ do not support it, there is absolutely no reasons for that.
Actually i just tested here and it does work, i just did File --> Import --> TIFF Virtual Stack... and my 1 Gb data file only eat up 60 Mb in the memory monitor, if i open it normally it indeed consume 1 GB of memory.

Curtis>
Indeed ImgLib2 is really interesting and powerful. Good to hear it also supports streamed writes now (it was not the case before i believe). We want to support it in future as well but as we actually do with ImageJ : transform our internal data format to ImgLib2 format and vice versa. We just cannot directly replace our internal data structure or we will instantanously lost backward compatibility with almost every plugins, also something that worry us a bit about ImgLib2 is the strong Generic based conception which is indeed elegant and powerful but which may confuse beginner developers.

[Ricard Delgado Gonzalo]

Curtis>

Hi Curtis, I already know about ImgLib2. It's a very fine library, but my goal in this project is a different one and I'd rather not spend time managing the access to the images but focusing on the algorithm that performs the analysis. The datasets I'm talking about are just time-lapse sequences. Thus, each image is rather small, but there are many of them.

Stephane>

Indeed, the Import>TIFF Virtual Stack works fine. However, I have my dataset structured as a series of images (circa 10GB) and not as a single TIFF file. The command on ImageJ is Import>Image Sequence

Ricard

[Stephane]

Hi Ricard,

Oh i understand, indeed ImageJ requires to load the virtual as a single TIFF file.
Doing Virtual Stack from a serie of image file would need a bit of work as you have to preserve the file list somewhere and affect a Z,T position to each file. I have some ideas about how deal with that in Icy but as i already told, that is just a planned feature.

- Stephane

[xavier heiligenstein]

Dear all,

I'm bringing back up this topic as I need to load a 38Gb dataset to process it and I don't know how to do it. I have Icy 1.6.1.1

Is there now a way to do that easily ?
Thanks for your help

Xavier

[Frédéric Marion-Poll]

I am currently working on this and wondering what would be the best approach. Programmatically, you can call ij.FolderOpener which allows you to open a virtual stack from a directory or from an AVI. Stephane has been working on a routine to transfer IJ images towards IcyImages which works well. Using this approach, you can write a program that load images one by one. I am able to read and analyze a sequence of 9887 images (jpg of 1280 x 720 pixels) with no problem related to this reading.

However, I presume your question is not "is it possible" but rather if is it integrated into Icy. The answer seems to be "not yet"... I think it is possible but someone has to work on it seriously to make a class that works and which can be integrated in a final package or in a open plugin.

fred