[micro-manager-general] ImageJ integration
Carlos G Mendioroz
as I'm trying to get a realtime (i.e. live) graph of pinhole
intensities, I "discovered" that ImageJ seems not aware that
micro-manager (uM?) has a stack of images available and is only aware of
the displayed ones.
(I'm not a uM expert, not even a user, neither an ImageJ one, so I may
be quite wrong)
ImageJ has some tools that work on stacks or hyperstacks that do not
recognize uM data as such (i.e. ImageJ sees the images as 2D data).
I've seen many tools that bridge this gap, but all of them seem to work
on static data (i.e. done experiments) but I would like to use them on
live ones.
Has anybody dealt with this already ? Is there a uM datastore -> ImageJ
hyperstack bridge of sorts ? I do not want to reinvent the wheel...
Thanks in advance,
--
Carlos G Mendioroz <tr...@acm.org>
_______________________________________________
micro-manager-general mailing list
micro-mana...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Carlos G Mendioroz
> Hi Carlos,
>
> Sorry for the delay, slipped through the cracks..
No worries, thanks for you time.
>
>> as I'm trying to get a realtime (i.e. live) graph of pinhole
>> intensities,
>
> It would be useful to understand a bit better what your end "product"
> would be. My guess is that you want to be able to start a live stream,
> have code that automatically finds pinholes (bright spots in the image),
> and produces data (and/or graphs) with intensities of these pinholes in
> all subsequent images. Is that about right?
Hmm, some.
Forgive my use of terms; as I said, I'm helping others here.
They use pinhole to describe points of interest, but after reading a
lot, I guess that is not the original meaning.
So the use case is realtime multiple ROI intensity tracking.
The ROIs will me manually setup on a live stream, and then they want to
have a multi plot of corresponging intensities, updating as the
aquisition goes. They want to use that as feedback for changing some
experiment variables.
>
>> I "discovered" that ImageJ seems not aware that
>> micro-manager (uM?) has a stack of images available and is only aware of
>> the displayed ones.
>
> Correct. You do not want to access Micro-Manager images through the
> ImageJ interface. When we started out we very much wanted to integrate
> MM with ImageJ as much as possible, but quickly ran into all kinds of
> limitations of ImageJ (don't forget that ImageJ was never designed for
> its current use, it grew beyond Wayne Rasband's wildest imagination).
> In the last refactoring of the MMcode, the interaction with ImageJ was
> restricted to the part where images are displayed, so ImageJ has very
> limited knowledge of the data.
>
>> ImageJ has some tools that work on stacks or hyperstacks that do not
>> recognize uM data as such (i.e. ImageJ sees the images as 2D data).
>
> What tools do you want to use that work on stacks?
There you have ROI manager, multiple ROI measurements, plots (even
"live" that e.g. change as you change a line of interest). And ImageJ
native plugins.
>> I've seen many tools that bridge this gap, but all of them seem to work
>> on static data (i.e. done experiments) but I would like to use them on
>> live ones.
>>
>> Has anybody dealt with this already ? Is there a uM datastore -> ImageJ
>> hyperstack bridge of sorts ? I do not want to reinvent the wheel...
>
> Use the MM api to access image data (i.e., see:
> https://micro-manager.org/wiki/Version_2.0_API). Once you have an MM
> Image, get an ImageJ ImageProcessor using the mm.data().ij()
> ImageJConverter
> (https://valelab4.ucsf.edu/~MM/doc-2.0.0-alpha/mmstudio/org/micromanager/data/ImageJConverter.html).
> Inserts these into an ImageJ stack anyway you want. MM Datastores do
> not fit 1:1 to ImageJ stacks or HyperStacks, although you could write
> convenience functions if you so desire.
The issue that I find is that the ImageJ GUI is not aware of the stack,
or at least I have not been able to do it in a proper way.
I have been able to get the ImagePlus processor, from which I can create
a stack and dynamically add uM frames, but the only way to make ImageJ
"aware" of it seems replacing the Display image. If I do that, ImageJ
sees the stack and the tools work, but I'm breaking many assumptions by
the code, I think.
If there is no other way to have a realtime multi ROI graph of
measurements... I guess that is the way it's going to be.
>
> Best,
>
>
> Nico
Thanks,
-Carlos
Carlos G Mendioroz
Nico Stuurman @ 29/03/2020 16:35 -0300 dixit:
> Hi Carlos,
>
> On 3/29/2020 10:42 AM, Carlos G Mendioroz wrote:
>> So the use case is realtime multiple ROI intensity tracking.
>> The ROIs will me manually setup on a live stream, and then they want to
>> have a multi plot of corresponging intensities, updating as the
>> aquisition goes. They want to use that as feedback for changing some
>> experiment variables.
>
> Is this a live stream of a single channel or multiple channels? Asking
> because you seem to really want a stack and I don't understand why.
> Also, I am pretty sure that others have developed similar plugins,
> although none were ever contributed back to Micro-Manager.
Hmm, let's define my understanding:
Image: 2D (XY)
Stack (ImageJ): series of images (3D) where the third dimension could be
Z, c or t.
Hyperstack: 4 or 5 dimension data.
So as far as I get, for ImageJ, a stack is just a series of images along
some axis (Z, c, t). If this is not so, I'm sure confusing you. Sorry.
>>> What tools do you want to use that work on stacks?
>> There you have ROI manager, multiple ROI measurements, plots (even
>> "live" that e.g. change as you change a line of interest). And ImageJ
>> native plugins.
>
> ROI manager works fine within Micro-Manager, even the ImageJ live Plot
> Profile works in Micro-Manager live mode, so if you want to stay
> completely on the ImageJ side of things, I would look at that code and
> modify to your purpose.
I don't want to stay anywhere :) I just want a usable option.
Are you saying that there is a way for stock Micro-Manager to make a
realtime t-axis graph of ROI intensities ? Great! Care to point me how
to do that ?
> However, if I would work on this myself, I would build a Micro-Manager
> plugin that uses the MM api. The advantage is that it will work with
> "live" multi-channel images and could be extended to do things like live
> ratio-tracing of multiple ROIs (useful for calcium imaging and similar
> applications).
I'm open to do it in the best way you see fit, given that you sure have
a better grasp of what could be useful even for the community.
>
>> The issue that I find is that the ImageJ GUI is not aware of the stack,
>> or at least I have not been able to do it in a proper way.
>
> In live imaging, be it ImageJ or MM, there is no stack, there is the
> current image that is continuously being replaced by the next one. You
> want to measure each image, store those measurements, and possibly
> display them live while the acquisition is ongoing.
Well, at least we are in synch: ImageJ is not aware of more than 2
dimensions. Thus any transversal analysis has to be done outside ImageJ.
>
> Best,
>
>
> Nico
Carlos G Mendioroz
> Hi Carlos,
>
> On 3/29/2020 12:50 PM, Carlos G Mendioroz wrote:
>
>> I don't want to stay anywhere :) I just want a usable option.
>> Are you saying that there is a way for stock Micro-Manager to make a
>> realtime t-axis graph of ROI intensities ? Great! Care to point me how
>> to do that ?
>
> different approaches to writing code.
>
> I would:
> - write a Micro-Manager plugin
> (https://micro-manager.org/wiki/Version_2.0_Plugins)
> - Subscribe to the LiveModeEvent
> (https://micro-manager.org/wiki/Version_2.0_API_Events)
> - When live mode starts, use the ImageJ ROIManager to query which ROIs
> are present on the image
> - Get the currently displayed Image, get an ImageProcessor, calculate
> mean intensity (and other statistics of so desired)
> - Push the data into a live graph created using JFreeChart.
>
> Not trivial at all, but I have heard people asking for things like this
> for a long time. Other ways are certainly possible.
>
>
Ok, at least I was not very confused :)
I will have to digest the plugin API, I don't mind sharing back the code
when it is done. Too bad we can not use simmilar functions that are
already there.
Given that I've found a loophole that seems to work (tricking ImageJ
into thinking that it has a stack) it might take time, but who knows ?
> Best,
>
> Nico
Thanks again,
-Carlos
Carlos G Mendioroz
> - write a Micro-Manager plugin
> (https://micro-manager.org/wiki/Version_2.0_Plugins)
> - Subscribe to the LiveModeEvent
> (https://micro-manager.org/wiki/Version_2.0_API_Events)
> - When live mode starts, use the ImageJ ROIManager to query which ROIs
> are present on the image
> - Get the currently displayed Image, get an ImageProcessor, calculate
> mean intensity (and other statistics of so desired)
> - Push the data into a live graph created using JFreeChart.
do live mode events get generated by MDA ?
Do you have a template plugin suggestion that I can use as a starting
point ?
I know Java and C++, but coming up with a code design ... I don't want
to reinvent the wheel.
TIA,
Carlos G Mendioroz
Nico Stuurman @ 30/03/2020 18:53 -0300 dixit:
> No, live mode events are generated when the user starts live mode. I
> thought the goal of the project was to track intensities of of ROIs
> during live mode? If you just want to do this at the end of an MDA,
> then thing are much easier.
Not at the end, during it.
Live mode does not get to do c/Z things, just visual t, right ?
The thing is realtime experiment feedback... but with all data being
taken, and MDA features available.
I guess MDA snaps do generate some event too, or it can be done if not.
Time to work now...
Carlos G Mendioroz
>> point ?
>
> (https://github.com/micro-manager/micro-manager/tree/master/plugins/Example/src/main/java/org/micromanager/plugins/example)
> is a good place to start. You may also want to look at the Tracker
> plugin
> (https://github.com/micro-manager/micro-manager/tree/master/plugins/Tracker/src/main/java/com/imaging100x/tracker),
> as it tries to do stuff with live data.
Stupid question:
how do I compile the plugin ?
I have already followed the instructions to have a build environment,
and Micro-Manager solution compiles (minus some references to 3rdparty
stuff and may be hidden/non redistributable code).
But in the solution explorer I don't see any of the plugins, including
the example one that is in the distro...
Now that I see, this is a C++ studio and those plugins are Java ...
Well, I said it was stupid.
Carlos G Mendioroz
> Did you have a look at:
> https://micro-manager.org/wiki/Writing_plugins_for_Micro-Manager ?
> Please note that those instructions are a bit outdated (as they were for
> 1.4), I'll put it on my todo list to update them for 2.0. In the mean
> time, decide which tools (Netbeans, Eclipse, Intellij, or plain old
> javac and text editor of your choice) you want to use, start setting up
> your build environment, and see where you get stuck;)
I had some long time ago, it seems :(
Ok, Eclipse it is, I'm not using the same machine that runs MM though.
Compiles fine, but somehow it is not getting recognized (after copying
to mmplugins, and restarting). I have JRE 1.7 with 1.6 compliance setting.
Any obvious gotchas ?
Carlos G Mendioroz
Nico Stuurman @ 01/04/2020 00:02 -0300 dixit:
> MM 2.0 is actually build with Java 8. It is possible that the
> annotations used for plugin discovery do not make it correctly in Java 6
> binaries (don't know, never tried), so I would definitely start
> switching to Java 8 (we use the binaries from AdoptOpenJDK these days).
>
> Look in the Corelog file in the Micro-Manager application directory. It
> will have lines like: 2020-03-31T15:31:22.562599 tid33064 [IFO,App]
> Found plugin org.micromanager.ratioimaging.RatioImaging@2a5dc8b. It may
> give clues to what the problem may be with your binary (or if it is not
> listed there, that is a clue too).
Updated to latest eclipse, switched to 1.8 compliance, and no news.
Corelog shows nothing, but also nothing about the SequenceBufferMonitor
plugin which is the one I'm using to compare (that works ok).