Stitching image fails in ideal conditions (laboratory)

[Володимир Гренюх]

Good day to all!

I write small project for biological lab, what captures image from microscope camera, and could stitch its in big panorama image

But there is certain my incompetence:

• I tried stitch images one by one using code from https://boofcv.org/index.php?title=Example_Image_Stitching -but one stitch cost 300-400 ms at one photo at stage of detecting points only  - without modifying images 
• After that I tried use code from https://boofcv.org/index.php?title=Example_Video_Mosaic because it show very good performance and quality - but at some moments (early or no) stitcher get fall as you can see in screenshots

All images is captured from flat surface, optical distortion and noise minimized

I would be extremely grateful for the help

Of course, in case of success, the project will be presented in open access

[trembla...@gmail.com]

Hi,

In contrary to conventional mosaic where images are taken following the rotation of the camera, microscope data is produced by translation.
The algorithm used in both examples you mentioned is an homography. Peter could correct me on that but, a transform more suited for pure tile indexing would be either be a simple Affine
or even better a PhaseCorrelation. The latter being more resilient to noise. Tough, I don't think it is currently implemented in boofcv as of yet but Affine is.

As I understand it, basic 2D microscope imagery stitching requires the data to be

1. images indexed at a constant interval forming a grid
   
2. a single plane, in other words, avoid zoom change between image capture
   
3. a significant overlap to detect relevant features

Maybe Peter can help show the basics to implement an Affine2D_F64 in regards to 2D data registration? 

I've looked at the code and I myself am intrest in such topic.

I think Boofcv could probably do what you are asking of it, but there might be a simpler solution for your problem.

Have a look at Fiji. It contains plugins especially suited for stitching large data-set up to Terabyte level. You can find the source code of each individual plugin on github.
The relevant part you are looking for are located in the Plugins menu, take a look at:

Fiji>Plugins>Stitching>Grid/CollectionStitching | PairwiseStitching

However, the main drawback from using those specific Fiji plugin is that they are dependent on java.AWT classes.

I think it would be cool and useful to have an example about this kind of data being process in Boofcv!

cheers,



Antoni

[Володимир Гренюх]

Hi, Antoni

I`m very glad for you for your answer

Yes, I have been looked for Fiji, but in future captured panorama will be segmented into pieces and sended into small neuronal network.

I try did all porcess automatically, because I dont belive that my student can perform all cell examples manually without loss quality of data

Unforunately, from  basic 2D microscope imagery stitching I must remove 1 position - "images indexed at a constant interval forming a grid" - we have microscope without auto-moving of samples

 I will try implement PhaseCorrelation in-hand, but as I am biologist and not very experieced programmer, that cost veeery big amount time

Wil be hope for best result - it reduce hundred hours from eye-breaking

[Antoni]

I see no problem feeding your captured images into a neuro-net after visualization is complete. Using Fiji, once your stitching is finished, you have the option to output a .txt file containing the relevant position of each tile, and afterward is you want, do operations on the data.

The problem in your case when you are registering data on the fly with Example_Video_Mosaic, is that you  still  must ensure the student move the sample accurately enough to keep the process going. If the sample is jerked out of the field of view of the camera, or moved too quickly, it will lose its reference and the process will crash. Boofcv does provide a way to recover, with object tracking algorithm, but I am not very familiar with it and don't know if compatible with low quality or distorted images.

The microscope does not have to be moved automatically to provide a suitable dataset to stitch in a grid format. The highly optimised algorithms contained inside ImageJ does a very good job at compensating for uneven overlap. But is has to have a minimum. You could start at about 20% overlap and increase if needed.

In case your supply or budget is limited, you could always borrow some sort of dial indicator to index your microscope slide. You would still use your camera setup but only take a snapshot when the slide is indexed at the correct position. You would name the snapshot accordingly to its position on the grid. In the worst case, if all of it has to be done manually, it would be very accurate and provide the students with an easy to increment and repeatable visual reference.

In my honest opinion, if you say you have limited experience in programming, your best bet would be to structure you data in a grid fashion and use an already highly optimized and working software.

However, if you still wish to implement a phase-correlation algorithm into Boofcv, you might want to have a look at this Github repo and get a grip of the process with an already working implementation.


cheers,

-Antoni

[Володимир Гренюх]

Hi again!
In weekend I tried stitch some sample images catched from lab system - using Fiji - and Stitching plugins fails

However I stiil hope did that stuff. Ideally I want create something like Microvisioneer

https://www.youtube.com/watch?v=HnQRaYGnXJs

Even this stage will be perfect.

About overlap - in most cases images has overlap 50% or more. 

Program automatically record 20-30 images per second, so speed of moving may not fails overall process to.

[Peter A]

This is actually not a trivial case for the techniques available in BoofCV. Point feature based algorithm will have problems since there are only a few point features, e.g. corners. The few point features in these images are also not distributed evenly, which is what you need for good image stitching.  With a bit of hand tuning you could problem improve the results but it would not be perfect.

Antoni is right that you shouldn't use a homography. If the motion is really just translation you can use a translation model, but I don't think there's one built into BoofCV but I have used them in the past. Affine would be better and more stable than a homography. A long time ago I played with a model that was just translation and scale which might be appropriate here.

If someone has a sample video I'll take a quick shot at it. 

The other thing is that what's in BoofCV is really the simplest stitching/mosaic algorithm out there. If someone know of a paper that describes a better approach that is known to work well I'll take a look at it. Not saying that I'll look at it anytime soon though...

Cheers,

- Peter

--
You received this message because you are subscribed to the Google Groups "BoofCV" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

"Now, now my good man, this is no time for making enemies."    — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

[Володимир Гренюх]

Peter and Toni - thank you both for the answers!

Many attempts have discovered a possible solution to the problem (spoiler: a redesigned way of feeding staff and slow motion of the prototype), but there is a somewhat worse problem: if the frames make a circle, then the final frames do not collide with the first.

As I understand, the last frame is synchronized with the precedence, and not with the whole image that is in front of it? Perhaps the solution to this problem may be this

[Володимир Гренюх]

If needs, I can make example video 

[Michal Spišiak]

Hello, could you please share an example video? I want to try develop what you mentioned, but I don't have any data to test on.

[Володимир Гренюх]

Unfortunately, I abandoned this project years ago

середа, 14 березня 2018 р. 16:25:18 UTC+2 користувач Володимир Гренюх написав:

[Peter A]

Did you try any other software that worked?

--
You received this message because you are subscribed to the Google Groups "BoofCV" group.

To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/7fc76b15-b9b7-4720-b317-45fe78d5feceo%40googlegroups.com.