Assisted depth from focus using a USB microscope attached to a 3D printer for motorized Z control

[Ahron Wayne]

So a while back I strapped a USB microscope onto a 3D printer and starting using it to scan around taking scanned (stacked and sttiched) pictures. That's actually not the whole story but it leads us to where we are now: 3D printer, USB microscope, trying to scan things. I've meddled in up to 5 axis for rotating scans but right now am focusing on what you can do with just 3 axis. I feel like this should be a big distributed project since there is absolutely a lot of functionality you can get out of it and it just works with normal GCode controls/USB connection. 

One thing I would like to be able to do is generate some kind of 3D visualization and extract 3D data from my scans. There are lots of ways I have/want to try, including stereographic vision with multiple cameras at once and plain old rotation, but right now I am interesting in depth from focus. I saw an early video on this with BoofCV and I feel like the results should be improved if you know where your camera actually is, and you take lots of pictures, not just one! 

Here is some data I took for a real life example and tilted callibration slide. I also have lots of scans incorporating 2D movement control too, if anyone is interested. My goal is to be able to set a low magnification scan and use it to roughly figure out where everything is so that you can selectively target it for a higher res scan more easily. At low resolution the depth of focus is on the order of a few millimeters, and at highest res down to 0.1, but I feel like you could improve on at least the latter value since the motion control is finer than this and you can watch it sweep through the whole object being in and out of focus. As it is the biggest drain on scans is either human labor to define the scan parameters or sheer time to brute force the scan as a cuboid, and then you also run the risk of crashing. 

How hard/impossible is this task? I have only used regular OpenCV, and even then really nothing intense. I can already see potential issues like the object size changing as you move your camera closer, but I really appreciate any advice or attempts at a first pass on the problem. Thank you! 

[Ahron Wayne]

Whoops, I mean you can improve on the former depth of focus resolution (at low magnification). My mistake. 

[Jairo Rotava]

Hello Ahron,

Nice project. I was planning a similar project, but the goal was to use these microscope cameras to take pictures of circuit boards with small smd components, stitch the images and get the full board image with enough resolution to identify the small components and tracks.  Do you think you can take some images of a circuit board (both sides) and share so I can start working on it? I know this is not exactly what you are asking for, but it is another application where you can use your setup. Please let me know if you are interested.

Regards

Jairo

--
You received this message because you are subscribed to the Google Groups "BoofCV" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/6e1c2d30-6849-4223-be39-944738cfcf65o%40googlegroups.com.

[Ahron Wayne]

Hey Jairo, glad to hear someone else is interested. I have done some things similar to that already, I think I can help you out. I think this segment from one of my videos is a good example of what you are talking about, where you can get a lower res image of the whole PCB (still a decent number of real pixels) and then can get a zoom showing a bit of it: https://www.youtube.com/embed/DP-Whj9F3KA?start=76&end=84&version=3

This hackaday post describes taking those two picture sets, and I just uploaded the original data in forms before and after stacking + stitching. The names of the files are X/Y/Z + #####, for example X05010Y10000Z00050 would mean that the image was taken at X = 50.10 mm, Y = 100.00 mm, Z = 0.50 mm. You can ignore the "R" which is technically in the same way with extrusion acting as a linear distance since none of these have been rotated. 

Ideally you would be able to do the stacking and stitching without external software or at least programmatically so you can do it all from the central scanning hub, but these were were stacked with helicon focus (I use picolay now) and stitched with microsoft ICE. Keep in mind that whenever you do stitching you are going to corrupt some information; ICE normally does an excellent job at for instance keeping the tracks from looking broken but it also introduces warp and color artifacts that depend on subtle variation in the component images (and is worse with the cheaper USB microscopes). These can be theoretically be fixed ahead of time by characterizing your system, however, and if your goal is just to get an accurate mosaic and don't mind if there are breaks around the stitching points then you should be able to just concatenate the images together without doing any kind of feature matching.

You might also be interested in this video where I show a basic proof of concept of visiting the interesting PCB points directly, by parsing the "centroid" pick and place file from easyeda for my own board: https://www.youtube.com/watch?v=zvpZBqr5DOI&feature=youtu.be

To unsubscribe from this group and stop receiving emails from it, send an email to boo...@googlegroups.com.

[Jairo Rotava]

Hi Ahron,

Thanks for the images, this is exactly what I was looking for. This is very good data. I'm planning to stitch them programmatically, using opencv or boofcv. I've used another tool for stitching: autostitch (http://matthewalunbrown.com/autostitch/autostitch.html). I noticed the images warping, much worse than what you got. Most stitching algorithms assume far field images, and that the camera is rotating, not translating. I guess this may be the source of some problems when stitching these microscope images (although your resulting image is pretty good in my opinion).

This is a side project, may take a while, but as soon I get something working I'll share it with you.

Regards

Jairo

To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/41e37334-40fc-4f81-94ff-ae83a4884d0fo%40googlegroups.com.

[Peter A]

Should be able to do something with these images. The depth from focus would probably be improved if it incorporated a model of the lens and took advantage that the focal length is fixed and  the distance known. However that would probably be a little bit more of a project than I want to do right now. I do think the existing software might produce something interesting and I'll give it a shot.

Stitching in BoofCV might get an update this year. There's some work for multi-view reconstruction which could be directly applied to image stitching.

To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/41e37334-40fc-4f81-94ff-ae83a4884d0fo%40googlegroups.com.

--

"Now, now my good man, this is no time for making enemies."    — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

[Ahron Wayne]

Hi Jairo, Peter, sorry for the slow reply. My hard drive actually failed just after these posts and I haven't done any scanning. Actually, if I hadn't uploaded the example data, it would have been lost, so that was a good thing!

@Jairo, I''m glad I can supply something helpful to your project. I have also experienced the warping you are talking about with Autostitch and other programs which assume a stationary but rotating camera. It is still kind of odd to me that there are limited tools to do mosaic stitching with a regular ordered grid, since the problem is much simpler than when trying to make true panoramas. Even stitching blind without blending would be welcome if it was programmatic, especially if it was combined with some easy way to calibrate the size of the image/pixel/displacement. You should know though that while the warping is caused by the software making assumptions about camera angle, it is also highly dependent on how flat/warped the individual component images are. The images I gave you were taken with a high end dinolite edge camera, as opposed to a cheap generic one, and there is a marked difference in warping no matter which software you use. For all of them though I think it should be possible to make improvements by calibrating your source images first.

@peter, I also appreciate any attempt at the depth from focus problem with BoofCV. Apart from being a bit of a project, do you think my imagined pipeline of depth from focus using one focal length/magnification, which provides the scan locations for a higher magnification scan, is at least theoretically feasible/robust? I still haven't made a serious attempt at it in either Boof or OpenCV, but it would be nice to get an idea of whether it's an idea worth pursuing over some other option I haven't thought of yet. I did found a depth map to gcode converter for CNC milling which feels like a bit part of the problem.

Thanks, And best regards,   

Ahron

To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/41e37334-40fc-4f81-94ff-ae83a4884d0fo%40googlegroups.com.

[Jairo Rotava]

Hi Ahron,

I'm sorry to hear you had issues with your hard drive. I hope you were able to recover the most important things.

Thanks for all the information, it will help!

Regards

Jairo

To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/65f53d01-7dae-441d-a92c-c27d90a64dd9o%40googlegroups.com.