Advanced camera headcam calibration: Too many misdetects

[Roman Valls]

Hello list,

I'm having a very persistent problem with this "issues and solution" step and I'd like to share the outputs with you just in case I'm missing something because I ran out of ideas on what could be causing this "Too many misdetects" error :/

I've posted the TRACE log over here: http://pastebin.com/JD4jRjHF

Please ignore the 1980-01-07 timestamp, I've adjusted the clock and yields the same error.

[Roman Valls]

Oh, sorry, forgot to include the machine config, here it is:

https://github.com/CCHS-Melbourne/Pick-N-Place/commit/4db4d20e25c914ebff911c56fbf973fa7e5916fb

Any help/guidance appreciated :-S

[tonyl...@gmail.com]

Please attach the log file from the session where this occurred. Make sure you had your logging level set to Trace during the calibration. If not, set the logging level to trace, try it again, and then post the resulting log file.

It may be that you need a better light diffuser on your camera lights - I can see a very strong reflection of the LEDs in the image you posted.

Tony

[Roman Valls]

The TRACE log is in my first email, pastebin link..

--
You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/Yn_mgos8iPY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/e84e7752-3f24-476e-885f-8fe48c5d3decn%40googlegroups.com.

[mark maker]

Hi Roman,

I agree with Tony, lighting seems insufficient/not diffused.

Seeing in the second photo that the table is probably white, the yellow-orange tint suggests ambient light is dominating, which means the LED light is very weak. This in turn might cause long exposure/reduced FPS, which could also cause issues with unreliable camera settling (motion blur, rolling shutter).

https://makr.zone/camera-fps-cpu-load-and-lighting-exposure/519/

Plus I'd like to add that the LED reflection being so much off center would mean a huge camera tilt and/or non-planar table surface.

Plus, it is hard to see in the small scale screenshot, but your fiducial seems extremely out of focus and irregular:

According to the visible status text it is at the Z coordinate 1, i.e. the Primary Calibration Fiducial, which means it should be (very near) the camera focal plane, and the PCB surface height Z. The secondary fiducial is bound to be slightly out of focus (naturally), but must still be reasonably defined, i.e. Z difference should not be excessive.

https://github.com/openpnp/openpnp/wiki/Vision-Solutions#calibration-primary-fiducial

Please post a native camera image, according to this:

https://github.com/openpnp/openpnp/wiki/FAQ#how-can-i-get-a-native-camera-image

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/2f5b25c8-87c3-4cc0-be93-75272b891177n%40googlegroups.com.

[tonyl...@gmail.com]

I looked at the log file at the pastebin link, but it does not appear to have been collected with the logging level set to trace so I can't tell much from it.

Tony

[Roman Valls]

You are right Tony, my bad... I was confusing the TRACE *search filtering* with the TRACE *level* for some reason and in my mind I thought that the former was the one active.

Mark, thanks as always for the detailed diagnosis, we made some improvements in lighting, diffusion and intensity of the light and made sure the camera mount is as leveled and straight as possible, adjusted the Z height and also the focus (quite hard to adjust with the camera ring, btw)... so we'll keep all that advice in mind, but see this next paragraph:

I had some time to dedicate to this and realized that one of the "fundamental" tests, namely: "Calibrate precise camera <-> Nozzle N1 offsets" was not "solved" yet so I went to run it (the one involving bits of paper circle cuts). Unfortunately this test just froze after the nozzle contacts the bit of paper, here are the TRACE logs:

https://pastebin.com/40Epb14p

Where the last command is "get position":

2023-08-22 21:36:30.663 GcodeAsyncDriver DEBUG: [GcodeDriver:COM11] commandQueue offer >> M114 ; get position

And then fails with the timeout error message shown in the attachment :/

We were quite puzzled and don't know what could be causing this... we know that M8/M9 G-Codes are related to the solenoid, but it seems to be working just fine when actuated manually? As in picking a component, activating the pump, then solenoid, etc...

[tonyl...@gmail.com]

This probably isn't your only problem, and I'm not sure how this happened, but...

You have this set of lines in your machine.xml file that look correct for controlling the solenoid valve (sending the M8/M9 commands):

            <command head-mountable-id="ACT166ef1f3c8e8bea5" type="ACTUATE_BOOLEAN_COMMAND">
               <text><![CDATA[{True:M8}]]></text>
               <text><![CDATA[{False:M9}]]></text>

            </command>

However, you also have this set of lines in your machine.xml file that look incorrect:

            <command head-mountable-id="ACT166ef1f3c8e8bea5" type="ACTUATE_DOUBLE_COMMAND">
               <text><![CDATA[{1.000:M8}]]></text>
               <text><![CDATA[{0.000:M9}]]></text>
            </command>

 

I'd keep that first set and get rid of the second and see what happens.

Tony

[Roman Valls]

Thanks Tony, spot on, that was the issue, removing those lines fixed it:

https://github.com/CCHS-Melbourne/Pick-N-Place/commit/3d069b746627d5fc28c940c68c04f53796a64ba9#diff-3e0b25e2e1f16f74e14bc5ac8f4d330e3f12135e40d3b7cbb4dfdaa9dbd75d73L939-L953

I don't know why or how those definitions got in there in the first place, to be honest :/

[Roman Valls]

I think I **finally** found the culprit of this test (advanced camera headcam calibration) failing: soft limits!

Here are the backtraces that appear while the test is going:

https://pastebin.com/R2D18RGv

And here is some context around those exceptions, leading to the test stopping for "Exceeded maximum error count - terminating calibration sequence":

https://pastebin.com/u7VtrE2x

Given that soft limits seem to be fine as in: we are not getting any hard limit triggering while the machine is on operation or I jog around. Could it be possible to adjust this calibration sequence so that it respects the current soft limit "bounding box" of the machine and/or not count those soft limit triggerings as errors towards the test itself?

... just suggestions from a regular user of "Issues and solutions", since I'm not familiar with that code I might be totally off, so take those with a grain of salt :-S

Cheers!

Roman

[tonyl...@gmail.com]

You really need to move your calibration fiducial so that it is not near the edges of the machine.  See https://github.com/openpnp/openpnp/wiki/Vision-Solutions#calibration-primary-fiducial

Tony

[Roman Valls]

Thanks for the hint, Tony...

"Do not place the fiducial at the edge of the machine motion range."

Sure, fair enough, we don't have it at the edge, but we'll move it more inwards... STILL, the error count for the advanced camera calibration metrics shouldn't be tied to exceptions related to soft limits, no?

And even if it does, the final error message should point that most of the errors were due to violations of those soft limit bounds, to guide the user towards addressing the misplacement of the calibration jig?

[tonyl...@gmail.com]

> STILL, the error count for the advanced camera calibration metrics shouldn't be tied to exceptions related to soft limits, no?

They should still count as errors for the metrics because they are limiting which parts of the image that have calibration data collected. When the camera model is fit to the data, it will probably be wildly off in those areas where no data was collected. The end result would be a lot of distortion of the image in those areas - probably not what anyone would desire.

>And even if it does, the final error message should point that most of the errors were due to violations of those soft limit bounds, to guide the user towards addressing the misplacement of the calibration jig?

Agreed, but there are so many different possibilities as what could go wrong it might be difficult to handle all cases. Maybe the best solution is to modify the error message to tell the operator to examine the logs for clues.

Tony

[Roman Valls]

In any case, moving the primary fiducial more inwards did work, now the advanced camera calibration passed on the first go after moving it a bit further away from the edge!

Now the camera view has quite a massive crop (I guess that's to sieve the unacceptable camera aberrations?). Any suggestions on how to further correct those camera issues and avoid that cropping?

Thanks again!

[Roman Valls]

Inspecting my own images, I suspect that the camera mount drifted again slightly off-center (by looking at the partial ring of light spots not being centered on the cropped preview axis)... I'll make sure that's not the case, but other than that:

How do you folks physically make sure that the camera is as co-planar as possible with the bed? Using a bubble level attached to the camera? Some more sophisticated technique involving lasers or sth else? :P

[tonyl...@gmail.com]

I suspect your camera is rotated at a negative angle (in a right-handed sense) about the X-axis so that it is pointed slightly towards the front of the machine - check the rotation errors in the Diagnostics section for the camera.  And read https://github.com/openpnp/openpnp/wiki/Advance-Camera-Calibration---Camera-Mounting-Errors

Tony

[tonyl...@gmail.com]

> How do you folks physically make sure that the camera is as co-planar as possible with the bed?

That's part of what Advanced Camera Calibration measures.  I don't think you can really judge by any external measurements of the camera body because you don't know how well its internal optics and sensor are aligned with its body. Note that you don't have to have it exactly co-planar with the bed - you just need to be good enough that your field of view isn't too restricted. The calibration takes care of correcting any remaining tilt.

Tony

[Roman Valls]

We swapped the down looking camera assembly and aligned it as best as we could visually (on all rotation, yaw, pitch...), but the advanced camera calibration still goes a bit crazy, see attached screenshot and logs, here's a snippet that blew my mind:

(...)

2023-10-10 22:08:00.042 AdvancedCalibration TRACE: Physical camera rotational errors from ideal (as measured by the right hand rule about each machine axis):
2023-10-10 22:08:00.042 AdvancedCalibration TRACE: Z axis rotational error = 33.28594658705103 degrees
2023-10-10 22:08:00.042 AdvancedCalibration TRACE: Y axis rotational error = 31.59228190535026 degrees
2023-10-10 22:08:00.042 AdvancedCalibration TRACE: X axis rotational error = 7.767912069351814 degrees

There can't possibly be that amount of degrees of skew from what I observe in the camera's viewport :/

[tonyl...@gmail.com]

Please post the entire log file and your machine.xml file.

How far did you back-up in I&S after the camera swap? My guess is that you can't simply start at the Advanced Camera Calibration step once you've swapped the camera.

What happens if you run Advanced Camera Calibration manually from its tab for the camera?

Tony

[Roman Valls]

I've decided to publish sync all logs, raw debug images publicly over this MEGA.nz share:

https://mega.nz/folder/TAdVjbRY#a9t0t1vMAnx3P7RML2k4Xg

So there you'll find all logs and latest machine.xml & co files. Sorry about the "undecrypted folder" names, it seems to be a MEGA.nz bug but if you enter those folders you'll see contents (logs and images from pretty much all history of that machine).

> How far did you back-up in I&S after the camera swap?

I routinely un-solve all the way to rough fiducial positioning so I run it all again basically.

> What happens if you run Advanced Camera Calibration manually from its tab for the camera?

Pretty much the same, massively cropped and skewed viewport after it finishes calibrating (successfully most of the attempts). I've been unsuccessful trying to find the LitePlacer camera specs (specially the depth of field) so that I can adjust the distance from the bed accordingly... do you happen to know those or recommend a reasonable distance that'll minimise focusing and other distortions?

On my latest experiments I've found that not running the advanced camera calibration can yield acceptable results and allow the machine to work fairly well (detecting fiducials and homing flawlessly)... before it was a bit hit and miss.

[tonyl...@gmail.com]

> Pretty much the same, massively cropped and skewed viewport after it finishes calibrating (successfully most of the attempts).

I looked at the calibration results (in your last machine.xml file) and they seem reasonable to me although your residual errors in the X direction are about twice as large as they are in the Y direction - possible mechanical issue with your X-axis or camera settling issue? Just for reference, I've attached my top camera results below. As you can see, your residual errors are more than 3X mine. Can you post a screen capture of the camera view displaying the massively cropped and skewed viewport? Have you tried correcting the rotation errors displayed on the Advanced Calibration tab (see the last paragraph of Camera Mounting Errors)? You can also from my results that I have been able to reduce the X and Y Camera Mounting Errors to less than a tenth of a degree (it takes some trial and error, but it can be done). BTW - any reason you are using 640x480 resolution for your top camera? I've been using 800x600 - that could account for some of the difference in residual error magnitude.

Tony

[Roman Valls]

Thanks for the pointers! I'll double check the X axis, I can tell is **slightly** stiffer than the Y axis.

Totally hear you about the camera resolutions, I didn't notice that, I guess it should be on I&S, that'll be valuable! I just changed it to 800x600 so we can compare more easily... not sure if it's worth setting it to 1024x768?

I've attached the cropped/skewed viewport after calibration from last week (and before changing resolutions now)... I do try to correct the rotation/height and general camera mounting errors you mentioned... thanks for that link, btw, I had no idea that the Z height induced rotation correction(s), seems very counterintuitive!

Off to loosening that X axis and re-calibrate it all, thanks again for the insights!

[Roman Valls]

For the record, I just got this message from advanced camera calibration… never failed this way before, logs and configs on Mega. Before changing the resolution, advanced camera calibration did not fail, I guess I'll have to move the headcam a bit further up since I noticed that at 800x600, the primary fiducial appears closer (zoomed in).

You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/Yn_mgos8iPY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/4587098f-9b71-4dfc-8a09-b4a6919815aen%40googlegroups.com.

[tonyl...@gmail.com]

Post the log from that failed attempt. My guess is you need a much better diffuser on your top camera lighting. All those bright led reflections is probably causing you lots of issues.

Tony

[Mike Menci]

Your LED Ring is leaving reflections from PCB or Table - try this ring ; https://www.ebay.com/itm/175531382556?var=474863878473 Warm or Cool White ! 

I just received some - this are quite strong and I think I need a dimmer with this rings.. 

Mike

[Mike Menci]

a photo attached   25mm OD - might not need the diffusion. 

Mike

[Roman Valls]

The fiducial pipeline seems to be oblivious to the ring of light dots (CV edit pipeline shows 1.0 score for the fiducial and never matches the rings of light).

But yes, I will definitely put a diffuser in there, thanks mike for those neat non-dotted lights, might buy them on Ali as well for good measure!

Attaching the log too... available on the MEGA.nz link I shared earlier since all files are sync'd everytime I use/change configs on the machine now:

https://mega.nz/folder/TAdVjbRY#a9t0t1vMAnx3P7RML2k4Xg/file/bQUm0TIA

And the link for ALL the logs if you are curious about the other errors/issues is this one:

https://mega.nz/folder/TAdVjbRY#a9t0t1vMAnx3P7RML2k4Xg/folder/2AsQhDYS

Thanks a ton again for the support, tomorrow I'll add a diffuser, but I'm skeptical that's the underlying issue.

Cheers!

[tonyl...@gmail.com]

You can't judge it by how well fiducial detection works during homing or when aligning a board. That is always done with the fiducial very near the center of the camera's field-of-view. Advanced Camera Calibration needs to be able to reliably detect the calibration fiducial anywhere in the camera's field-of-view and that includes those areas where you are having bright led reflections.  Advanced Camera Calibration is probably the most stressing measurement of your machine's combined capabilities - mechanically, electrically, and optically. If it is failing or giving bad results, it is almost certainly pointing to one or more unresolved problems with your machine in any of those areas.  You'll be ahead of the game if you fix them now before you try running jobs and discover problems there.

BTW - it is nice to have all your files saved in one place, but we/I don't have the time nor inclination to search through a hundred different log files - in the future please point us to a specific log file.

Tony

[Artem Stanchak]

З повагою, Артем Станчак

менеджер "Bitwire", SMT Engineer

tel: +38(067)3117978


"Ми втілюємо Ваші ідеї у життя"

   

вт, 14 лист. 2023, 16:25 користувач Mike Menci <mike....@gmail.com> пише:

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/b803470f-ff79-4907-8266-5e7cda7bf87cn%40googlegroups.com.