Surely my fault - but advanced camera calibration 'broken' vision

[JW]

I've been slowly working through setting up my new machine build, which has been going great. I had completed basic camera calibration using the dual-Z height rig a couple of weeks ago, and had good (but not perfect) jogging. Just a fraction of a mm out at the corner of the camera view due to the lens distorsion.

I've got nozzle changes working, runout comp etc.

However, I ran advanced camera calibration on the down camera, and whilst it's improved jogging accuracy, it's essentially totally blocked out the vision by application some sort of mask, as per the below image. 

I completed advanced calibration of the up camera, and that too has resulted in essentially only the nozzle tip being visible.

Furthermore, the machine now fails homing due to errors in runout comp - which worked perfectly before doing the advanced camera calibration.

Fully expecting this to be something I've done, so I guess my question(s) are:

1. How do I undo advanced camera calibration
2. Has anybody seen this before/knows what causes this

Also very oddly, in the down looking camera view - the diffused circle of the down camera ring light is now no longer centered on the camera view, it's heavily skewed, and the image aggressively distorted (so much so, the crosshairs out outside of the ring light reflection), but in reality this is almost perfectly centered, and showed as such in the camera preview before advanced cal. Will grab a screenshot of this tomorrow.

[tonyl...@gmail.com]

Please post a copy of your machine.xml file. I'll take a look at your results.

Tony

[JW]

Thanks Tony, file attached.

I see the sections in there for the advanced camera calibration, but not sure what I'm looking for...

[tonyl...@gmail.com]

I looked at your top (down looking) camera results and your residual errors don't look too bad. However, it does appear your camera is mounted with quite a bit of tilt - see the Camera Mounting Error fields on the Advanced Calibration tab for the camera (shown below). Your camera is tilted over -10 degrees about the x-axis and over 7 degrees about the y-axis (the camera is looking towards the front and left of your machine rather than straight down).  You probably should try to correct the tilt of your camera and rerun the Advanced Calibration to get those errors to be under half a degree or so. Please read Camera Mounting Errors for more information.

Your bottom (up looking) camera has terrible residual errors (see below, some are greater than 25mm!). I believe you need to install a light shield behind your nozzle so that the bottom camera is only seeing the nozzle tip and not a bunch of other bright objects in the scene. Once you've done that, rerun Advanced Calibration on the bottom camera.

Tony

[JW]

Hi Tony,

Thanks for looking into this, I really appreciate it. But... now I'm really confused. The camera isn't skewed at all, I've directly measured it, and both cameras are within a fraction of a single degree of square to the machine bed.

I know you can't see it in the pictures (even more so when I haven't held the camera straight), but here's a few for reference anyway, but I've measured it - and the tilt just isn't there.

That's what confused me the most, prior to running advanced cal, the soft diffused light reflection in the down camera was perfectly centered around the crosshair, but having completed the calibration, it's now a good ~15mm off the crosshair location.

When you say re-run calibration, do I need to 'undo' the calibration first, or does re-running it simply overwrite the results of the first calibration? I ask mainly because the camera preview is now so 'wrong' (I don't know how else to describe it), I don't know if the what's shown in the preview is what's used in the calibration, or the image earlier in the pipeline.

Thanks again!

[JW]

OK, it occurred to me I was recently involved with a vision project (nothing to do with OpenPnP) where we had some challenges recognising large fiducals with OpenCV, but for some application specific reasons.

Anyway, thought it would be worth a look - my primary fiducial was 6mm, so I dropped this to 3mm, recalibrated everything. Same problem persists.

I've re-measured the camera, the mounting tilt is 0.179degrees worst axis.

This image is before advanced calibration. Job performance isn't great, and there's certainly some visible distortion, but you can also see the camera ring light is almost perfectly cantered on the crosshair.

This after re-calibration using advanced calibration, and I've noticed you can just apply/not apply the cal using a switch in machine setup, so this is with. Jog performance is much better near the centre of the image, but actually much worse near the edges. You can very clearly see too, the ring light reflection is now no where near the middle of the image anymore.

Very confused on this one...

[JW]

Same before and after images for up camera

Interesting what's left in the image after applying the calibration, is not what the camera is looking at at the cross hair point, without the calibration

[tonyl...@gmail.com]

> I've re-measured the camera, the mounting tilt is 0.179degrees worst axis.

How are you measuring this? I assume you are just measuring the tilt of the camera body/PCB. As stated in the wiki, it's the optical axis that you need to be concerned about. You probably have no idea how well the lens and image sensor are aligned with the camera's PCB.

For your bottom camera, you still need to mount a shield behind your nozzle tip(s) to block the view of bright objects before you can expect any good results from its calibration. You will need this anyway when you try to use bottom vision to align parts.

Tony

[JW]

You're correct, yes - this is the camera PCB tilt measured only.

That said, if the sensor were >10 degrees out, with no calibration applied - is there something else going on in OpenPnP that would mean the ring light's image is still centered?

Ref the up looking camera, I'm not saying I don't need the shield - but I don't understand why? I have good runout comp performance on all nozzle sizes, and the vast majority of parts fall within the green nozzle tip background - all the other features in the corners of the image (other nozzle, machine parts etc) are all well out of where parts will be. What purpose does the shield serve in that case?

[mark maker]

Firstly, I don't think it is just camera tilt. Something may be off with the fiducial detection.

To eliminate that factor, please use a 1mm printed and glued down fiducial, as recommended. I don't know what you are using right now, but 3mm is still rather large and if you are using one of your table holes, or the "confetti" seen in one photo, don't! I also recommend against using a PCB fiducial, especially when you have HASL finish, as in your photo. The reflection messes it up.

The fiducial has to be flat and 2D and matte, as the camera calibration will have to look at the fiducial from the extreme "corner of its eye":

https://github.com/openpnp/openpnp/wiki/Vision-Solutions#calibration-primary-fiducial

Note, this is markedly different, for instance, in Visual Homing, in the nozzle offsets confetti test, or in normal PCB fiducial detection, where the fiducials are iteratively centered in the camera (and its light) to eliminate such 3D and reflection errors. That's why it is acceptable to use a confetti or PCB fiducial there (but not here).

If this does not solve the problem, I would next question your machine motion.

> That said, if the sensor were >10 degrees out, with no calibration applied - is there something else going on in OpenPnP that would mean the ring light's image is still centered?

Yes. The only thing that could be wrong there is when your PCB holder/machine table is not parallel to the X/Y moving axes. But I don't believe that.

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/efb0e0cc-ff70-4946-9b47-4cb5589faac7n%40googlegroups.com.

[JW]

Hi Mark,

I agree, it does seem to behave like it's something to do with fiducial alignment, as the camera tilt is definition minimal - whether we're talking sensor or body tilt, I think so solely because the ring light is so central in the view (which you can see in the videos I've uploaded for an unrelated thread on here).

I'm not using the confetti as the fiducial, they was just left on the machine bed from doing the nozzle <> camera offset step in I&S. The primary and secondary fiducials at printed on 70um thick matte adhesive labels, and stuck directly to the machine bed + a small step for the secondary. Albeit 3mm in diameter, I missed the 1mm requirement, so I'll reprint them and try again.

I've not measured the alignment of the machine bed to the head, but giving the machine construction I'd estimate it to be a fraction of a mm across the whole machine.

Will feed back here once I've retried with smaller fiducials.

[JW]

The other thing I thought, is the fiducials are printed with a 204dpi industrial label printer, so the circles aren't perfect, they are essentially 'granular circles'. That said, during the visual homing, the green circle drawn on the homing fiducial is at least to my eye, perfectly visualised each time.

Presumably some Gaussian blurring in the vision pipeline would mitigate this anyway?

[tonyl...@gmail.com]

> Something may be off with the fiducial detection.

While I can't entirely discount this, the fact that the residual errors (for the down looking camera) are relatively small, would indicate to me that the fiducial is being (fairly) reliably detected.

If you are in doubt of camera tilt, disable Advanced Calibration, place a piece of quality graph paper (I use some with 2mm grid spacing) flat on your machine table, select Reticle->Grid, select Reticle->Options->Units per Tick to match the graph paper, and try and align the paper grid to the camera's grid. You may need to adjust the top camera's virtual Z value to get the grid sizes to match exactly all the way out to the edges of the image. If you have no tilt, you should be able to match the paper grid very well to the reticle grid. If you have tilt, the paper grid will grow and/or shrink towards the edges of the image.  Below is an example of what I have on my machine showing the same image with the Ruler reticle and with the Grid reticle (both with Advanced Calibration enabled).

    

[JW]

Ok, so - great news.

I've managed to get up and down advanced vision calibration completed, as shown in this video. The problem was as Mark suggested - the lenses on both my up and down facing camera were moving under high acceleration.

I've fixed them in place with PTFE tape on the lens thread in addition to the lock screw, and advanced calibration completed just fine - and the images are no longer masked out, jogging is good, and straight lines are shown as shown (I have a checkerboard on the machine bed, rather than the graph paper).

However, I can no longer park the machine after completing the calibration, with OpenPnP saying it can't drive to lower than the soft limits, as expected, but why is trying to drive lower anyway? As seen in the video, my park location is 0,0,0 and has been working well the whole time until completing advanced vision calibration.

It is always -0.284mm it tries to send the machine to during park now.

Massively appreciate everybody's input getting to this point!

[JW]

Camera positions are still 0,0,0 too after the calibration, so I don't think it's that..

[JW]

Ah, no they're not - not on the advanced cal page. So, what does this mean? How does the camera position move? Is that because the actual center of the image has been mapped 0.284 away from the physical camera center?

[tonyl...@gmail.com]

Easiest thing to do is open up your lower soft limit on your x-axis a little bit (set it at say -0.5mm rather than 0.0mm). The reason is that Advanced Calibration detected a slight tilt (0.424 degrees around the y-axis) so it compensates for that with the Calibrated Head Offsets (0.284mm in the x direction). With Advanced Calibration active, the crosshairs now intersect at the image pixel that corresponds to the one perfectly vertical ray as opposed to the image pixel that was at the center of the raw image (which corresponds to a tilted ray). This means to get the crosshairs aligned over the homing fiducial, it now needs to drive the x-axis 0.284mm further to the left.  Whatever you do, DON'T try to adjust the offsets on the Position tab once Advanced Calibration is active - that is the wrong thing to do.

Tony

[JW]

Hi Tony,

Understood, - thank you. That helps me get my head around what is going on. Just before your post, I just set my park to X3 Y-3, but that achieves the same thing I think.

[tonyl...@gmail.com]

Ok, I misread you post and thought it was failing during homing. Yes, you can just move the park location slightly or adjust the soft limits slightly.

Oh, one other thing you should do is slide the Crop All Invalid Pixels sliders all the way to the left on both your cameras. That will crop your images to remove the curved edges from the images - some pipelines get confused with those curved bits.

Tony

[JW]

Ah no sorry, homing worked fine as that was just sending G28 to Duet, it was parking that was the issue.

Good top on the pixel crop too, thanks.

Great to have gotten this one resolved, appreciate all the help!

[Town Java]

After the advanced visual calibration, the displayed content is offset and the image is distorted? What is the reason for this?

JW <julian...@gmail.com> 于2024年6月23日周日 01:11写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/438f11df-0d06-4529-bb1f-bfad9f720de6n%40googlegroups.com.

[tonyl...@gmail.com]

The reason is something is not correct. We need more information - post some pictures and your machine.xml file for a start.

Tony