Advanced camera calibration - invalid pixel cropping - question

[vespaman]

Hi all,

So back after a long nice summer, I'm having a go at making my (dual) up vision cameras more optimized for my components. 
However, I am struggling to understand how the "Crop All Invalid Pixels <--> Show All Valid Pixels slider" actually change the vision performance.. 

In my current setup, it looks to be changing unit per pixel from below 20 up to about 50, and the place resolution as a consequence of that. 

Now, what I wonder is how one should think regarding this; is the worse resolution linear to the whole picture, or is the worse number in reality only in the outskirts of the picture? I.e. if I open up max (show all pixels), will also a small component (e.g. 0402/0201) suffer in the middle of the pic, or will this worse number only apply to larger components, such as connectors (that covers a big part of the picture (big enough not to fit the pic if "crop all" is selected) etc?

 - Micael

[Toby Dickenson]

Most vision operations use a circular mask, so pixels in the corners of the image (or at the edges, if you have a letterbox aspect ratio) are not particularly useful. I would happily crop away anything that improves the view of the pixels that matter.

Toby

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/openpnp/fd1182b7-5758-4b5b-9af1-3e2268a18858n%40googlegroups.com.

[vespaman]

I have already cropped away the letterbox, I'm currently using a fairly square 1080x960 setting of the camera. And if I am able to keep this image size, my largest component is visible in one shot.

But the cost is the reduction of resolution according to the advanced camera setup. But the doc's says 

Invalid pixels (usually displayed as black) on the edges of the image may result due to the image processing that compensates for errors in camera mounting and lens distortion. While mostly aesthetic, changing this setting does change the Units Per Pixel for the camera.

Now, I don't see any black in the corners, but I guess that the errors comes from lens distortion or problem properly finding the black hole of the black nozzle tip in the outer regions during calib.

The "While mostly aesthetic" line sounds to me, that maybe the actual resolution in the center might not be affected, only at the outer areas, which is probably ok, since at least my large components are not fine pitch at all. But maybe the Units Per Pixel is widely used in the vision system, and therefore also attributes to worse resolution/accuracy throughout?

 - Micael

[Toby Dickenson]

What you are describing is outside my experience of the Advanced Camera Calibration. Can you share some images? This image from the wiki shows a small impact on units-per-pixel.

https://user-images.githubusercontent.com/50550971/135167265-86140afa-a279-4c7c-8bbe-4fdcd8cdedfa.gif

To view this discussion visit https://groups.google.com/d/msgid/openpnp/6f754df5-ccad-4e8a-a5ce-3854a3d7f6cfn%40googlegroups.com.

[vespaman]

I guess the units per pixel change will depend on the errors found? 

This is what I get on first camera;

The second camera gives similar result;

A strange thing, that I noticed (don't remember if this is how it used to be before?), is that the tool center is changing drastically after the advanced calibration has been done. The pictures shows the tool centered! (I manually set the center on the Position tab prior to enabling the advanced calibration).

So now I need to reset the tool center, but this does not make sense to me. The difference is, as can be seen 3/5mm

Second camera has a bigger mounting error than the first, and those numbers seems rather high to me.. Maybe this is the reason for the tool center change, "Camera Mounting Errors" in the wiki seems to confirm this.

Maybe the camera mount is still inferior. Thing is, this is the 5.23534 million iteration of the mount. 

 - Micael

[vespaman]

Well, after measuring the distance and doing the trigonometry, the 3/5mm makes sense. So this is one thing that could be improved. 

But I'm not sure this explains the invalid pixel stuff. But maybe it does, since two sides in particular is essentially clipped more, given the angle. But I don't see how the math ends up in the 10mm "invalid pixel crop" on the second camera, it looks to be twice the 3/5mm angel error somehow.

  - Micael

[Jan]

Hi Micael!
IIRC advanced camera calibration "reorders" the sensor pixels such that
the imaging surface is perpendicular with respect to the camera view.
This almost always shifts the image center (unless you do not have any
mounting errors). If you right-click the camera view, you can choose to
move the active tool to the camera center. This will take this offset
into account.
Btw: your mounting errors look quite small to me with 3° as max. on the
y axis of the second camera.
Looking at your screenshots I'm surprised that the effective field of
view of your second camera (Upp2) is larger then the physical field of
view. That's somewhat unexpected to me.
Do you see larger errors in the collected data?

Jan

On 13.09.2025 23:44, vespaman wrote:
> Well, after measuring the distance and doing the trigonometry, the 3/5mm
> makes sense. So this is one thing that could be improved.
>
> But I'm not sure this explains the invalid pixel stuff. But maybe it
> does, since two sides in particular is essentially clipped more, given
> the angle. But I don't see how the math ends up in the 10mm "invalid
> pixel crop" on the second camera, it looks to be twice the 3/5mm angel
> error somehow.
>
>   - Micael
> lördag 13 september 2025 kl. 18:55:55 UTC+2 skrev vespaman:
>
> I guess the units per pixel change will depend on the errors found?
> This is what I get on first camera;

> Screenshot_20250913_182147.png

>
> Screenshot_20250913_182228.png
>
> The second camera gives similar result;

> Screenshot_20250913_183155.png

> Screenshot_20250913_183247.png
>
>
> A strange thing, that I noticed (don't remember if this is how it
> used to be before?), is that the tool center is changing drastically
> after the advanced calibration has been done. The pictures shows the
> tool centered! (I manually set the center on the Position tab prior
> to enabling the advanced calibration).
> So now I need to reset the tool center, but this does not make sense
> to me. The difference is, as can be seen 3/5mm
>
>
> Second camera has a bigger mounting error than the first, and those
> numbers seems rather high to me.. Maybe this is the reason for the
> tool center change, "Camera Mounting Errors" in the wiki seems to
> confirm this.
>
> Maybe the camera mount is still inferior. Thing is, this is the
> 5.23534 million iteration of the mount.
>
>  - Micael
> lördag 13 september 2025 kl. 11:28:46 UTC+2 skrev to...@tarind.com:
>
> What you are describing is outside my experience of the Advanced
> Camera Calibration. Can you share some images? This image from
> the wiki shows a small impact on units-per-pixel.
>
> https://user-
> images.githubusercontent.com/50550971/135167265-86140afa-

> a279-4c7c-8bbe-4fdcd8cdedfa.gif <https://user-
> images.githubusercontent.com/50550971/135167265-86140afa-
> a279-4c7c-8bbe-4fdcd8cdedfa.gif>

>
> On Fri, 12 Sept 2025 at 22:01, vespaman <micael....@gmail.com>
> wrote:
>
> I have already cropped away the letterbox, I'm currently
> using a fairly square 1080x960 setting of the camera. And if
> I am able to keep this image size, my largest component is
> visible in one shot.
> But the cost is the reduction of resolution according to the
> advanced camera setup. But the doc's says
>

> /Invalid pixels (usually displayed as black) on the edges of

> the image may result due to the image processing that
> compensates for errors in camera mounting and lens
> distortion. While mostly aesthetic, changing this setting

> does change the Units Per Pixel for the camera./

> However, I am struggling to understand /how/ the

> "Crop All Invalid Pixels <--> Show All Valid Pixels
> slider" actually change the vision performance..
> In my current setup, it looks to be changing unit
> per pixel from below 20 up to about 50, and the
> place resolution as a consequence of that.
>
> Now, what I wonder is how one should think regarding
> this; is the worse resolution linear to the whole
> picture, or is the worse number in reality only in
> the outskirts of the picture? I.e. if I open up max
> (show all pixels), will also a small component (e.g.
> 0402/0201) suffer in the middle of the pic, or will
> this worse number only apply to larger components,
> such as connectors (that covers a big part of the
> picture (big enough not to fit the pic if "crop all"
> is selected) etc?
>
>  - Micael
>
> --
> You received this message because you are subscribed
> to the Google Groups "OpenPnP" group.
> To unsubscribe from this group and stop receiving
> emails from it, send an email to
> openpnp+u...@googlegroups.com.
> To view this discussion visit https://
> groups.google.com/d/msgid/openpnp/

> fd1182b7-5758-4b5b-9af1-3e2268a18858n%40googlegroups.com <https://groups.google.com/d/msgid/openpnp/fd1182b7-5758-4b5b-9af1-3e2268a18858n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the
> Google Groups "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to openpnp+u...@googlegroups.com.
>
> To view this discussion visit https://groups.google.com/d/

> msgid/openpnp/6f754df5-ccad-4e8a-
> a5ce-3854a3d7f6cfn%40googlegroups.com <https://
> groups.google.com/d/msgid/openpnp/6f754df5-ccad-4e8a-
> a5ce-3854a3d7f6cfn%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openpnp+u...@googlegroups.com

> <mailto:openpnp+u...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/

> openpnp/6e02ab9d-d181-469c-922e-7eea1251a04an%40googlegroups.com
> <https://groups.google.com/d/msgid/openpnp/6e02ab9d-
> d181-469c-922e-7eea1251a04an%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

[vespaman]

Hi Jan,

IIRC advanced camera calibration "reorders" the sensor pixels such that
the imaging surface is perpendicular with respect to the camera view.
This almost always shifts the image center (unless you do not have any
mounting errors).

 

Yes, I think I understand this now. Maybe it was not such a large shift before (with my old camera setup), so I did not pay much attention to it.

If you right-click the camera view, you can choose to
move the active tool to the camera center. This will take this offset
into account.

 

Good, so in fact I don't need to reset the camera/nozzle center position on the 'Position' then. Maybe I shouldn't even. 

 

Btw: your mounting errors look quite small to me with 3° as max. on the
y axis of the second camera.

OK, good to know! I thought maybe it was a lot.

 

Looking at your screenshots I'm surprised that the effective field of
view of your second camera (Upp2) is larger then the physical field of
view. That's somewhat unexpected to me.

Yes, to me too. I don't understand that at all. First camera (Upp1) also has this when showing all pixels.

 

Do you see larger errors in the collected data?

No, I don't think so. But maybe I am not reading things right?

This is the second camera at both Z readouts, with outliers enabled.

I think this is about the same errors that I used to have with my old cam setup.

As a side note;

One thing that I have learnt to be the perhaps biggest problem with doing multiple up-looking cameras, is the advanced camera calibration; since it shift things around, it therefore makes the granular ganged alignment extremely difficult. And since I still don't fully understand how everything ties together, I don't really have an idea on how to attack this problem. 

I guess the simple answer is to make sure there is very little mechanical/physical errors between the cameras. But accomplishing this seems to be very hard.

The original chmt camera holder has set screws pushing the lens for angle adjustment. And I remember it also had shims where the holder met the base plate. Maybe I need to follow this method somehow.

  - Micael

[Jan]

Hi Micael!

On 14.09.2025 13:18, vespaman wrote:
[...]> If you right-click the camera view, you can choose to

> move the active tool to the camera center. This will take this offset
> into account.
>
> Good, so in fact I don't need to reset the camera/nozzle center position
> on the 'Position' then. Maybe I shouldn't even.
>

In fact - IIRC - Tony said, that this is not recommended or shall even
be avoided. If you don't like the offset, I'd recommend to run without
advanced camera calibration. According to your residual errors, there
seems to be not much lens distortion to correct for. So there is not
much more then just the angular error.

> Btw: your mounting errors look quite small to me with 3° as max. on the
> y axis of the second camera.
>
>
> OK, good to know! I thought maybe it was a lot.
>

I tried to minimize the error on my camera and it was hardly possible to
get below 1° even though I had very small spacers at hand. I'm also not
sure what the error on this values is.

> Looking at your screenshots I'm surprised that the effective field of
> view of your second camera (Upp2) is larger then the physical field of
> view. That's somewhat unexpected to me.
>
>
> Yes, to me too. I don't understand that at all. First camera (Upp1) also
> has this when showing all pixels.
>

I hope Tony can explain that some day...

> Do you see larger errors in the collected data?
>
>
> No, I don't think so. But maybe I am not reading things right?
> This is the second camera at both Z readouts, with outliers enabled.
>

No, I don't see them either. Seems that you're setup is quite solid for
what Advanced Camera Calibration requires. All errors are in the order
of a few UPP.

> As a side note;
> One thing that I have learnt to be the perhaps biggest problem with
> doing multiple up-looking cameras, is the advanced camera calibration;

> since it shift things around, it therefore makes the granular /ganged
> alignment/ extremely difficult. And since I still don't fully understand

> how everything ties together, I don't really have an idea on how to
> attack this problem.

> I guess the simple answer is to make sure there is /very little/

> mechanical/physical errors between the cameras. But accomplishing this
> seems to be very hard.
>

At the end I'd recommend to use nozzle tip calibration. This shall
result in an offset error that compensates for the camera mounting as
well. (Actually I'm not 100% sure, if that's already the case because -
as you said - the nozzle is usually moved to the corrected cameras
location, which is fine for single camera, but you would need a it to
compensate for the camera mounting error instead: move the nozzle to a
fixed location and compensate the camera such, that the nozzle tip
appears in its center. Maybe that's something you need to enhance too...)

> The original chmt camera holder has set screws pushing the lens for
> angle adjustment. And I remember it also had shims where the holder met
> the base plate. Maybe I need to follow this method somehow.
>

I see both on my machine but I'm unsure if the set screws where for
correction or rather for fixing the lens only. On all the cameras I've
seen so far, the mounting of the lens with respect to the sensor is
somewhat arbitrary. It seems that no one cares about the optical axis of
the lens to hit the sensor in the center and under 90°. Btw: do you have
any idea how to verify/adjust that? I would be glad to know. I usually
glue the lens holders to the PCB to fix this alignment. If it would be
adjusted, it would be even better...

Jan

[vespaman]

Hi Jan,

seems to be not much lens distortion to correct for. So there is not
much more then just the angular error.

This is interesting. Can you understand why the invalid pixel cropping on both cameras a) crops on the right side only, and b) up to twice the angle error distance? Or maybe 'b' isn't twice, but a while that just happens to look like twice?

 

I'm also not sure what the error on this values is.

I wonder about the error as well, but I am assuming it is very low at the moment.

 

> Looking at your screenshots I'm surprised that the effective field of
> view of your second camera (Upp2) is larger then the physical field of
> view. That's somewhat unexpected to me.
>
>
> Yes, to me too. I don't understand that at all. First camera (Upp1) also
> has this when showing all pixels.
>
I hope Tony can explain that some day... 

At the end I'd recommend to use nozzle tip calibration.

By this, do you mean the confetti calibration?  

 

On all the cameras I've
seen so far, the mounting of the lens with respect to the sensor is
somewhat arbitrary. It seems that no one cares about the optical axis of
the lens to hit the sensor in the center and under 90°.

 

Yes, this is my experience too! I am pretty sure a significant part of my current problem is partly this, but I have found no accurate way to adjust this.

On the cameras I play with now, only two screws (with some play of course) holds the M12 lens holder. I guess in normal applications, this is not important, and I think it is accelerated with larger sensors like the AR0234 and imx290.

And, of course, since I am using a 8mm lens, the distance between sensor and nozzle tip is almost 10 cm. I have not found any really good M12 specified for min focus length in this area, even the camera supplier tried to help without much success. And creating a special lens with this in mind was a bit too expensive for my liking. 

After giving this some thought, my current plan is to redo the holder once again (at least now I know what not to do ;-) ), adding 3 set screws to each camera, to be able to tilt the cameras individually. These screws I think must be made easily accessible from above (through the camera port in the top plate). Not sure if I can get enough precision with 3d printed parts, I'm thinking machined aluminium at the end, but testing with 3d printed. Possibly resin printed parts as a mid way test.

Regardless what solution, I think the working order must be

1. Get the angle errors under control on both cameras. Use advance calibration to verify. Maybe I don't need too many steps in the advanced calibration (to speed things up).

2. Then move them X/Y

3. Then do vision calibration

4. Repeat step 2

5. Verify angle and repeat as needed (hopefully not, but 3d printed, I'm pretty sure that the angle will change when moving the cameras in step 2)

I currently think, that I don't need to worry too much about camera rotations (fingers crossed).

- Micael

[Jan]

Hi Micael!

On 15.09.2025 17:20, vespaman wrote:
[...]

>
> At the end I'd recommend to use nozzle tip calibration.
>
>
> By this, do you mean the confetti calibration?
>

No, the confetti calibration is for the nozzle to down-looking camera
offset. I mean the nozzle tip runout and offset calibration that is
usually done after loading a nozzle tip. It's part of the nozzle tip
configuration wizard.

> On all the cameras I've
> seen so far, the mounting of the lens with respect to the sensor is
> somewhat arbitrary. It seems that no one cares about the optical
> axis of
> the lens to hit the sensor in the center and under 90°.
>
> Yes, this is my experience too! I am pretty sure a significant part of
> my current problem is partly this, but I have found no accurate way to
> adjust this.
> On the cameras I play with now, only two screws (with some play of
> course) holds the M12 lens holder. I guess in normal applications, this
> is not important, and I think it is accelerated with larger sensors like
> the AR0234 and imx290.
> And, of course, since I am using a 8mm lens, the distance between sensor
> and nozzle tip is almost 10 cm. I have not found any really good M12
> specified for min focus length in this area, even the camera supplier
> tried to help without much success. And creating a special lens with
> this in mind was a bit too expensive for my liking.
>

You can buy special lenses that can be shifted and/or tilted out of
their well aligned position. That is usually used to image tilted
surfaces. I assume that a shifted lens holder therefore results in a
tilted image surface and/or addition distortions at the edges.

> After giving this some thought, my current plan is to redo the holder
> once again (at least now I know what not to do ;-) ), adding 3 set
> screws to each camera, to be able to tilt the cameras individually.
> These screws I think must be made easily accessible from above (through
> the camera port in the top plate). Not sure if I can get enough
> precision with 3d printed parts, I'm thinking machined aluminium at the
> end, but testing with 3d printed. Possibly resin printed parts as a mid
> way test.
>

You may try to add a spring that applies a constant force onto the set
screws. That shall allow you to use them reliably even without a
perfectly play-free machanical design by always pulling the camera into
the same direction.

> Regardless what solution, I think the working order must be
> 1. Get the angle errors under control on both cameras. Use advance
> calibration to verify. Maybe I don't need too many steps in the advanced
> calibration (to speed things up).
> 2. Then move them X/Y
> 3. Then do vision calibration
> 4. Repeat step 2
> 5. Verify angle and repeat as needed (hopefully not, but 3d printed, I'm
> pretty sure that the angle will change when moving the cameras in step 2)
> I currently think, that I don't need to worry too much about camera
> rotations (fingers crossed).
>

It would be probably easier if you could measure/calibrate the position
of the nozzle with respect to your camera. That would relax the camera
alignment requirements a lot...

Jan

[vespaman]

Hi Jan,

 I mean the nozzle tip runout and offset calibration that is
usually done after loading a nozzle tip. 

OK, yes I am using that. But I am not sure if that changes anything regarding the issue of ganged camera sensor/optical alignment.

 

You may try to add a spring that applies a constant force onto the set
screws. That shall allow you to use them reliably even without a
perfectly play-free machanical design by always pulling the camera into
the same direction.

Yes, I have been calculating with this, using fine threaded M3 set screws, one full turn is about 1° if the set screw is pushing directly on the sensor ass'y, but perhaps can move it further out. (at about 30mm distance between the set screws, I get about 0.6° on a full turn, which ought to be enough.

But I need to think this through more - I am currently considering to ditch the M12 mount for M16 mount instead, this looks to give better lens selection for larger sensors. Ideally, I'd machine my own lens holder, and therefore be able to integrate a lot of adjustments directly into it. That is probably overdoing it, but it is interesting to include that as a possible path forward. 

 

  - Micael

[vespaman]

Today I realized something I should have thought about earlier. The advanced camera calibration isn't really just about the camera, it is as much the nozzle travel direction.

I tested this by doing multiple cam1 calibrations against nozzle 1, I got about 0.2° max difference between the runs. I guess this can, to some extent, be attributed to the fact that I only have 4mm Z difference between the two sub-tests.

Then I tested cam1 against nozzle 2, and got different result. These results where about 0.5° different in Y direction against nozzle 1.

I'm not sure what to make of this information yet, but it makes me wonder about possible problems with multi nozzle machines - isn't there a risk that the advanced calibration makes things worse for the nozzle(s) that is not calibrated? Or am I missing something here?

Maybe the idea is that multiple nozzles has less deviation between them, than the camera mount direction, and that it is therefore still better than uncalibrated.

If I misunderstood this, then I guess I could have gotten the above results because of inaccuracy of the tests. 

The reason for me to start thinking about the nozzle directional z, was that I had an idea to mill both camera sensors/lens holders  in one solid piece with the same distance as between the nozzles. But I guess this cannot work given the above.

Yesterday I found what I hope, a much better lens, which hopefully lets me use a higher Z difference between the sub tests. But of course they will not arrive for another couple of weeks.  But it will be interesting to see if greater Z gives less deviation between runs.

 -  Micael

[vespaman]

Hi Jan,

Coming back to something you wrote earlier..

 

> Good, so in fact I don't need to reset the camera/nozzle center position
> on the 'Position' then. Maybe I shouldn't even.
>
In fact - IIRC - Tony said, that this is not recommended or shall even
be avoided. If you don't like the offset, I'd recommend to run without
advanced camera calibration. According to your residual errors, there
seems to be not much lens distortion to correct for. So there is not
much more then just the angular error.

I have been thinking to disable the advanced camera calibration, but it is not clear for me, what the consequences of doing that is.

Do you know?

I mean, I guess some loss of accuracy will occur on up vision check, or perhaps even on placing? 

Remember that I am doing my up vision check a highest Z, which I guess also you fly-over-guys also are about to do?

Maybe the best thing is to try to mechanically optimize the angle on each nozzle/camera combo using advanced camera calibration, then, once that is done, disable advanced camera calibration.

 - Micael

[Jan]

Hi Micael!

On 18.09.2025 12:21, vespaman wrote:
> Today I realized something I should have thought about earlier. The
> advanced camera calibration isn't really just about the camera, it is as
> much the nozzle travel direction.
> I tested this by doing multiple cam1 calibrations against nozzle 1, I
> got about 0.2° max difference between the runs. I guess this can, to
> some extent, be attributed to the fact that I only have 4mm Z difference
> between the two sub-tests.
> Then I tested cam1 against nozzle 2, and got different result. These
> results where about 0.5° different in Y direction against nozzle 1.
>

That could also indicate different tilting angles of the two nozzles.
The 0.2° may surf as an indicator for the error on the angles the
method has. However, as it is derived by repeating the procedure it can
only cover the statistical error. Systematic errors are invisible for
that and can only be revealed by careful analysis of the method.
The the end the angle is not so important if you run initial (and
advanced) camera calibration on your intended Z level because for the
same Z no angular correction is needed. If the nozzles are tilted
differently, you'll likely see a small related offset. However, I'm not
100% sure because the nozzle location/offset is measured at PCB Z (using
confetti) and the offset of the nozzle with respect to the up-looking
camera is compensated by the nozzle tip runout calibration...

> I'm not sure what to make of this information yet, but it makes me
> wonder about possible problems with multi nozzle machines - isn't there
> a risk that the advanced calibration makes things worse for the
> nozzle(s) that is not calibrated? Or am I missing something here?
> Maybe the idea is that multiple nozzles has less deviation between them,
> than the camera mount direction, and that it is therefore still better
> than uncalibrated.
>

Certainly yes. The more you measure, the more errors each measurement
adds to the final result... At the end its a question what you need and
how good the results are. OpenPnPs bottom vision relies on calculating a
rectangular boundary from the image. This involves multiple
transformations and pixel-by-pixel calculations. So the uniformity of
the image causes very likely a systematic error that is inherit to the
setup. Advanced camera calibration seems to try to measure and correct
for that. Assuming that the camera chip, its mounting and the lens are
the key origins and the motion system and the nozzle tip are almost
perfect, it likely increases the performance independently of the
nozzle. However, the requirements are at least limited by the step size
which is in the order for about 20um and even 100um are likely
acceptable for 0.4mm QFNs or 0402 passives. According to the residual
errors you showed, your uniformity seems to be well below 100um.

Jan

[Jan]

Hi Micael!

On 18.09.2025 12:36, vespaman wrote:
> Hi Jan,
> Coming back to something you wrote earlier..
>
> > Good, so in fact I don't need to reset the camera/nozzle center
> position
> > on the 'Position' then. Maybe I shouldn't even.
> >
> In fact - IIRC - Tony said, that this is not recommended or shall even
> be avoided. If you don't like the offset, I'd recommend to run without
> advanced camera calibration. According to your residual errors, there
> seems to be not much lens distortion to correct for. So there is not
> much more then just the angular error.
>
>
> I have been thinking to disable the advanced camera calibration, but it
> is not clear for me, what the consequences of doing that is.
> Do you know?
> I mean, I guess some loss of accuracy will occur on up vision check, or
> perhaps even on placing?
>

In older times no one was using advanced camera calibration because it
was not available. As advanced camera calibration is based on
measurements it not necessarily improves the performance. Remember that
we have see people here that suffer from detection problems in the edges
that render it useless. IIRC advanced camera calibration is the
successor of the lens distortion correction used on older setups. Using
a good setup with a good lens is IMHO the first thing to opt for and
only if that can not be improved, things like advanced camera
calibration could be used to compensate for the remaining imperfections.
However, the results will never be "perfect" and we do not need
"perfect" results. The uniformity of the advanced camera calibration
measurements you posted look quite good to me, so I think you may safely
try to run without.

> Remember that I am doing my up vision check a highest Z, which I guess

> also you /fly-over-guys/ also are about to do?
>
I do the same for quite some time now (still without fly-by-vision) and
I don't see any issue. You my go back in the archives where Mark
discussed possible issues if PCB-Z is not Bottom-Vision-Z. IIRC there
are none if nozzle tip runout calibration is used.

> Maybe the best thing is to try to mechanically optimize the angle on
> each nozzle/camera combo using advanced camera calibration, then, once
> that is done, disable advanced camera calibration.
>

Remember, you only need the angle if the image is not uniform (varying
UPP due to tilting) or if you're using the camera at a different Z then
you calibrated it for (which is not and never was recommended). In the
first case Advanced Camera Calibration will likely compensate for the
tilted image making the UPP more uniform and in the second case you need
a new UPP that represents the value for the new Z.

Jan

[Jan]

Hi Micael!
Yesterday I had some more ideas on how to handle multiple up-looking
camera. First, I'd try to make sure that OpenPnP handles offsets
correctly. Assuming that nozzles and cameras will never be perfectly
aligned their offset can be calculated using their calibrated locations.
This offset could then be eg. evenly distributed across them by moving
the center between the nozzles to the center between the cameras. Now
bottom vision needs to take the offset between nozzle and camera into
account. I'm pretty sure, that this is currently not supported because
with a single up-looking camera, each nozzle is always moved to the
location of the camera. I'd verify and implement that using a
significantly different offset between cameras compared to offset
between nozzles. (Such a patch could certainly be merged into general
OpenPnP because it will cancel out for single-up-looking-camera machines.)
With this offset in place you could try to calibration each cameras
location eg. using the primary nozzle. Then one would need to think
about if runout calibration could be done using two cameras (and maybe
in parallel) or if it would be better to use just a single camera. Maybe
two cameras are better because then the camera thats used for bottom
vision is also used to measure the runout...
In any case, I'd suggest to start without advanced camera calibration
as this adds an other layer of complexity.

Jan

[vespaman]

Hi Jan,

Remember, you only need the angle if the image is not uniform (varying
UPP due to tilting)

Yes, exactly, but the easiest way to get the angles is with Advanced Calibration, at least, I don't know any other (simple) way. But once I have mechanically trimmed away the angle errors (well, hitting "Good Enough"), I'll disable Advanced Calibration.

In my case, I tend to want my camera sensor quite far away from the tip, in order to get ok results with the small lenses that I am forced to use - most of them has a M.O.D. (minimal optical distance) way longer than we mostly want in PnP. Using these (usually cheap) small lenses anyway, gives focus issues in the outer region vs the center, and therefore very short depth of field.

Before (single camera) I had great success with those cheap zoom lenses, even on larger sensor chips, but they are too big to be fitted side by side.

...so I have about 10+cm currently, and at this distance, the angle issue is causing big deviations.

Hopefully the new lenses that I ordered will work, they are specified for M.O.D. 10cm, and the term macro lens is in the sales bla-bla.
If they are good, I can open up my cropping a bit more if I need to. And perhaps also come closer to the nozzle tip again.

Thanks for elaborating with me!

 -  Micael

[tonyl...@gmail.com]

Sorry, I've been out of town and am now just trying to get caught up on the discussion.

First, a bit about Advanced Camera Calibration: It is intended to take the place of what used to be several independent and manual calibration steps, namely axis flipping (mirroring), units per pixel calibration, z-axis rotational error correction, and lens distortion correction. Beyond that, it also detects and corrects x-axis and y-axis rotational errors (camera tilt). It performs all this by fitting a mathematical model of the camera (see the gory details here) to the data collected during the calibration. The model is then used to compute a pixel remapping that gets applied to each image provided by the camera before the image is displayed to the operator and/or processed by the pipeline stages. Note that the pixel remapping occurs regardless of whether Advanced Calibration is active or not so using Advanced Calibration does not "slow" the pipeline down.

Now on the OP's original question: What exactly does the "Crop All Invalid Pixels <--> Show All Valid Pixels slider" do? To answer that you first have to understand what happens when the corrections mentioned above are applied to the camera images. Below is an illustration showing the same rectangular object as viewed through a camera (with an exaggerated amount of error) both before and after corrections are applied. 

Note how the corrections distort the boundaries of the image but make the objects that appear in the image appear as undistorted (which is what we want). Unfortunately, all of the software that processes and displays images expect the images to be rectangular, so we need to somehow fit the non-rectangular corrected image into a rectangular box. That is what the "Crop All Invalid Pixels <--> Show All Valid Pixels" slider is all about as shown in the illustration below. We can basically zoom-in far enough that we get a rectangle filled with only valid pixels which means all the invalid pixels are cropped from the image, or we can zoom-out enough that all the valid pixels are included in the image but at the expense of adding invalid (black) pixels to fill in the missing areas:

The slider just smoothly varies from one extreme to the other. It is typically recommended to move the slider all the way to the left (crop all invalid pixels) after calibration is complete. The main benefit of this is to eliminate sharp gradients in the image at the valid-invalid pixel boundaries that may get detected by some pipeline stages giving erroneous results. Note that although moving the slider around does change the units per pixel scaling for the camera it shouldn't really have much effect on the machine's ultimate performance - this is just a digital zoom and can't change the physical camera's units per pixel.

Tony