Advanced Camera Cal - Java error

[David Griffiths]

I have encountered a repeatable Java error when attempting an Advanced Camera Cal on my UP Camera.  I decided to change the camera res to 1280x720 so I needed to recal.

Files are here:

https://www.dropbox.com/sh/7pikr9028w2wnpw/AAAjCQmYHycENeHlBdoXxwsYa?dl=0

Cheers,

DG

[Ian Arkver]

Looks like there's no Nozzle Tip on the head? Background behind the NPE popup shows no nozzle tip selected.

[David Griffiths]

Hi Ian, 

I am not sure what you are looking at to see if there is a nozzle tip loaded? 

DG 

[Ian Arkver]

This...

I can't remember whether it's necessary to have the tip mounted on the head (according to OpenPnP, rather than physically) and/or selected in the jog panel, but it would seem possible that it's related. Mark or Tony would no doubt know for sure.

[mark maker]

If you are using Issues & Solutions to advanced-calibrate the camera, you will be reminded if the nozzle tip is missing (or thought to be missing by OpenPnP).

https://github.com/openpnp/openpnp/blob/af43024e03f6d9e98c738cff8fb1424be52c487b/src/main/java/org/openpnp/machine/reference/solutions/CalibrationSolutions.java#L1206-L1209

I guess this check is missing if you directly use the Advanced Calibration from the camera wizard.

I'll try to keep this in mind and fix it some time.

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/2f714def-5c11-4227-9361-181d4d0f981an%40googlegroups.com.

[tonyl...@gmail.com]

If run it from the camera wizard, the very first instruction is: 

I guess it doesn't actually check that the operator has followed the instruction though.

Tony

[mark maker]

I guess users that are quite new to OpenPnP might misunderstand the instructions and just physically load the nozzle tip, rather than also using the OpenPnP "load" button.

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/de1f0ce1-45fe-49c6-bffb-e6d16e4a41e1n%40googlegroups.com.

[tonyl...@gmail.com]

Agreed - the instruction should be more specific.

[David Griffiths]

Ah yes.  I don't think I ever noted that the loaded tip was displayed there Duh!

I would normally load it using OpenPNP but I was probably in a hurry and just physically loaded it. 

I have now done the advanced camera calibration. Some comments and questions:

- the cal seemed to repeat itself - ie went through the whole process twice including prompting for jogging etc

- twice it said the results didn't culminate (or something similar) and suggested increasing the steps

- the results don't look anywhere near as good as I had previously on the higher resolution. Thoughts?

- where do I find Adv Camera Cal in Issues and Solutions?

- I now have a residual footprint shown on the down camera - see 2nd pic.

Many thanks, DG

[tonyl...@gmail.com]

> - the cal seemed to repeat itself - ie went through the whole process twice including prompting for jogging etc

The calibration collects data at two different Z heights - is that what you mean by going through the whole process twice?

>- twice it said the results didn't culminate (or something similar) and suggested increasing the steps

Please post the log file and the resulting machine.xml file so I can take a look.

>- the results don't look anywhere near as good as I had previously on the higher resolution. Thoughts?

Can you be more specific about which results?  It appears you have a high aspect ratio camera - do you really need that?  Usually, on the bottom camera, the only pixels that are usable are those in the largest circle you can inscribe within the image, so I usually recommend cropping it so that the width and height of the image are the same (make it a square rather than a narrow rectangle and then calibrate).  That way calibration doesn't need to collect data way out in those corners that will never be used, and you don't really lose any functionality.

>- where do I find Adv Camera Cal in Issues and Solutions?

>- I now have a residual footprint shown on the down camera - see 2nd pic.

I'll let Mark answer those

Tony

[mark maker]

>- where do I find Adv Camera Cal in Issues and Solutions?

Disable the advanced calibration the on the camera Advanced Calibration tab, then go to Issues & Solutions, then press Find Issues & Solutions. The advanced camera calibration should appear as a solution.

This is assuming you used the Issues & Solutions system properly through the milestones, solving all the required prerequisites, and are now in the Calibration milestone

>- I now have a residual footprint shown on the down camera - see 2nd pic.

It happens when you select a package with footprint on the Packages tab.

@Everybody, I also sometimes find that distracting, so if there are some ideas as to when it should and when it should not appear, please speak up 😁.

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/7c88d918-76d8-4734-a61b-ba3e677a4ef0n%40googlegroups.com.

[David Griffiths]

Thanks Ian & Mark. 

I can understand two Z heights for Down camera calibration but on Up camera?? 

The log file for today would be quite large but I will post it in the morning. 

The results I had before, as shown on the graphs, were tightly grouped dots on the 2nd graph and more or less straight lines on the top graph. Sadly I didn't take a screenshot.

Yes the cameras I have are higher res than I need (designing the machine has been a big learning curve). I will take your advice and turn on cropping (I think I had it on before changing the resolution). 

Mark I followed the steps you mentioned but I couldn't see the Advanced Camera Cal. Exactly what is it called? 

I think the footprint should only be shown when you are in the Packages tab. But having said that I haven't fully thought though the purpose of displaying the footprint. 

Cheers DG 

[mark maker]

> Mark I followed the steps you mentioned but I couldn't see the Advanced Camera Cal. Exactly what is it called?

1. must be in Calibration milestone or further
2. must have already done the preliminary camera calibration of the down-looking camera
3. must have done the nozzle offsets and Z calibrations
   
4.  the advanced camera calibration Enabled switch must be off.
   

For the down-looking camera, the preliminary camera and nozzle offsets and Z calibrations must have been done on both Z levels.

For the up-looking camera you must still have done the preliminary camera and nozzle offsets and Z calibrations of the down-looking camera first, but only on the first Z level.

In case you're wondering, why the preliminary camera calibration is still needed when we have the advanced camera calibration, it is a good question, but there are technical chicken-and-egg considerations that require a two-step approach distributed in two different milestones. The preliminary calibration is much simpler, but therefore also much more robust. If it fails, there is a usually an easy to understand problem with basic camera settings or motion that must be solved first.

So it is  a good thing to have stable preliminary camera performance asserted, before we even attempt to do the advanced calibration. The latter is much, much more ambitious in terms of what it addresses (camera tilt, lens distortion etc.). If the advanced calibration fails, it is hard to find the reason, if it is intermingled with basic camera or motion problems.

So in short:

Always use Issues & Solutions to guide through the calibration steps in the most robust order, carefully and constantly checking all the prerequisites for the next steps.

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/1fddf009-dac4-4f03-88ec-3bbee2a627d8n%40googlegroups.com.

[David Griffiths]

Hi Ian,  files are here:

https://www.dropbox.com/sh/7pikr9028w2wnpw/AAAjCQmYHycENeHlBdoXxwsYa?dl=0

Mark, Those are the steps I had already taken. I'll have another look if I get a chance today - very busy day though. Thanks for the response.

DG

[Litterio Andrea Guainella]

Hi Mark,

>  @Everybody, I also sometimes find that distracting, so if there are some ideas as to when it should and when it should not appear, please speak up

I would like to give my contribution as it is a very comfortable feature and I agree with you that it is sometimes distracting.

However I noticed this behavior (wanted?) if I'm in the packages section and I select a package that has a footprint, the representation appears correctly on the camera, however if I change tab it remains superimposed.

I noticed that the only way to make it disappear is, in addition to changing tabs, making a single click (with the left mouse button) anywhere on the camera as you can see in gifA below (maybe a refresh/repaint java issue?).

In my opinion it is a super convenient feature to always be able to print the fingerprint (for the packages that have configured it) during job phase, pay attention to "above all" the positioning direction (see rotation in job parameter).

This in reference to how the part in the feeder is configured (i.e tape, feeder, etc..) therefore it would be very useful to understand if the direction of rotation/positioning is correct.

All this maybe optionally activated from an icon in the toolbar or next to the menu for the link of the parts.

LAG

[mark maker]

Hi LAG,

thanks for having a look.

Unfortunately, your tests without a working and running camera is not conclusive there. The footprint is overlaid in a filter handler which paints on the frames broadcast by the running camera.

If you want to test this quickly, just backup/rename your machine.xml, OpenPnP will then create a new one with a simulated machine with simulated cameras.

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/91c8b52e-d475-4787-b2c3-5441cb575ba3n%40googlegroups.com.

[Ian Arkver]

Hi DG. It's actually Tony that asked for the diag info, not me. Hard to follow who says what in these threads sometimes, but Tony's the Adv Cam Cal guru.

CU
Ian

[Litterio Andrea Guainella]

Yes Mark,

I've tried and you are right...

With simulated camera when click to package the selected footprint appear to camera and when change tab footprint disappear.

When click to feeder, the footprint appear to camera but if change tab the footprint remaining to camera

LAG

[David Griffiths]

Hi Mark, I found the Adv Camera Cal in Issues and Solutions. Must have had a 'man look' before, but in my defence the fact that it is a 'Suggestion' might have made me skip over it. ;-)

Tony, files as requested are here: https://www.dropbox.com/sh/7pikr9028w2wnpw/AAAjCQmYHycENeHlBdoXxwsYa?dl=0

The second sequence is at the different Z, but that is not made clear in the prompts.  What purpose does the second Z make for the UP camera and how is the second Z configured?

Cheers DG

[tonyl...@gmail.com]

Looks like you attempted to run the calibration three times.  On the first two attempts, the model failed to converge to a solution.  Without going into a lot of detailed analysis of the log, I would guess that it was having a lot of problems correctly detecting the nozzle tip in those far corners of the image.  On your third attempt, it appears everything worked ok.  I don't know what might have changed between those runs - room lighting maybe?  I don't see anything that would indicate that the last calibration is bad.  The size of the residual errors look reasonable to me.  The map shows that larges errors are occurring in the corners which bolsters my speculation as to what the problem was with the first two attempts.

>The second sequence is at the different Z, but that is not made clear in the prompts.

It should say something like "Collecting at Z height 1 of 2" (or 2 of 2) as the data is being collected. I believe Mark has hard coded the second Z height to be 2mm above the first height. From a mathematical convergence standpoint, I think it would be better if that difference was larger but I don't think we have any easy way to deduce how much headroom the nozzle has.   In any case, we need to collect data at two different Z heights just because of all the unknowns that we are solving for (the same equations are used for both up and down looking cameras).   For instance, imagine trying to deduce the focal length of the camera lens by moving an object around in front of the camera without also knowing its distance from the camera.  We actually know neither the focal length nor the distance to the camera but by looking at the object at one unknown distance and then changing the distance by a known amount, we can deduce both.

Anyway, I'd still recommend you crop the image (using the controls on the Advance Calibration tab) to be a square (looks like your current resolution is 720x1280 so I'd crop it to 720x720) and then repeating the calibration.

Tony

[David Griffiths]

Thanks for the explanation Tony. I take my hat off to the mathematical gymnastics you perform! 

I have cropped the UP camera since the files I posted. I will post new files tomorrow. 

I think the earlier plots that I thought looked better had three extreme outliers that made all the other readings look better when the plots scaled. 

One thing I don't understand is with the camera res at 1920X1020 I was seeing about 40uM per pixel but at 1280x720 res I am seeing about 20uM per pixel. Seems back to front?? 

Cheers DG 

[tonyl...@gmail.com]

> One thing I don't understand is with the camera res at 1920X1020 I was seeing about 40uM per pixel but at 1280x720 res I am seeing about 20uM per pixel.

Is the field-of-view the same at both resolutions, i.e., how much of a ruler you can see at the two different resolutions?

[mark maker]

Side note: on my ELP camera the FullHD and the HD modes both use native pixels, i.e. the Units per Pixel (µm) should remain the same, i.e. the HD mode simply crops the sensor image, but allows for double the FPS. Other smaller resolution modes, however, scale the image.

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/12f9abc1-530c-4172-ab78-6394aa95eb0en%40googlegroups.com.

[tonyl...@gmail.com]

Yeah, the camera is probably doing something like internally cropping the 1920x1020 down to 640x360 and then up-sampling 2:1 to get the 1280x720.

Tony