Fiducial vision improved for reflective/HASL fiducials

[ma...@makr.zone]

Hi all,

I've just improved fiducial vision, introducing a new Parallax View-Points method, where the fiducial is detected at an angle from two opposite and symmetric view-points. This will eliminate the old problem that the dark camera lens is reflected in the fiducial, instead of the bright light diffuser. A cheap alternative to co-axial lighting.

Obviously, taking two images instead of one will be slightly slower, but on the other hand I optimized fiducial vision to stop the iterative multi-pass detection as soon as the fiducial is detected accurate enough.

Other aspects of the Fiducial Vision have also been improved.

See the details here:

https://github.com/openpnp/openpnp/pull/1565

Available in the OpenPnP test version. Allow some minutes to deploy.

Any help in testing this in your real world situations is very welcome! :-)

_Mark

[Zdenko Stanec]

Cheers Mark,

Looks promising as always, I will try to test this over the weekend and give you some info.

Zdenko,

[Blowtorch]

You are such a star Mark, thank you for this.  I always had problems with detecting the fiducials on my panels, and planned to change the lighting for the camera, but it is such as hassle to get to those parts.  Will try and revert back as soon as time permits.

[mark maker]

@Blowtorch, @Zdenko, others,

have you been able to try? Any results?

I would also appreciate some ImageDebugWrite images, as personally I don't have any real HASL PCBs. Edit the pipeline and enable the "deb0" stage:

Then do some fiducial vision, with, and without parallax.

Then you'll find images in the  $HOME/.openpnp2/org.openpnp.vision.pipeline.stages.ImageWriteDebug folder. Please send some.

I can then also illustrate the Wiki entry better.  Thanks a lot! 😎

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/1ba23588-47f9-48c9-8b7d-8a12382d63a7n%40googlegroups.com.

[Zdenko Stanec]

Hey Mark,

Apologies Mark, had a lot of work lately. 🤠

I will need to recalibrate the Top camera as the second PnP machine is still in the build phase, after that I will do some tests and share some pictures here.

Zdenko,

[Balázs Kiss]

Hi Mark,

we are also highly interested in the change you delivered, but are now unable to test it. We plan to deploy the new test version in 2 weeks in a new, more secure environment and would gladly share our experiences also. Thank you for your efforts in advance!

Best regards,

Balazs

[Josef Plasil]

Hi Mark,

I can test it immediately, but how could I switch on the Parallax View-Points method? Is this somewhere in Fiducial Vision Pipeline? Otherwise the process of fiducial points detection seems to me the same as previously, i.e. camera is focused on fiducial in center of view. BTW, I have got the latest OpenPnP test version (downloaded today) installed, but last date in change list is 2023-05-03 :-o. Or is there any download link to have more actual version, please?

Regards, Josef.

Dne pátek 30. června 2023 v 7:42:33 UTC+2 uživatel kbhu...@googlemail.com napsal:

[mark maker]

Hi Josef,

not every feature is listed in the Release Notes (especially in the test version). Look at the date at the top.

You'll see the new Parallax Diameter setting in the Fiducial Locator settings:

Instructions for Use are here:

https://github.com/openpnp/openpnp/pull/1565

_Mark

You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/20rW8feVfbI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/ca41c5c3-e668-4bf5-9d95-4816cbcd630dn%40googlegroups.com.

[Josef Plasil]

Dear Mark,

thanks. OK. Confirmed, everything works like a charm.

I'm sending pictures:  https://we.tl/t-nvXTBAazxm  they are too big to send them attached to this message ... and with picture of Fiducial Vision Pipeline settings, which works for me perfectly, both - with or without Parallax View-Points method.

Regards, Josef.

[Zdenko Stanec]

Hey Mark, 

I tested the Parallax feature which worked nicely for me.

Sharing some photos, please find them on this link: https://we.tl/t-URI3INIV54

- Parallax_0deg - 5, 10, 15, 20, 25mm

- Parallax_45deg - 5, 10, 15, 20, 25mm

- Parallax_90deg - 5, 10, 15mm

- Parallax with Diag - 10, 15, 20, 25mm

- one Normal photo, and Normal photo with Diag

How does "Max Vision passes" work, if I understand it right from the https://github.com/openpnp/openpnp/pull/1565,  it should only "try" a "n" amount of time if the fiducial is not recognized in the process right?

Zdenko,

[Zdenko Stanec]

It seems some pictures are broken on the previous link, sharing a new link: https://we.tl/t-ryfdsCD3m5

Zdenko,

[Jim Young]

Mark, thanks for the new feature! I was having some problems with getting reliable fiducial sensing on some boards with highly reflective surfacing and this new sensing mode works very reliably.

I do have one question - What is the purpose of the Parallax Angle setting? If Parallax Diameter determines the distance between the two readings, with the fiducial in the center, this also creates an angle between the known fiducial position and the sensing position. I just left Parallax Angle at zero, which seems to work fine.

[mark maker]

Thank you Josef!

It seems your diffuser is very diffuse and covering very close to the camera (nicely done, please share your construction!) . Your fiducials are a bit roughed up and have quite regular edge beveling that reflects outwards, so the vision surely works even without parallax.

But still: it is shown that with parallax it appears brighter and fuller.

Perhaps one day you get a PCB with more reflective HASL (I've seen examples by users with almost mirror-quality), or where the beveling is very asymmetric (seen this too), so the parallax option could still become useful. 😎

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/3a304ee4-8148-45ed-872f-962dd3470de6n%40googlegroups.com.

[mark maker]

Thank you Zdenko,

your diffuser seems to have a large hole, right?

You need to go all the way out to get half decent reflection, and for large parts of the center it is almost black:

In fact it is so black that it contrasts enough with the PCB material, so fiducial vision works too. 😂

If you ever want to improve the diffuser, move it as close a possible to the camera lens front without causing shadows, and then make the smallest hole for the camera to peek through.

> How does "Max Vision passes" work, if I understand it right from the https://github.com/openpnp/openpnp/pull/1565,  it should only "try" a "n" amount of time if the fiducial is not recognized in the process right?

The vision must always give a recognized fiducial, otherwise it will fail immediately, so it's not about "not recognizing", it's about iteratively improving the result.

The "ideal" behind all this is symmetry. With symmetry we can eliminate many kinds of errors, because they happen on both side equally, so the can be cancelled out.

With the normal fiducial detector (without parallax) we want the fiducial to be in the center of the camera, or more importantly perpendicularly below the camera position. This way we are sure that any imprecision in the assumed scale (Z of the PCB and/or in the calibration of Units per Pixel), or distortions of the camera lens or whatever do not matter, because however large or small these errors are, they can be assumed to be symmetric, i.e. the same on both sides of the fiducial.

With the parallax fiducial detector, this is even more important, obviously, because these errors (if present) will become quite large due to the large distance, so we absolutely need to cancel them out.

After a vision pass where we detect the fiducial (with either method), we get the detected location. If that location is quite far away from what we expected, we can no longer assume that symmetry helped us cancel out the errors. So we re-center the camera to the detected location and repeat. The errors due to missing symmetry will typically be small in comparison with how far we were off, so we can assume that each time we repeat this, the result will get better and better.

The "Max Vision passes" limits how many times this can happen. However, ideally it will stop before that, due to the detected offsets being small enough.

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/2c740ae9-2547-4b7a-9242-5e89a71b9267n%40googlegroups.com.

[mark maker]

> I do have one question - What is the purpose of the Parallax Angle setting?

The Parallax Angle determines the angle at which the opposite view-points are spaced from each other in the X/Y plane. Choose an angle which favors the faster axis of your machine: 0° favors X, 90° favors Y.

Other angles might be used to target a specific LED pair in a LED ring reflection.

Using just a random image from the group here for illustration: As you see, some diffusers cannot completely equalize the brightness (that's okay). So it might make sense to rotate the parallax so it puts a single LED reflection smack on the fiducial.

(image by Philip)

_Mark

--
You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/20rW8feVfbI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/b4d5ac90-93bc-4c0c-8236-371617ea3e8fn%40googlegroups.com.

[Zdenko Stanec]

Hi Mark,

I had a similar design to what you described in the past and I was not so happy because I can see the reflection of the LED-s on the PCB surface. :D

This was the design, close as possible to the camera with M37 thread to screw and test some standard lenses. (there is no diffusor on this picture but I had one).

After that, I went with this design which pretty much eliminated reflection in the middle of the PCB, still with M37 thread to test some lenses.

Zdenko,

[Jim Young]

"The Parallax Angle determines the angle at which the opposite view-points are spaced from each other in the X/Y plane. Choose an angle which favors the faster axis of your machine: 0° favors X, 90° favors Y."

It was the X,Y part I didn't get. Makes perfect sense now. Thanks!

[Josef Plasil]

Hi Mark,
thanks for your appreciation :-), I am sending the pictures of construction attached.
The diffusor is made from old LCD TV, where such plastic material (structured plastic glass of 2mm thickness) is available. I milled it in my milling machine, therefore it is accurate. The diffusor is "sticked" by magnets on screws. Everythig is clearly visible at pictures ;-).
I can see two failings in my design up to the moment:
1) Y axis has one stepper motor driving also shaft for 2 linear guide rails (you can see it at picture 1b...) - this is most probably (in combination with TMC2209 driver for this motor,  flexibility of GT2 6mm chain, and 12V of power supply voltage :-o) the reason for error of approx. 0.15 mm in Y movement (you recognized it in other thread ;-) ).

2) strange behaviour of bottom camera components position / rotation compensation ... the components are placed not so accurately (rotation is OK and there are  different position deviations on the PCB approx 0.5 mm). And the issue is here despite the fact button "Test Alignment" works perfectly and also "Position the camera at the placement's location" button works very well. I still don't know, how could I dignose this issue, but I am working on it. If you have some idea, it will be appreciated.

Regards, Josef.

Dne neděle 2. července 2023 v 13:23:23 UTC+2 uživatel ma...@makr.zone napsal:

[Josef Plasil]

I am sending the view on bottom camera with backlight yet - it is completely same as top one. I am using successfully bottom camera without diffuser , I have just decreased exposition.
Regards, Josef.

Dne neděle 2. července 2023 v 17:16:48 UTC+2 uživatel Josef Plasil napsal:

[Jorgen Nielsen]

Hi Mark

Sorry I have been away then busy catching up with other  stuff.  I will test this coming week.

You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/20rW8feVfbI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/2a4aa374-bb13-5d34-349c-e7fd61a6de88%40makr.zone.

[Blowtorch]

Hi Mark

Installed latest test version 2023-07-08_18-14-24.8a4e93d

New fiducial locator vision is great - for the first time, I find it fiducials are useable.  Before I was forced to set placement using components.

I generated 3 examples, shared with you via dropbox, each set of pics in own subfolder.  What is interesting with the panel I used, is that the break away rails are covered with dots the same size as fiducials, but it finds them 100% anyway.  Happiness.  Thank you.

Would you like me to do another panel and generate diags?

Regards Jorgen 

[Blowtorch]

Hi Mark

OK I did a second panel, uploaded all fiducial images to dropbox folder shared with you.

Regards Jorgen

[mark maker]

Hi Blowtorch,

> New fiducial locator vision is great - for the first time, I find it fiducials are useable.  Before I was forced to set placement using components.

Very glad to hear that  😅

> I generated 3 examples, shared with you via dropbox, each set of pics in own subfolder. 

Thanks for trying to share the images, however (as already explained in the PM), I would not like to register with Drop Box, so please either use a different service or attach the best few images (before/after) directly to posts here.

> What is interesting with the panel I used, is that the break away rails are covered with dots the same size as fiducials, but it finds them 100% anyway.  Happiness.  Thank you.

Then all is good 😎, but still, I'm also interested. Do you use the latest stock pipeline? With the DetectCircularSymmetry stage?

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/6ad1d2c2-08ec-41cd-9f76-cca0488052d4n%40googlegroups.com.

[Blowtorch]

OK herewith some pics attached.  The others are similar, shout if I should upload more?

Re pipeline, not sure how to select different pipelies.  I do remember having to create a part and package to define the fiducial, but that was last year sometime and I really cannot remember how I did it.  If I edit the pipeline, here is the text:

<cv-pipeline>
   <stages>
      <cv-stage class="org.openpnp.vision.pipeline.stages.CreateFootprintTemplateImage" name="template" enabled="true" footprint-view="Fiducial" minimal-image-size="false" x-offset="0.0" y-offset="0.0" rotation="0.0" max-width="0.0" max-height="0.0" property-name="footprint">
         <pads-color r="255" g="255" b="255" a="255"/>
         <body-color r="0" g="0" b="0" a="255"/>
         <background-color r="0" g="0" b="0" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="debug_template" enabled="true" prefix="fidloc_template_" suffix=".png"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="template_gray" enabled="true" conversion="Bgr2Gray"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageCapture" name="image" enabled="true" default-light="true" settle-option="Settle" count="1"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="image_gray" enabled="true" conversion="Bgr2Gray"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="debug_original" enabled="true" prefix="fidloc_original_" suffix=".png"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MatchTemplate" name="match_template" enabled="true" template-stage-name="template_gray" threshold="0.699999988079071" corr="0.8500000238418579" normalize="true"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageRecall" name="recall_image" enabled="true" image-stage-name="image"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawTemplateMatches" name="draw_matches" enabled="true" template-matches-stage-name="match_template"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertModelToKeyPoints" name="results" enabled="true" model-stage-name="match_template"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawKeyPoints" name="draw_keypoints" enabled="true" key-points-stage-name="results"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="debug_results" enabled="true" prefix="fidloc_results_" suffix=".png"/>
   </stages>

</cv-pipeline> 

[mark maker]

Thanks! Nice case.

First, I think you should add more parallax, the reflection in the green PCB is more intensive out there, and the fiducial is still at "half moon", it should be "full moon" 😁:

> Re pipeline, not sure how to select different pipelines.

This is the old pipeline. Can you please try with the new stock pipeline?

How?

1. Make sure to have View / Selections in Tables / Linked enabled.
2. In the Placements table, select the fiducial in question.
   

Go to the Parts tab, the fiducial part should be selected, and then select the  Stock Fiducial Vision Settings:

Go to the Fiducial Vision Settings detail tab and press the Specialize for ... button:

Then press Pipeline Edit and enable the "deb0" and "deb1" stages, for debug image writing:

Also note the presence of the DetectCircularSymmetry stage that indicates this is in-deed a modern stock pipeline.

Then retry with this pipeline. The source images should now be in color (that's how I like them for the Wiki 😎).

Thanks for your help in testing and documentation! 💯

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/5b2d2a78-744b-4750-8e70-8e6c41cb0756n%40googlegroups.com.

[Blowtorch]

Hi Mark

OK, so pulled in the new pipeline.  Set Parallax Diameter to 12, as manually jogging to the left and right revealed the fiducial was well lit at 5mm, but better at 6mm offset.  I placed a ruler with mm marks so you can see exactly what is happening.

Thank you for all the help you have provided!

Regards Jorgen

[mark maker]

Wow, now that fiducial is truly "full moon". 😎

But it still uses the old pipeline, as is evident by the rectangles with scores.

And once you've chnaged the pipeline, please also provide

• Source image of the centered fiducial (no parallax).
  
• Source (not result) image of the parallax fiducial
  

Thank you, 😁

_Mark.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/3709607c-df95-4ef9-9e3c-7f12759661ebn%40googlegroups.com.

[Blowtorch]

Not sure now how I had selected the pipeline previously, but I did do edit and see the correct items.  I think I updated package but then did check from job, where the job maybe had the old defined?

Anyway, did it all again, and here is something interesting.  Using new pipeline, and doing a job based, whole panel fiducial check of all 4 fiducial, the system does NOT correctly find all 4.  It does find something, and end successfully, but with the board location being offset incorrectly.  Not good if a real job!

Now, using the old pipeline, it does find all 4 just fine.  The problem, is that one panel off mine, with the light green dots.  This fools the new pipeline, but not the old one.

The issue now, is that 1 check is generating 29MB of pictures.  I tried running a "resize", but that only brings it down to 15MB.  Can I select specific pics for you and upload, or should I zip and send whole lot?

[Blowtorch]

snapshot of fiducial, centred, and snapshot of fiducial, offset by 6mm

[Blowtorch]

Results from testing fiducial, starting +- 0.5mm from actual fiducial... 

[mark maker]

Yeah, I feared as much. That's why I wanted you to try, honestly. 😇

As you correctly guessed, the problem is the many very similar green-coated dots and it seems those are equally attractive for DetectCircularSymmetry.

The "clean" way to adjust this is by reducing the fiducial locator  Max. Distance to be less than half the distance to the nearest green-coated dot of your PCB. From your images, and assuming the typical 1mm fiducial and 2mm mask diameter, it seems the nearest green dot is about 1.5mm away, so ~0.7mm is the Max. Distance you can live with:

Go to Machine Setup / Vision / Fiducial Locator and adjust it accordingly, press Apply:

So you need to make sure, you got your