Fiducial Pipeline Released

[Jason von Nieda]

Hi all,

The conversion of the fiducial system from internal code to CvPipeline has been completed and released to the develop / Latest branch. You can download it from http://openpnp.org/downloads/ as "Latest". 

More information about this change is available at https://github.com/openpnp/openpnp/blob/develop/CHANGES.md#2017-07-02.

Since this is a large change with ramifications for board placement, I have also released a new Maintenance version of OpenPnP without the change. This contains all fixes up the change right before the fiducial conversion. If you find a blocking problem with the fiducial change you can downgrade to this version by downloading the Maintenance version from http://openpnp.org/downloads/.

Please try the new system and let me know if you have any feedback!

Thanks,

Jason

[Marek T.]

Hi Jason,
I've tested it today.
My problem was mistaken detection of the fiducial from the neighboring boards instead of proper one. Added CircleMask with needed diameter and problem eliminated.
Second problem was that at 1Mpx (where more details) fiducials recognition didn't worked at all for me. It could be improved by changing the lightning (or exposure) but it was not stable, not universal and totaly sh*ty looking for human eye. So I've modified a bit the default pipeline parameters and now it's perfect. Good looking and detects every fiducials no matter how much are shining or dirty.
On the moment I can't see anything cons this option, great for me! :-). I've not played to much with other things than fiducials but haven't found anything not stable working in this version yet.

[Jason von Nieda]

That's great to hear! Thanks for reporting back!

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/ac9d27c2-1ff3-4a7d-91ad-19eaf53c8c24%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marek T.]

I'm not sure is it only in this version but:
Bottom camera/Transformation OffsetX and OffsetY, no matter the value is 1 or 0001 the executed correction is 0.01mm. Values containing dot or comma (0.01 or 0,01) are not accepted at all. Is it in agree with intentions out some bug? OffsetR works normally, i.e. 0.5 is accepted and means 0.5deg.
At occasion:
- is it problem to add zoom sliders for cameras view?
- is it problem to add on the move scale value 90 between 10 and 100 (often usable for manual rotation)?

[Jason von Nieda]

The offsets are in pixels, so they must be whole numbers.

You can zoom the camera view using the scroll wheel or two finger scrolling on a track pad.

Adding 90 to the jog increments would be fine, but we may want to consider instead adding a new slider for rotation and one for Z since all three tend to be in different ranges.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/fb43c5fd-4b85-4fad-8aa5-cdaa8f9314d5%40googlegroups.com.

[Marek T.]

Of course I know about the scroll mouse, but slider would be more convenient because much faster in situation when you want make quick really large zooming (or go back from large zooming). Or at least two switches "max zoom" and "min zoom" instead if not slider. I mean down looking camera mainly.

New sliders for R and Z good idea, but probably much more time required to do it. Adding 90 more primitive but probably much easier/faster to do until you add sliders in future.

Both my proposals are only suggestions how to make life easier but not critical if problematic...

[Team14]

Dear Jason,

I've just tested the fiducial detection base on a customized CV pipeline, but without any luck!
When I test the CV pipeline, I get a positive detection, but when I click on the fiducial check for the boards, I get the error, that the first fiducial could not be found.

The result stage is the same as in the default pipeline.




Here my pipeline:

<cv-pipeline>
   <stages>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageCapture" name="0" enabled="true" settle-first="true"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="13" enabled="true" prefix="bv_source_" suffix=".png"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.BlurGaussian" name="10" enabled="true" kernel-size="15"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskCircle" name="4" enabled="true" diameter="200"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="1" enabled="true" conversion="Rgb2Gray"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.Threshold" name="12" enabled="true" threshold="180" auto="false" invert="false"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.FindContours" name="5" enabled="true" retrieval-mode="List" approximation-method="None"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.FilterContours" name="9" enabled="false" contours-stage-name="5" min-area="4500.0" max-area="6000.0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskCircle" name="11" enabled="true" diameter="0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawContours" name="7" enabled="true" contours-stage-name="5" thickness="1" index="-1">
         <color r="255" g="255" b="255" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DetectCirclesHough" name="circle" enabled="true" min-distance="50" min-diameter="75" max-diameter="85" dp="1.0" param-1="80.0" param-2="10.0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskCircle" name="black image" enabled="true" diameter="0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawCircles" name="2" enabled="true" circles-stage-name="circle" thickness="2">
         <color r="255" g="0" b="18" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MinAreaRect" name="result" enabled="true" threshold-min="1" threshold-max="255"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageRecall" name="14" enabled="true" image-stage-name="4"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawRotatedRects" name="8" enabled="true" rotated-rects-stage-name="result" thickness="2" draw-rect-center="true" rect-center-radius="10" show-orientation="false"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="15" enabled="true" prefix="bv_result_" suffix=".png"/>
   </stages>
</cv-pipeline>

Have I missed something?

Kind regards, Karl

[Marek T.]

Have you set properly 0/0 of the board? When you send camera to this 0/0 the fiducial is located in area of the camera vision?

[Cri S]

You need to have a stage named "results" , not "result" and you need
to have return
a list of keypoints, example are simpleBlob or to convert contours to keypoints.

2017-08-16 12:19 GMT+02:00, Team14 <off...@team14.at>:
> Dear Jason,
>
> I've just tested the fiducial detection base on a customized CV pipeline,
> but without any luck!
> When I test the CV pipeline, I get a positive detection, but when I click
> on the fiducial check for the boards, I get the error, that the first
> fiducial could not be found.
>
> The result stage is the same as in the default pipeline.
>
>
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/6ed3cbcb-f12c-422b-8020-371004486b5d%40googlegroups.com.

[yaddatrance]

My guess is that the diameter detected circle is different from the diameter you set for the fiducial. I ran into that problem recently and had to nudge the setting for fiducial diameter in the footprint editor.

[Team14]

Thank you Cri S, using simple blob and renaming the stage to results with 's' helped.

[Team14]

Im using this new feature now, and it seems, there is a timing bug.

when I use the "Auto Setup", after clicking on the first part, the image seems to be taken too early (the preview flashes to white too early). The result is a very poor user experience, since the "Auto Setup" mostly doesn't work.

As a reference, here is an example cv pipeline:

<cv-pipeline>
   <stages>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageCapture" name="1" enabled="true" settle-first="true"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskRectangle" name="0" enabled="true" width="230" height="480"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="2" enabled="true" conversion="Bgr2Gray"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.BlurGaussian" name="predetect" enabled="false" kernel-size="9"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DetectFixedCirclesHough" name="results_" enabled="false"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DetectCirclesHough" name="results" enabled="true" min-distance="80" min-diameter="40" max-diameter="45" dp="1.0" param-1="80.0" param-2="10.0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="5" enabled="true" conversion="Gray2Bgr"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawCircles" name="6" enabled="true" circles-stage-name="results" thickness="1">
         <color r="255" g="0" b="0" a="255"/>
      </cv-stage>
   </stages>
</cv-pipeline>

Using this feeder now is really annoying - I do like much more the AdvancedLoosePartFeeder.

[Team14]

PS: a workaround is to move first as exactly to the center of the first part, before clicking on it.

[Jason von Nieda]

Thanks for the report, will look into this.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/14f5be24-1a3b-4c02-9de3-a5e6b5cf055b%40googlegroups.com.

[Team14]

I'm sorry, but I threw this into the wrong thead.
I was talking about the CV-pipeline for the ReferenceStripFeeder.

I have big troubles on setting up this one with the "Auto Setup". I think it has to do with my backlash compensation, which is set to a very slow speed for 0.5mm

When I turn this off, it works as expected, but with reduced precision.

Should I open an Issue on GitHub?

[Cri S]

Probably the m400 code is missing on backlash gcode section.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+unsubscribe@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/a19e9ac0-b7f6-4442-ac57-14b49f870c7b%40googlegroups.com.

[Richard Sim]

In that case, the CvPipeline you posted is quite wrong. Please refer to the default pipeline for ReferenceStripFeeder - you should be using DetectFixedCirclesHough as the output (name="results"), as the feeder expects circles not rects.

[Richard Sim]

Oops sorry, I was looking at the CvPipeline you posted in the previous message - I missed that you posted a different pipeline in another message.

Going by that CvPipeline, disable the DetectCirclesHough and re-enable DetectFixedCirclesHough then rename it back to "results" (there should be no reason to be using DetectCirclesHough in ReferenceStripFeeder, and if it's necessary then it means that your camera's units-per-pixel is set incorrectly). Other than that, I suspect that Cri is correct about the missing M400.

[Michael G.]

Just ran into this, too. Trying to setup a ReferenceStripFeeder with autosetup using the default pipeline but added a stage to write debug-image (see below).
It seems some pictures from the camera are taken too early? For testing I have set the settle time of downlooking camera to 1000ms but still having the effect of rolling shutter in images. A G4 P0 is present for the grbl controller, checked that. Any ideas why this happens? What can I test further?

<cv-pipeline>
   <stages>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageCapture" name="1" enabled="true" settle-first="true"/>

      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="2" enabled="true" conversion="Bgr2Gray"/>

      <cv-stage class="org.openpnp.vision.pipeline.stages.BlurGaussian" name="predetect" enabled="true" kernel-size="15"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DetectFixedCirclesHough" name="results" enabled="true"/>

      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="5" enabled="true" conversion="Gray2Bgr"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawCircles" name="6" enabled="true" circles-stage-name="results" thickness="1">
         <color r="255" g="0" b="0" a="255"/>
      </cv-stage>

      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="debug_write" enabled="true" prefix="feeder_debug" suffix=".png"/>
   </stages>
</cv-pipeline>

[Jason von Nieda]

Currently on vacation so won't be able to look at this until next week. One thing to check is whether the settle option is checked in the image capture stage. It should be.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/2f1fbfd1-3056-428d-ac39-da50f6bd6fed%40googlegroups.com.

[Richard Sim]

I would expect some (most) of those debug images to show a rolling shutter effect, due to the way the setup wizard works. The wizard runs the pipeline on the raw camera feed so that it can display helpful information (various colour circles) in the camera view in realtime - so saving an image every time the pipeline runs will indeed result in a lot of images with blur/rolling shutter visible. However, the actual wizard part that's handling the interactions (Click first part, second, etc) should be doing the normal settle-and-capture behaviour.

Michael, what incorrect behaviour are you actually seeing in the wizard? If the camera isn't moving and is over the tape, do the holes show as correctly detected in the camera view? Something else to try - when you click the Auto Setup button, hold down Alt/Option at the same time as you click the button, which will show some additional coloured debug circles in the camera view, which may help highlight any issues you have.

[Michael G.]

Am Mittwoch, 23. August 2017 23:33:59 UTC+2 schrieb Richard Sim:

I would expect some (most) of those debug images to show a rolling shutter effect, due to the way the setup wizard works. The wizard runs the pipeline on the raw camera feed so that it can display helpful information (various colour circles) in the camera view in realtime - so saving an image every time the pipeline runs will indeed result in a lot of images with blur/rolling shutter visible.

Ah, ok, that makes sense and explains why I got so many pictures after autosetup on the hard drive if using ImageDebugWrite in the pipeline.

 

However, the actual wizard part that's handling the interactions (Click first part, second, etc) should be doing the normal settle-and-capture behaviour.

SettleFirst is enabled in the pipeline. For testing purposes I set the settle time to 1000ms (was using the default 250ms, using the recommended ELP cameras).

 

Michael, what incorrect behaviour are you actually seeing in the wizard? If the camera isn't moving and is over the tape, do the holes show as correctly detected in the camera view?

Holes are detected stable, but many other points, too. The other points don't seem to be stable and change quite often (see the two pictures).

      

 

Something else to try - when you click the Auto Setup button, hold down Alt/Option at the same time as you click the button, which will show some additional coloured debug circles in the camera view, which may help highlight any issues you have.

The debug-circles during autosetup seem well to me. green if +-1 hole from current hole away otherwise blue, red or yellow.

 Attached three screenshots with the result of the autosetup. I'm pretty sure I clicked on holes in the setup progress ;) The result was made with a settle time of 1000ms and it seems as if the time was respected correctly by the setup routine.

Sometimes the autosetup works as expected, sometimes not. I can't recognize a system behind... Any ideas how to improve? Have I set something wrong?

thank you very much!

[Michael G.]

additionally right after a click on a hole in camera view I get attached messages sometimes. Let me add that the autosetup doesn't fail every time but quite often.

[Jason von Nieda]

Please keep the reports coming. I am on vacation and will address all this on Monday.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/cceb41f2-0ffd-49a5-b729-88e46abfe542%40googlegroups.com.

[Richard Sim]

ok a couple of issues I see:

1) You're meant to click the centre of the *component*, not the *hole*. Clicking the hole will actually make it ignore the hole (due to it checking for holes only a minimum distance from the selected component, to avoid picking up any holes in the component itself).

2) Having two pieces of tape visible in the camera at once is an issue currently; there isn't very good logic around selecting the correct tape.

3) The exception you're seeing occasionally is a bit odd - it looks like your pipeline is sometimes missing the "results" stage, or at least the stage isn't actually outputting anything.

So:

1) Click the components in the tape instead. :)

2) This is next on my list to address - make the auto setup wizard smarter about selecting the correct piece of tape when there are multiple in view. What's there now is just a direct port of the old system.

3) I'll add a check around that code, or more appropriately look into how that can happen in the first place (I was under the impression that code was safe, but clearly it isn't always?)

Thanks for the detailed report!

[Michael G.]

Hello Richard,

Am Donnerstag, 24. August 2017 21:45:27 UTC+2 schrieb Richard Sim:

ok a couple of issues I see:

1) You're meant to click the centre of the *component*, not the *hole*. Clicking the hole will actually make it ignore the hole (due to it checking for holes only a minimum distance from the selected component, to avoid picking up any holes in the component itself).

Clicking on the holes worked for me in earlier versions, so I never just thought about clicking the components - totally ignoring the small hint to click the component in the camera view...

Maybe I just had luck - used the machine not that much until today, since I'm still setting it up. The feeders are the next to come now.

2) Having two pieces of tape visible in the camera at once is an issue currently; there isn't very good logic around selecting the correct tape.

Okay, for now I added a mask to the pipeline, just to be sure this is not the problem. Here the current pipeline:

<cv-pipeline>
   <stages>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageCapture" name="1" enabled="true" settle-first="true"/>

      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskRectangle" name="0" enabled="true" width="600" height="800"/>

      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="2" enabled="true" conversion="Bgr2Gray"/>

      <cv-stage class="org.openpnp.vision.pipeline.stages.BlurGaussian" name="predetect" enabled="true" kernel-size="25"/>

      <cv-stage class="org.openpnp.vision.pipeline.stages.DetectFixedCirclesHough" name="results" enabled="true"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="5" enabled="true" conversion="Gray2Bgr"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawCircles" name="6" enabled="true" circles-stage-name="results" thickness="1">
         <color r="255" g="0" b="0" a="255"/>
      </cv-stage>

      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="debug_write" enabled="false" prefix="feeder_debug" suffix=".png"/>
   </stages>
</cv-pipeline>

 

3) The exception you're seeing occasionally is a bit odd - it looks like your pipeline is sometimes missing the "results" stage, or at least the stage isn't actually outputting anything.

Here I got another one, just popped up right after I clicked in the camera view:

 

So:

1) Click the components in the tape instead. :)

 

Okay, roger that :)
 

2) This is next on my list to address - make the auto setup wizard smarter about selecting the correct piece of tape when there are multiple in view. What's there now is just a direct port of the old system.

3) I'll add a check around that code, or more appropriately look into how that can happen in the first place (I was under the impression that code was safe, but clearly it isn't always?)

If you have something to test for me I will do.

 

Thanks for the detailed report!

Thank you for looking into this!

[phon...@gmail.com]

For removing false/instable hough circles, insert canny processing, example using you'r image:

Here i have inserted additional stage of simpleblob filter that detect false positive.

Mark this stage with result and if present , remove the circles near this points.

The keypoints are drawed with blue color, small circles in this image.

If you do adaptive thresholding, you could separate strips. Exemaple here:

and the resulting strips, i have selected it manually from the two strips found, 

Using thresholding it is possible to check what pocket is empty or not.

I doin't know if this or full automatic strip setup is really wanted or just complicates it,

because of general difficult understanding of vision pipeline potential on people that 

don't have low level vision background.