Bottom Vision fails, I cannot find the reason (R2012 metric)

[Team14]

Hi,

can someone please help me?

I'm searching for the solution for some hours.

I've this pipeline for the bottom vision:

<cv-pipeline>
   <stages>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageCapture" name="9" enabled="true" settle-first="true"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="13" enabled="true" prefix="R2012_" suffix="_bottom1.png"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="2" enabled="true" conversion="Bgr2Gray"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.Normalize" name="11" enabled="true"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskCircle" name="7" enabled="true" diameter="150"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.BlurMedian" name="5" enabled="true" kernel-size="15"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.Threshold" name="1" enabled="true" threshold="70" auto="false" invert="false"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.FindContours" name="3" enabled="true" retrieval-mode="List" approximation-method="Simple"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.FilterContours" name="6" enabled="true" contours-stage-name="3" min-area="1000.0" max-area="2000.0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MinAreaRectContours" name="result" enabled="true" contours-stage-name="6"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageRecall" name="8" enabled="true" image-stage-name="9"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawContours" name="12" enabled="true" contours-stage-name="3" thickness="3" index="-1">
         <color r="233" g="30" b="30" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawContours" name="4" enabled="true" contours-stage-name="6" thickness="2" index="-1">
         <color r="10" g="209" b="39" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawRotatedRects" name="10" enabled="true" rotated-rects-stage-name="result" thickness="1" draw-rect-center="true" rect-center-radius="5" show-orientation="true">
         <color r="255" g="0" b="255" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="14" enabled="true" prefix="R2012_" suffix="bottom_2.png"/>
   </stages>
</cv-pipeline>



The debug images are attached.

The pipeline works with minor adjustments in the FilterContours stage for R1608 (metric) perfectly.

When I pick up the part from the feeder and then click "Test Alignment" in the "Parts Tab", I get the error message attached.
"Bottom Vision Alignment failed for part R2012-10R on nozzle Nozzle1. No result found.

The same error comes druing job processing.

I have no idea any more, where I should look for differences.

The bottom vision works perfectly for the R1608 parts.

Thanks in advance,
Karl

[Cri S]

You have a highly dynamic (instable) pipeline and have omitted to blur
the image before
color to gray conversion

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openpnp/6ae6dd0d-d6ac-45b3-bf48-f4ff52c2970b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Karl Zeilhofer]

Perhaps i didn't express myself clear enough.

When I do the setup, it works perfectly on each refresh. Also the debug images show the detected rectangle, independently of running the pipeline manually in the pipeline editor or with the test alignment button. But I get allways this exception when I hit that button or during automatic job processing.

[Cri S]

Then check the debug images that are generated during automatic job run.
As example it is possible that settingTime is wrong or something
similar llike no m100
on gcode setup and as result the component totally or partially
missing or AGC gives
totally different illumination, or just camera lag because of new
stripfeeder code.
I have added noise and altrought near pixel limit, the image is stable and my
initial assumtion was wrong.

This is simpler pipeline that don't filter in case if it's because AGC.

<cv-pipeline>
<stages>
<cv-stage class="org.openpnp.vision.pipeline.stages.ImageRead"
name="0" enabled="true"
file="C:\Users\xxx\Downloads\R2012_8509766091084345810_bottom1.png"/>
<cv-stage
class="org.openpnp.vision.pipeline.stages.BlurGaussian" name="9"
enabled="true" kernel-size="5"/>

<cv-stage
class="org.openpnp.vision.pipeline.stages.ConvertColor" name="2"
enabled="true" conversion="Bgr2Gray"/>
<cv-stage class="org.openpnp.vision.pipeline.stages.Normalize"
name="11" enabled="true"/>
<cv-stage

class="org.openpnp.vision.pipeline.stages.BlurGaussian" name="13"
enabled="true" kernel-size="7"/>
<cv-stage class="org.openpnp.vision.pipeline.stages.MinAreaRect"
name="result" enabled="true" threshold-min="100" threshold-max="255"/>

<cv-stage class="org.openpnp.vision.pipeline.stages.ImageRecall"
name="8" enabled="true" image-stage-name="9"/>
<cv-stage

class="org.openpnp.vision.pipeline.stages.DrawRotatedRects" name="4"
enabled="true" rotated-rects-stage-name="result" thickness="2"
draw-rect-center="true" rect-center-radius="5"
show-orientation="false"/>
</stages>
</cv-pipeline>

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/c01c5068-085a-4897-9fed-6bc145d2d773%40googlegroups.com.

[Karl Zeilhofer]

Thanks Cri, for your improved pipeline - i will give it a try, but there is only few hope.
The debug images i've provided, where created during the "live test" with the "Test Alignment" button.
So it is obvious, that the pipeline found a result, but anyway I recieve this exception.

Could it be related to a missing Pipeline.release()?

In the HeapFeeder we also have another quite strange phenomenon, that I need there some "special dummy move" of the head, before calling pipeline.process()
If I skip this dummy move, the pipeline doesn't return any results - I have still do some investigations there with debug images.

Could it be, that all this is somehow related to a thread synchronization problem?

[Cri S]

NO
NO
You don't have understand my response.
If you look at the real debug images produced during job run you
probably see the component and nozzle is missing from image or at
least from image center.
This is because wrong settings of settleDelay or missing M100
statement on Gcode driver
if you are using smoothie. Annother source of problems could be the
camera delay lag
generated by the StripFeeder Wizard if the other two settings are
correct, but all have the
same resulting debug image type, missing component and nozzle from image center.

The problem with skipped setup steps like camera delay test is, that
it cannot be tested
after camera is installed on machine and without that test you are
forced to guess a time
and add a lot of security timing to it degrading cph performance.

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/8ca75e00-d94c-468b-a453-6b6638d62302%40googlegroups.com.

[Team14]

Cri, it seems, we don't understand each other.

I will try agian to explain the situation:

I get the exception even then, when the saved debug-images (captured and rendered rotated rect) show a perfect result.
I have allways a 1000ms settle time for the camera, just to be very sure, any motion and lighting has stabilized, and the delay of the video stream has been passed.
We are at the point of getting everything working. Speed optimization comes in the next step.

Greets,
Karl

[Team14]

PS: attached are images from a real cycle:
turn on light, move to camera, process the pipeline, turn off the light.

As you can see: there is no motion-blur, there is no under or over exposure (since the camera is configured with a constant exposure, also the white balance is set manually), the nozzle is perfectly centered.

PPS: the issue with the "needed" dummy move was solved by the pipeline.release()

[Team14]

PPPS: in the ReferenceBottomVision.findOffsets() the pipeline.release is missing!
I think this is the source of our problems... I'll try to fix it here.

see this issue:
https://github.com/openpnp/openpnp/issues/631

[Cri S]

As you have seen on bottom2, the result have be found..
If you run prerotate there are two pipeline runs and 1000ms can be too
low if m100 is missing from gcode.
Can you confirm you use M100 and for rotation too ?
Please check the situation with and without prerotate.

I suspect without prerotate it works and with prerotate it don't work.
With prerotate there should be 4 images, not just 2 and maybe you just
don't look at
the correct images.

If this info don't produce anything, please post the machine.xml file
and tell what motion controller you use.

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/4e347f8e-fa7b-4a98-a00b-9880057511fc%40googlegroups.com.

[Marek T.]

If you go with the part from the feeder settle time or M100 are important of course. But standing with part over the camera you can use Test Allignment switch how many times you want too. And do you always get this error in that situation? If it is so then it's not settle time or missing M100 in my opinion.

[Jason von Nieda]

A failure to release a pipeline should not cause a result failure. I think the problem lies elsewhere. In addition, I’m not sure what the purpose of the M100 is that is being referred to. M100 is not common, I suspect you mean M400.

I am away from my desk for the next 8 hours or so. I will look into this issue when I get back and will try to help.

Jason

On Tue, Oct 17, 2017 at 6:31 AM Marek T. <marek.tw...@gmail.com> wrote:

If you go with the part from the feeder settle time or M100 are important of course. But standing with part over the camera you can use Test Allignment switch how many times you want too. And do you always get this error in that situation? If it is so then it's not settle time or missing M100 in my opinion.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/3148ae76-e66e-4d4f-9db3-8bd9273df9d2%40googlegroups.com.

[Cri S]

sorry, yes, i'm work with linuxCNC or Grbl, and use G4P0, and
remembered the 3d printer
equivalent wrong, naturally it is M400 and not M100 on smoothie, i think.

> https://groups.google.com/d/msgid/openpnp/CA%2BQw0jy%2Br3X8JikzD_JGRmo7JjEkSuX8R27p6kgL3szQKUWVeA%40mail.gmail.com.

[Marek T.]

Suggested by Cri with this M100 :-).
But as Karl it described at beginning, it doesn't smell like too early moment of the part "scanning". Specially when he says that has no problem with other size part...

[Team14]

Regarding the M100, M400:
TinyG supports no M100 or M400 command.
I'm working with the TinyG, the MOVE_TO_COMPLETE_REGEX is .*stat:3.*

Here is an example log, of a long distance run (with backlash compensation enabled):

2017-10-17 17:19:27 ReferenceNozzle DEBUG: Nozzle1.moveTo((477.660000, 369.770000, 0.000000, 0.000000 mm), 1.0)
2017-10-17 17:20:38 ReferenceHead DEBUG: H1.moveToSafeZ(1.0)
2017-10-17 17:20:38 ReferenceNozzle DEBUG: Nozzle1.moveToSafeZ(1.0)
2017-10-17 17:20:38 ReferenceCamera DEBUG: USB Down Camera.moveToSafeZ(1.0)
2017-10-17 17:20:38 ReferenceActuator DEBUG: Ventil.moveToSafeZ(1.0)
2017-10-17 17:20:38 ReferenceActuator DEBUG: Drucksensor.moveToSafeZ(1.0)
2017-10-17 17:20:38 ReferenceActuator DEBUG: Ueberlastschalter.moveToSafeZ(1.0)
2017-10-17 17:20:38 ReferencePasteDispenser DEBUG: D1.moveToSafeZ(1.0)
2017-10-17 17:20:38 ReferenceCamera DEBUG: moveTo((92.340000, 0.230000, NaN, 0.000000 mm), 1.0)
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(G1 X0.5000 Y0.5000   F16000, 7000)...
2017-10-17 17:20:38 GcodeDriver TRACE: [/dev/ttyUSB2] >> G1 X0.5000 Y0.5000   F16000
2017-10-17 17:20:38 GcodeDriver TRACE: [/dev/ttyUSB2] << tinyg [mm] ok>
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 G1 X0.5000 Y0.5000   F16000, 7000) => [tinyg [mm] ok>]
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(G1 X0.0000 Y0.0000   F480, 7000)...
2017-10-17 17:20:38 GcodeDriver TRACE: [/dev/ttyUSB2] >> G1 X0.0000 Y0.0000   F480
2017-10-17 17:20:38 GcodeDriver TRACE: [/dev/ttyUSB2] << tinyg [mm] ok>
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 G1 X0.0000 Y0.0000   F480, 7000) => [tinyg [mm] ok>]
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand((G1 X0.0000 Y0.0000   F16000), 7000)...
2017-10-17 17:20:38 GcodeDriver TRACE: [/dev/ttyUSB2] >> (G1 X0.0000 Y0.0000   F16000)
2017-10-17 17:20:38 GcodeDriver TRACE: [/dev/ttyUSB2] << tinyg [mm] ok>
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 (G1 X0.0000 Y0.0000   F16000), 7000) => [tinyg [mm] ok>]
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => []
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:38 GcodeDriver TRACE: [/dev/ttyUSB2] << posx:397.184,posy:307.490,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => [posx:397.184,posy:307.490,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5]
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => []
2017-10-17 17:20:38 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:39 GcodeDriver TRACE: [/dev/ttyUSB2] << posx:293.890,posy:227.551,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => [posx:293.890,posy:227.551,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5]
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => []
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:39 GcodeDriver TRACE: [/dev/ttyUSB2] << posx:191.651,posy:148.427,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => [posx:191.651,posy:148.427,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5]
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => []
2017-10-17 17:20:39 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:40 GcodeDriver TRACE: [/dev/ttyUSB2] << posx:88.357,posy:68.489,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5
2017-10-17 17:20:40 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => [posx:88.357,posy:68.489,posz:0.000,posa:0.000,feed:16000.00,vel:16000.00,unit:1,coor:1,dist:0,frmo:1,stat:5]
2017-10-17 17:20:40 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:40 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => []
2017-10-17 17:20:40 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:40 GcodeDriver TRACE: [/dev/ttyUSB2] << posx:0.564,posy:0.549,posz:0.000,posa:0.000,feed:16000.00,vel:434.92,unit:1,coor:1,dist:0,frmo:1,stat:5
2017-10-17 17:20:40 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => [posx:0.564,posy:0.549,posz:0.000,posa:0.000,feed:16000.00,vel:434.92,unit:1,coor:1,dist:0,frmo:1,stat:5]
2017-10-17 17:20:40 GcodeDriver DEBUG: sendCommand(null, 250)...
2017-10-17 17:20:40 GcodeDriver TRACE: [/dev/ttyUSB2] << posx:0.000,posy:0.000,posz:0.000,posa:0.000,feed:480.00,vel:0.00,unit:1,coor:1,dist:0,frmo:1,stat:3
2017-10-17 17:20:40 GcodeDriver DEBUG: sendCommand(/dev/ttyUSB2 null, 250) => [posx:0.000,posy:0.000,posz:0.000,posa:0.000,feed:480.00,vel:0.00,unit:1,coor:1,dist:0,frmo:1,stat:3]
2017-10-17 17:20:40 ReferenceCamera DEBUG: moveTo((92.340000, 0.230000, -40.600000, 0.000000 mm), 1.0)

The string "stat:3" appears after the positioning is finished.

Note: in my case 3 lines of Gcodes are sent to the TinyG, 1) position + backlash compensation 2) end position 3) in braces (Gcode comment): the endposition

Could it be that there is anything with the parts.xml or packages.xml wrong? I'm thinking of ambivalent (duplicate) entries or something like this.

@Marek: yes, I get this error with this R2012-10R allways. With the 1608 resistors it works reliably. As I said, I just coppied the pipeline from these R1608 and changed the filder contours settings.

What I've noticed on reviewing the debug-images (thousands) is, that the nozzle has not allways the same diameter on the image, therfore it seems like the z-level is not allways the same. For the 1608 resistors the heights varies between 0.42 and 0.45mm. This small difference is too small for the diameter variations (192px vs. 176px), where most of the images show a nozzle with 176px.

[Cri S]

The old tinyG driver had that wait to move. I don't suggest to change
driver for now.
Can you tell us if you are using prerotate or not ?

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/82511f48-6eb6-4c88-bc9e-ffa4245d165e%40googlegroups.com.

[Team14]

Normally we use prerotate, but we also tried it without.
But when I click the "Test Alignment" button, it shouldn't make any difference, right?

[Cri S]

Some pieces:
script as example log.bsh inside script dir:
org.pmw.tinylog.Configurator.currentConfig()
.formatPattern("{date:yyyy-MM-dd HH:mm:ss:SSS}-{level}:
{thread}({thread_id}) {class_name}.{method}:{line}()\t{message}")
.activate();

This enable millisecond logging instead of second resolution logging
and logging line.
Inside pipeline, make a Script as second argument as example, that
link the pipeline to machines propertie
example:

org.openpnp.model.Configuration.get().getMachine().setProperty("pipeline",pipeline);
i=org.openpnp.model.Configuration.get().getMachine().getProperty("pipeline-id");
if(i==null) i=0; i++;
org.openpnp.model.Configuration.get().getMachine().setProperty("pipeline-id",i);
s="Starting Pipeline nr "+i;
org.pmw.tinylog.Logger.info(s);

After the error you then have access to the pipeline. Unfortunatly the
vision pipeline editor
automatically update the pipeline and don't have a listen only mode,
it could be usefull.
With a script it is possible to dump the images and models or print it.

Annother thing, you should find out how to get a status report from
motion controller,
and include that inside the starting ScriptRun, after image capture
and at the end of pipeline. This then gives you the logs if after
image the cnc have moved or not.
Eventually you could write a script that wait until the motion is
finished and at camera position. For this you can use
sendCommand(command,timeout) ,

[Jason von Nieda]

Hi Karl,

I'm looking into this issue now. A few initial thoughts:

1. In this email you said "PS: attached are images from a real cycle:" - can you just confirm that you mean the two image from the start of the thread?

2. It seems some things have changed between the machine.xml and the pipeline you posted. Can you please send me your current machine.xml as it stands now?

3. The difference in nozzle size that you mentioned is very concerning. This indicates to me that a Z move might not be getting processed correctly. This could be due to a quirk in TinyG. TinyG has a bug (IMHO, they disagree) where it will queue small moves and process them in bulk. This is fine for a CNC machine but it's disastrous for a pick and place machine. For this reason I no longer recommend using TinyG. I've posted some information about this here: https://github.com/openpnp/openpnp/wiki/TinyG#quirks

Unfortunately, the only way around this bug is to make small moves very slowly. This requires some custom code and because of this and a few other quirks in TinyG I've basically given up on it.

4. To fully diagnose this issue, please do the following process. I realize you have already done some of this, but it's important to do it all, in order, at this time so that we have a clean set of data to work with:

Debugging Process:

1. Set up the pipeline so that it works well in the pipeline editor.

2. Make sure part height is set accurately for the given part. I realize that parts may vary a bit in practice - that's okay.

3. Make sure the pipeline includes the two ImageWriteDebug stages as it currently does.

4. Make sure that OpenPnP is in Debug logging mode by setting it in the Log panel.

5. Turn off pre-rotate.

6. Start the machine, home it and in as few moves as possible, perform the bottom vision operation three times. You can use Test Alignment for this or run a single part job three times.

7. When this is complete, go to Menu -> Help -> Submit Diagnostics and include at least the machine.xml, parts.xml, packages.xml and debug images.

8. Send me the resulting URL.

This should give me all the information I need to determine what is happening. 

Thanks,

Jason

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/ad87da5b-0581-4a07-ae78-c91d9f151db4%40googlegroups.com.

[Team14]

Dear Jason,

for the software, I refer to our branch feature/heapfeeder-logic2 commit c0b086af98f017c3179f4e30b7fe6def43ad0112

I've done now the test exatly as you described, but now with a new feeder (ReferenceLoosePart Feeder, named jason-feeder ;) ),
and surplisingly, it worked as expected.

Here the data you requested: https://gist.github.com/anonymous/230b6c580b6ea471301c3a3d42c73294

I was curious now, if the bottom vision will fail again with our HeapFeeder - and surprisingly it worked.
Then I tried the HeapFeeder with another part, the C2012-100n-ESD. There it failed again.
Here the data for the further steps I did: https://gist.github.com/anonymous/854136e8ac8af2c8beeead6c9a70d7ae
(there I had to change the pipeline for the C2012-100n-ESD - this isn't saved in the machine.xml, right? that you shoud find here (sent after restarting OpenPNP): https://gist.github.com/anonymous/912f90f13392bfa7257daeedfb6c1700


ad 1.
in PS i ment the images, i attached in PPS
sorry, I did forget them to attach

ad 2.
please see the debug data

ad 3.
thanks for your insigths of TinyG. It's interesting. We are now debugging small offsets between the nozzle and the camera which vary over the table coordinates. But that's a topic on it self.
We have ordered now a Smoothie Board. We really don't want to waste time with firmware bugs.

Thanks for all your help,
Karl

[Jason von Nieda]

Hi Karl,

The pipelines are included in the machine.xml and with Submit Diagnostics.

Unfortunately, I have to ask that if you are wanting help debugging an issue, please test without third party code. It's a lot of work for me to compare your code with mine to see what could possibly be causing the issue. A good test is to see if the issue happens with no third party code. If it doesn't, that might be a sign that some of your charges are of concern.

One thing I did notice that might be a concern is that you have a Vision.PartAlignment.Before script that is failing. You can see this by looking in the log for "2017-10-17 23:43:01 VisionUtils WARNING: javax.script.ScriptException: TypeError: null has no such function "actuate" in <eval> at line number 10". Perhaps this is related to the problem?

If resolving the scripting issue doesn't help, please try to reproduce the issue with the stock OpenPnP code and do Submit Diagnostics if it fails. It will help if you can tell me if it fails every time, or just sporadically. 

Thanks,

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/0882c831-d621-4263-b80b-56e46ce54fe6%40googlegroups.com.

[Team14]

Dear Jason,

thank you for your support.
I've reset git now to the commit b5e6dbb34ff6c1608352fffeaef8bba62213ace7 (latest develop branch)
I've also renamed the script folder from Events to Events~ to disable all event based scritps.
I cleaned up the machine XML, so there are no HeapFeeder instances any more.
Also the affine transform is removed from the machine.xml

The bottom vision fails now for this capacitor with 100%.
Here is the log data: https://gist.github.com/anonymous/5f79e5454b596f3a484b86e2818f1a80

Again I've noticed the different z-level in the debug images (whin in a set of only about 20 images).

Perhaps I can improve the time resolution of the log messages as Cri has suggested.

Greets,
Karl

[Cri S]

Reviewing you'r pipeline, the reason is clear, you don't return the
correct result.
Bottomvision require RotatedRect and not a list of RotatedRects.
If you use Keypoints (simpleBlob) or any other possible result it must
be converted to
RotatedRect or in case of list, only one item selected from list.
You could perform it using various stages or simple as on default pipeline,

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/faf101b4-f7e6-463d-87e5-0d2e0bf23038%40googlegroups.com.

[Team14]

You are my hero!

Couldn't such an important information be shown to the user somewhere?
It's the second time for me, that I fought with the bottom vision - and in both cases the return value was incorrect.
The first time I returned "results" instead of "result" - what the hack? In C, C++, Java and any language the return value ist the return value. Why mixing the names between different pipelines?
Also the user has to smell this "top secret" return name - or he reads the Java code - which is not what an ordinary user should do.

Is there anywhere a demo page of all the stages? There could something like this easily be mentioned in bold red big letters.
Some sort of overview with short demo images would also be very nice. Is there something like this, or should I start such a table in the wiki on GitHub?

Thank you all for your help!
Karl

[Jason von Nieda]

Nicely spotted Cri! Glad to hear it's resolved.

A few answers to your questions are inline:

Couldn't such an important information be shown to the user somewhere?

Yes, this would be a good improvement. As always, pull requests are welcomed :)

 

It's the second time for me, that I fought with the bottom vision - and in both cases the return value was incorrect.
The first time I returned "results" instead of "result" - what the hack? In C, C++, Java and any language the return value ist the return value. Why mixing the names between different pipelines?

This appears to be a mistake. I just looked at all the places that pipelines are used in the codebase and in all cases *except* bottom vision we look for "results". Bottom vision looks for "result". I'll update the bottom vision code to look for both "results" and "result" and update the default pipeline to use "results" to be consistent with the rest.

 

Also the user has to smell this "top secret" return name - or he reads the Java code - which is not what an ordinary user should do.

I agree. I'm open to suggestions on how to improve this. It is difficult to find a good balance between making the pipeline editor too specific to any one caller and making it too generic, as it likely is now.

 

Is there anywhere a demo page of all the stages? There could something like this easily be mentioned in bold red big letters.

 

https://github.com/openpnp/openpnp/wiki/Bottom-Vision#default-pipeline describes the default pipeline.

 

Some sort of overview with short demo images would also be very nice. Is there something like this, or should I start such a table in the wiki on GitHub?

The page linked above is the place to go for bottom vision information, so if you'd like to improve that page please feel free. All OpenPnP Wiki pages are editable by anyone with a Github account.

A few simple improvements could help us avoid problems like this in the future:

1. Add a new function helper for vision operations that will "coerce" a value into another format. Something like CvStage.getModelAs(Class type); This function would do things like convert a Circle to a KeyPoint, a KeyPoint to a RotatedRect, RotatedRect to List<RotatedRect>, etc. This way authors of pipeline code would use this to ensure that whenever the user came up with in the pipeline, they could get the value type they needed without a bunch of if/else.

2. We could add a checkbox to the stages that lets the user indicate which stage is the expected result. This would avoid the naming issues. Since the name is loosely coupled to begin with, it's very easy to make this kind of mistake. Adding a checkbox would be explicit.

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/7029bade-fa16-46e5-8c2d-d9e22105c5fe%40googlegroups.com.

[Cri S]

Comments to the suggested improvements
results/result it could have sense that bottomvision have annother
keyword as topvision and fiducials in order to not use the wrong
pipeline.
1) prerotate works only with RotatedRect. if other angles are
converted to RotatedRect,
then depending on passed values, it can produce wrong results.
2) i find this more problematic especially when copy pipelines.
Annother method could be to use the workingModel() of the pipeline.
Most pipeline passes automatically the result model to the end of
pipeline, for the others,
setModel stage must copy it again. If after finding the result only
the image is manipulated,
rotated or drawed, then the result model is automatically the working
Model of pipeline.
Templatematch (with angles), keypoints (surf,fast,sift,...) passes
true angles 0-360 degree CW ,
instead rotatedRect passes 0-90 degree CW relative to lower left vertice.

>> <https://groups.google.com/d/msgid/openpnp/7029bade-fa16-46e5-8c2d-d9e22105c5fe%40googlegroups.com?utm_medium=email&utm_source=footer>

>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/CA%2BQw0jxabACzuoJdqUkss1DZFC7oHsK%3DuGKd%3Dbp7MvyxO-tmGA%40mail.gmail.com.

[Cri S]

opencv uses CCW degrees Mat images uses a different coordinate system
as openpnp.

[Jason von Nieda]

I made a few changes last night to help with these kinds of errors in the future. Please see https://github.com/openpnp/openpnp/blob/develop/CHANGES.md#2017-10-18

Jason

[Team14]

Thanks for that! I've already merged them.
A bit missleading is the message for the default failure - no part on the nozzle. But this should easily be ignorable.

Greets,
Karl

[Jason von Nieda]

Hi Karl,

Which message do you mean? And in what cases did you see it?

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/73f6c9b9-3ee0-4397-ab6a-99e78b681296%40googlegroups.com.

[Karl Zeilhofer]

The message comes in the case, where the pileline couldnt find the object. E.g. because of a missing part on the nozzle.

What I would expect is one message, if the pipeline is misconfigured, i.e. there is no stage named results with the type RotatedRect. And another message for "no rect could be found". I'm not sure, if this is techncally possible, but from the users point of view I would like to have this ditinguation.

Greets, Karl

[Jason von Nieda]

Ah, good point. I'll improve that.

Thanks,

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/8eb29c7b-86ed-439d-80ed-8943c0083c63%40googlegroups.com.

[Karl Zeilhofer]

Great, thanks

[Jason von Nieda]

This is now fixed. See https://github.com/openpnp/openpnp/commit/16ab399fccce65c0d4fbdb9c128417b542911850

On Sun, Oct 22, 2017 at 2:28 PM Karl Zeilhofer <zeil...@gmail.com> wrote:

Great, thanks

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/db044681-eb72-4496-92a9-debd8a180390%40googlegroups.com.

[Team14]

I think this isn't fixed, as I suggested.

I had now aigain a typical case:
A resistor was bottom up on the nozzle:



This is not what I would expect as a message.

I think this is technically not that easy to distinguish.
Is it possible, to check the pipeline before processing, if there is the correct results type?
If that is possible, and the results type is incorrect, throw an according exception like in the screen shot above.
Otherwise, if the return type is not a rotated rect, we can assume, that the pipeline failed in any stage to produce the result. The should then show a message like "could not find any results. please check the part on the nozzle or the vision pipeline."

[Cri S]

It is easy to check after pipeline processing.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+unsubscribe@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/8881536b-5411-4991-9c0d-bc961a4379dc%40googlegroups.com.

[Team14]

@Cri, could you pleas clarify your statement?

PS: there is my pipeline:
<cv-pipeline>
   <stages>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageCapture" name="0" enabled="true" settle-first="true"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="13" enabled="true" prefix="bv_source_" suffix=".png"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.BlurGaussian" name="10" enabled="true" kernel-size="11"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskCircle" name="4" enabled="true" diameter="90"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ConvertColor" name="1" enabled="true" conversion="Rgb2Gray"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.Threshold" name="12" enabled="true" threshold="100" auto="false" invert="false"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.FindContours" name="5" enabled="true" retrieval-mode="List" approximation-method="None"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.FilterContours" name="9" enabled="true" contours-stage-name="5" min-area="400.0" max-area="1000.0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MaskCircle" name="11" enabled="true" diameter="0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawContours" name="7" enabled="true" contours-stage-name="9" thickness="2" index="-1">
         <color r="255" g="255" b="255" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.MinAreaRect" name="results" enabled="true" threshold-min="100" threshold-max="255"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageRecall" name="14" enabled="true" image-stage-name="0"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawContours" name="2" enabled="true" contours-stage-name="9" thickness="2" index="-1"/>
      <cv-stage class="org.openpnp.vision.pipeline.stages.DrawRotatedRects" name="8" enabled="true" rotated-rects-stage-name="results" thickness="1" draw-rect-center="false" rect-center-radius="20" show-orientation="true">
         <color r="0" g="255" b="8" a="255"/>
      </cv-stage>
      <cv-stage class="org.openpnp.vision.pipeline.stages.ImageWriteDebug" name="15" enabled="true" prefix="bv_result_" suffix=".png"/>
   </stages>
</cv-pipeline>

[Cri S]

Here code from my bottomvision that handle that. As @vonnieda have
heavly reformatted the source code i assume it cannot copy and pasted
into current source. Hovewer it show
how it can do.

if (((pipeline.getResult("result")).== null)) {
throw new Exception("Missing result stage named
\"result\" for " + part.getId()
+ " on nozzle " + nozzle.getName() );
}

if (((pipeline.getResult("result")).model instanceof RotatedRect)) {
RotatedRect rect = ((RotatedRect)
(pipeline.getResult("result")).model);
angle = angleNorm(angleNorm(angle) + angleNorm(
(rect.size.width < rect.size.height) ? 90 +
rect.angle : rect.angle));
pos = rect.center;
} else
if (((pipeline.getResult("result")).model instanceof KeyPoint)) {
KeyPoint key= ((KeyPoint) (pipeline.getResult("result")).model);
angle = angleNorm(angleNorm(angle) + key.angle;
pos = key.pt;
} else
if (((pipeline.getResult("result")).model instanceof List<KeyPoint>)){
if((List<>((pipeline.getResult("result")).model)).size()>0) {
KeyPoint key= ((List<KeyPoint>)
(pipeline.getResult("result")).model).get(0);
angle = angleNorm(angleNorm(angle) + key.angle;
pos = key.pt;
} else {
throw new Exception("Bottom vision alignment failed
for part " + part.getId()
+ " on nozzle " + nozzle.getName() + ". No
result found.");
}
} else
if (((pipeline.getResult("result")).model ==null)) {
throw new Exception("Bottom vision alignment failed
for part " + part.getId()
+ " on nozzle " + nozzle.getName() + ". No
result found.");
} else {
throw new Exception("Bottom vision returned wrong Type
of result for " + part.getId()
+ " on nozzle " + nozzle.getName() );

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to openpnp+u...@googlegroups.com.

> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/c57d27f0-761b-46c0-b351-f98981e713fa%40googlegroups.com.

[Jason von Nieda]

The code now throws three different error messages:

*  If no stage named "results" is found: "ReferenceBottomVision (%s): Pipeline error. Pipeline must contain a result named '%s'."

* If a "results" stage is found but the result is empty (null): "ReferenceBottomVision (%s): No result found."

* If a "results" stage is found and the result is not null, but it is not a RotatedRect: ReferenceBottomVision (%s): Incorrect pipeline result type. Expected RotatedRect.

In this case, the pipeline errored: 

CvException [org.opencv.core.CvException: cv::Exception: /Users/jason/Projects/opencv/opencv-2.4.11/modules/imgproc/src/contours.cpp:1913: error: (-215) points.checkVector(2) >= 0 && (points.depth() == CV_32F || points.depth() == CV_32S) in function minAreaRect

]

This is because there were no contours that passed the filter, so MinAreaRect failed. 

So, I think this is really a bug in MinAreaRect. It should probably not blow up if there's no results, but instead should just return null, indicating no results. It's not error if there's nothing there, per se, it's just that there is nothing there.

So, I've updated MinAreaRect to return null if there are no points to process, and I think this should improve things for you. I've also updated the "wrong type" error message to include the type of the value that is incorrect, to give another hint to the user. This has just been pushed, and will be available in 10 minutes or so.

Thanks,

Jason

On Fri, Oct 27, 2017 at 7:37 AM Team14 <off...@team14.at> wrote:

I think this isn't fixed, as I suggested.

I had now aigain a typical case:
A resistor was bottom up on the nozzle:

This is not what I would expect as a message.

I think this is technically not that easy to distinguish.
Is it possible, to check the pipeline before processing, if there is the correct results type?
If that is possible, and the results type is incorrect, throw an according exception like in the screen shot above.
Otherwise, if the return type is not a rotated rect, we can assume, that the pipeline failed in any stage to produce the result. The should then show a message like "could not find any results. please check the part on the nozzle or the vision pipeline."

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/8881536b-5411-4991-9c0d-bc961a4379dc%40googlegroups.com.

[Cri S]

Ah, yes, the exception get stored inside model, so there should be a
check for exceptions too
in order to give better info.
The error here is " points.checkVector(2) >= 0 " that just indicates,
that there are no points
(pixels) present.

Inside MinAreaRect this code generate the error.
MatOfPoint2f pointsMat = new MatOfPoint2f(points.toArray(new Point[]{}));
RotatedRect r = Imgproc.minAreaRect(pointsMat);
pointsMat.release();
return new Result(null, r);

If before this point, there was error checking like
if(points.size()==0) return null;

just null get returned. It is not a fault of opencv code, it is just
that this type of errors,
that is now popular on this type of pipelines is not checked.
At time of writing the code, different pipelines was in use with where
this distinction was
not important.

>> <https://groups.google.com/d/msgid/openpnp/8881536b-5411-4991-9c0d-bc961a4379dc%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .

>> For more options, visit https://groups.google.com/d/optout.
>>
>

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/CA%2BQw0jzyo1k-4GEK7hkN1RjcSUj-5Ck61BkXQncuUkPONUUUGA%40mail.gmail.com.