ReferencePushPullFeeder: Smarter via CV or we shouldn't even try?

[Roman Valls Guimerà]

Hello Mark and other OpenPnP experts,

We are test driving a custom PnP drag feeder designed to be manufactured very fast and cheaply with a laser cutter and a significantly reduced amount of 3D printed material (faster prints) while also requiring minimal assembly efforts since it doesn't involve any electronics (like the original PushPullFeeder, but way faster to make and put together):

https://github.com/PancakeLegend/Pancake-Feeder

The feeder works fairly well with OpenPnP's PushPullFeeder, it's a small and beautiful minimalistic thing:

The problems we are facing look a bit like this, only sometimes: the machine tries (and obviously fails) to pick a component where there's none (the feed operation goes fine for a few components in a row and then randomly decides to pick an empty "box" in the paper strip):

In the pictures above, after feeding (advancing the tape with the nozzle tip), the machine tries to pick and fail a missing component on the paper strip, so here's the first question:

Would it make sense for the PushPullFeeder to be smarter and detect missing components on the paper strip, always picking components instead of empty "boxes" in the strip? Also avoiding to perform the feed operation on non-empty boxes?

This question comes with the premise that due to our design requirements, those feeders have relatively poor tolerances, such as the reel/strip paper getting ever so slightly (microscopically?) deformed when feeding (push pull) moves and therefore not having a fully consistent, repeatable advance of the tape... which folks will argue that the vision system should compensate for **and it does**, but only to a degree: we now get 9 components placed in a row without manual intervention but sooner or later the accumulated errors creep up and not enough tape is fed, even using the statistics-based "Until Confident" (default) PusPullFeeder setting.

On top of that issue, there's another failure mode that involves the nozzle tip not lingering for long enough on the feeder to buildup vaccum fast enough and pick the part properly... I believe that setting is "pick dwell time", which I tried to bump up to 500ms and higher without much success :/ 

So the question is: after our ContactProbeNozzle contacts the component on the feeder, which setting can we tweak to have it dwell there for, say, 500ms to 1 second?

Building on top of that failure mode, when it tries to pick a non-present component from the feeder and then proceeds to the bottom vision only to not detect any component, this error pops up, predictably:

And I've set a "Pick retry" of 3 in bottom vision settings... always fails at the first try though, when the bottom camera only sees the nozzle tip instead of the component. Then my two questions would be:

Can I force this step to either go to retry a pick on the feeder if no component is found on the tip OR go to discard and try to pick again to be safe?

Can I have this error not be a total job halt situation without resorting to scripting? 

In other words, what I'm doing right now when that bottom vision error pops up is clicking on "Stop job (red square stop button)" and then "Start job (green triangle button)" and it resumes where it left off fine... so I presume this behavior should be builtin/automated somehow?

Last but not least, this is not an ask but more of a laugh together on yet another thing that goes wrong with pick and place machines: (old?) paper-based reels where the peel plastic takes some paper with it, biasing the feed operation and messing stuff up more than needed (easy fix IRL by humans, doesn't need fix by software):

Jokes aside, and as usual, the current config settings for our machine are updated (and current) here:

https://github.com/CCHS-Melbourne/Pick-N-Place/tree/master/config

[Jan]

Hi Roman!
I'm using the ReferencePushPullFeeder for a drag feeder with a separate
drag pin to move the tape. There are a few things I'd suggest to check
to overcome the issues you're seeing:

1) there are knobs to disable vision updates on any or all axis of the
drag move sequence. You can use the one along the tape to prevent drifts
across multiple feed/vision operations. However, you've to configure the
drag move such, that your nozzle (?) safely hits the hole from feed to
feed in that direction. I usually drag ~0.2mm more then needed and go
back by that distance in the final step. That covers the free space
between drag pin and sprocket hole and the wiggle in the drag pin itself
and a safe release of the drag pin.

2) pick dwell time (defined at nozzle and nozzle tip level) is the value
you'd like to check when picking is not reliable. Make sure your valve
has the correct coordination configured so that the dwell time starts
after opening the valve. I've about 30cm tubing between nozzle tip and
valve and use 20ms.

3) If you have vacuum part on detection enabled and configured (at
nozzle tip level) the job processor will retry feeding/picking as
configured. However, I'd suggest to fix the configuration first before
relying on retries.

Jan

On 15.04.2024 15:41, Roman Valls Guimerà wrote:
> Hello Mark and other OpenPnP experts,
>
> We are test driving a custom PnP drag feeder designed to be manufactured
> very fast and cheaply with a laser cutter and a significantly reduced
> amount of 3D printed material (faster prints) while also requiring
> minimal assembly efforts since it doesn't involve any electronics (like
> the original PushPullFeeder, but way faster to make and put together):
>
> https://github.com/PancakeLegend/Pancake-Feeder
>
> The feeder works fairly well with OpenPnP's PushPullFeeder, it's a small
> and beautiful minimalistic thing:
>

> pancake_feeder.png
>
> The problems we are facing look a bit like this, only sometimes: the
> machine tries (and obviously fails) to pick a component where there's
> none (the feed operation goes fine for a few components in a row and
> then randomly decides to pick an empty "box" in the paper strip):
>

> IMG_3835.JPG
>
> And I've set a "Pick retry" of 3 in bottom vision settings... always
> fails at the first try though, when the bottom camera only sees the
> nozzle tip instead of the component. Then my two questions would be:
>
> Can I force this step to either go to retry a pick on the feeder if no
> component is found on the tip OR go to discard and try to pick again to
> be safe?
>
> Can I have this error not be a total job halt situation without
> resorting to scripting?
>
> In other words, what I'm doing right now when that bottom vision error
> pops up is clicking on "Stop job (red square stop button)" and then
> "Start job (green triangle button)" and it resumes where it left off
> fine... so I presume this behavior should be builtin/automated somehow?
>
> Last but not least, this is not an ask but more of a laugh together on
> yet another thing that goes wrong with pick and place machines: (old?)
> paper-based reels where the peel plastic takes some paper with it,
> biasing the feed operation and messing stuff up more than needed (easy
> fix IRL by humans, doesn't need fix by software):
>

> IMG_3836.JPG
>
> Jokes aside, and as usual, the current config settings for our machine
> are updated (and current) here:
>
> https://github.com/CCHS-Melbourne/Pick-N-Place/tree/master/config
>

> --
> You received this message because you are subscribed to the Google
> Groups "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openpnp+u...@googlegroups.com
> <mailto:openpnp+u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openpnp/fc7968ea-6af0-44c6-90ab-d66eea5d9093n%40googlegroups.com <https://groups.google.com/d/msgid/openpnp/fc7968ea-6af0-44c6-90ab-d66eea5d9093n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[mark maker]

Hi Roman,

there are many questions in that post. I'll answer off the top of my head, so take it with a grain of salt 😉

1. Regarding empty part pockets: currently there is no vision of the parts themselves taking place, in none of the OpenPnP feeders. It would likely be quite difficult to implement robustly. Plus we want to assume that not every feed uses vision. When you use the "until confident" statistical analysis, vision should stop after a few feeds. Vision requires extra moves and camera settling etc. i.e. it is expensive and we want to avoid it.
2. Regarding retry after vision failed: OpenPnP does not currently support this. It only supports retry in the feed&pick loop and in the alignment itself, i.e. it retries the bottom vision operation, which happens so fast you might not notice (check the log). A retry loop across JobProcessor steps is not supported, because everything is currently done "per nozzle-full". On multi-nozzle machines it would have to go back to the feed&pick step but then only for the failed nozzle(s). Such a "masked" operation is not currently supported. And single nozzle machines get no special treatment. Currently, as Jan already mentioned, the only advice is using vacuum sensing, as it can happen inside the feed&pick step, i.e. inside the retry loop.
3. Regarding halting the whole job: there is a Error Handling option Defer, which will skip the part and you can then later restart the job with just the failed parts. You can set it on multiple placements at once using select/right-click:
   
   I have not tested it.
   
4. Note, there has been much, much talk about improving automatic error handling, but never any real implementation. It is very complex if you want it to actually improve the situation and not risk wasting whole tapes of expensive parts.
   
5. Regarding the errors accumulating: see Jan's answer. Disable Vision calibration on the axis in drag direction.
6. Regarding the vacuum dwell time: again I recommend vacuum sensing. It can actively monitor the vacuum and dwell as long as needed to establish a certain level. This time may vary a lot e.g. after a failed pick where a lot of pressure got lost. Always dwelling for a very long time makes the machine very slow!
   https://makr.zone/openpnp-advanced-vacuum-sensing-part-on-part-off-detect/421/
   
7. Regarding failed picks: also check the Contact Probing you mentioned, it might not work correctly, i.e. not make a sealed vacuum contact. Be aware of its Z offset caching, it might apply outdated cached Z offsets when you changed something on machine or feeder (yes, it has fooled me too!). When in doubt, look into the log or set probing to EachTime.
   

_Mark

On 15.04.2024 15:41, Roman Valls Guimerà wrote:

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/fc7968ea-6af0-44c6-90ab-d66eea5d9093n%40googlegroups.com.

[Roman Valls Guimerà]

Thanks everyone for the insights!

We already had vacuum sensing setup but it turns out that we unchecked a few of the checkboxes that configure **when** to vacuum sense: before pick, alignment, after pick.

After checking those boxes up *and* setting 3x3 retry attempts at the feeder level, it's working much better, thank you! Luckily I didn't have to set any dwell time, so no slowdown either :)

I also now understand better the complexity of error management when there are potentially expensive components at stake and multiple nozzles (fascinating!).

W.r.t the contact probe sensing: is there a way to start the SOLENOID+PUMP right before it tries to perform a pick? Right now it seems like SOLENOID+PUMP are activated right when the contactnozzle actuator clicks, which is arguably too late to build up proper vacuum?

Cheers!

Roman

[Jan]

Hi Roman!

On 16.04.2024 09:47, Roman Valls Guimerà wrote:
[...]

> W.r.t the contact probe sensing: is there a way to start the
> SOLENOID+PUMP right before it tries to perform a pick? Right now it
> seems like SOLENOID+PUMP are activated right when the contactnozzle
> actuator clicks, which is arguably too late to build up proper vacuum?
>

The vacuum pump can be controlled at the head level. You can specify
when to switch it on and how log OpenPnP will wait until it continues.
For the valve, there is the machine coordination that you have to
configure. But this would only change the valve earlier, not later then
required. After switching the valve to vacuum, the pick dwell time is
used to delay operation before the part is lifted.

Jan

> Cheers!
> Roman
> tisdag 16 april 2024 kl. 02:32:33 UTC+10 skrev ma...@makr.zone:
>

> __

>
> Hi Roman,
>
> there are many questions in that post. I'll answer off the top of my
> head, so take it with a grain of salt 😉
>

> 1. *Regarding empty part pockets: *currently there is no vision of

> the parts themselves taking place, in none of the OpenPnP
> feeders. It would likely be quite difficult to implement
> robustly. Plus we want to assume that not every feed uses
> vision. When you use the "until confident" statistical analysis,
> vision should stop after a few feeds. Vision requires extra
> moves and camera settling etc. i.e. it is expensive and we want
> to avoid it.

> 2. *Regarding retry after vision failed: *OpenPnP does not

> currently support this. It only supports retry in the feed&pick
> loop and in the alignment itself, i.e. it retries the bottom
> vision operation, which happens so fast you might not notice
> (check the log). A retry loop across JobProcessor steps is not
> supported, because everything is currently done "per
> nozzle-full". On multi-nozzle machines it would have to go back
> to the feed&pick step but then only for the failed nozzle(s).
> Such a "masked" operation is not currently supported. And single
> nozzle machines get no special treatment. Currently, as Jan
> already mentioned, the only advice is using vacuum sensing, as
> it can happen inside the feed&pick step, i.e. inside the retry loop.

> 3. *Regarding halting the whole job*: there is a *Error Handling*
> option *Defer*, which will skip the part and you can then later

> restart the job with just the failed parts. You can set it on
> multiple placements at once using select/right-click:
>

> I have not tested it.

> 4. Note, there has been much, much talk about improving automatic

> error handling, but never any real implementation. It is very
> complex if you want it to actually improve the situation and not
> risk wasting whole tapes of expensive parts.

> 5. *Regarding the errors accumulating:* see Jan's answer. Disable

> Vision calibration on the axis in drag direction.

> 6. *Regarding the vacuum dwell time:* again I recommend vacuum

> sensing. It can actively monitor the vacuum and dwell as long as
> needed to establish a certain level. This time may vary a lot
> e.g. after a failed pick where a lot of pressure got lost.

> A/lways/ dwelling for a very long time makes the machine very slow!
> https://makr.zone/openpnp-advanced-vacuum-sensing-part-on-part-off-detect/421/ <https://makr.zone/openpnp-advanced-vacuum-sensing-part-on-part-off-detect/421/>
> 7. *Regarding failed picks:* also check the *Contact Probing *you

> mentioned, it might not work correctly, i.e. not make a sealed
> vacuum contact. Be aware of its Z offset caching, it might apply
> outdated cached Z offsets when you changed something on machine
> or feeder (yes, it has fooled me too!). When in doubt, look into

> the log or set probing to *EachTime*.

>
> _Mark
>
> On 15.04.2024 15:41, Roman Valls Guimerà wrote:
>> Hello Mark and other OpenPnP experts,
>>
>> We are test driving a custom PnP drag feeder designed to be
>> manufactured very fast and cheaply with a laser cutter and a
>> significantly reduced amount of 3D printed material (faster
>> prints) while also requiring minimal assembly efforts since it
>> doesn't involve any electronics (like the original PushPullFeeder,
>> but way faster to make and put together):
>>
>> https://github.com/PancakeLegend/Pancake-Feeder
>> <https://github.com/PancakeLegend/Pancake-Feeder>
>>
>> The feeder works fairly well with OpenPnP's PushPullFeeder, it's a
>> small and beautiful minimalistic thing:
>>

>> pancake_feeder.png
>>
>> The problems we are facing look a bit like this, only sometimes:
>> the machine tries (and obviously fails) to pick a component where
>> there's none (the feed operation goes fine for a few components in
>> a row and then randomly decides to pick an empty "box" in the
>> paper strip):
>>

>> IMG_3835.JPG
>>
>> And I've set a "Pick retry" of 3 in bottom vision settings...
>> always fails at the first try though, when the bottom camera only
>> sees the nozzle tip instead of the component. Then my two
>> questions would be:
>>
>> Can I force this step to either go to retry a pick on the feeder
>> if no component is found on the tip OR go to discard and try to
>> pick again to be safe?
>>
>> Can I have this error not be a total job halt situation without
>> resorting to scripting?
>>
>> In other words, what I'm doing right now when that bottom vision
>> error pops up is clicking on "Stop job (red square stop button)"
>> and then "Start job (green triangle button)" and it resumes where
>> it left off fine... so I presume this behavior should be
>> builtin/automated somehow?
>>
>> Last but not least, this is not an ask but more of a laugh
>> together on yet another thing that goes wrong with pick and place
>> machines: (old?) paper-based reels where the peel plastic takes
>> some paper with it, biasing the feed operation and messing stuff
>> up more than needed (easy fix IRL by humans, doesn't need fix by
>> software):
>>

>> IMG_3836.JPG
>>
>> Jokes aside, and as usual, the current config settings for our
>> machine are updated (and current) here:
>>
>> https://github.com/CCHS-Melbourne/Pick-N-Place/tree/master/config
>> <https://github.com/CCHS-Melbourne/Pick-N-Place/tree/master/config>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "OpenPnP" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to openpnp+u...@googlegroups.com.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/openpnp/fc7968ea-6af0-44c6-90ab-d66eea5d9093n%40googlegroups.com <https://groups.google.com/d/msgid/openpnp/fc7968ea-6af0-44c6-90ab-d66eea5d9093n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openpnp+u...@googlegroups.com

> <mailto:openpnp+u...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/68b88389-50f0-41a2-b85c-81ce57729868n%40googlegroups.com <https://groups.google.com/d/msgid/openpnp/68b88389-50f0-41a2-b85c-81ce57729868n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[mark maker]

See also:

1. Pump Control:
   https://github.com/openpnp/openpnp/wiki/Setup-and-Calibration_Vacuum-Setup#pump-control-setup
2. To improve fast vacuum: how a reservoir tank for the vacuum can be run on hysteresis:
   https://makr.zone/vacuum-sensor/192/
   

_Mark