problems in lighting

504 views
Skip to first unread message

Cees Leeuwinga

unread,
Dec 17, 2020, 8:09:54 AM12/17/20
to OpenPnP
After recent update there is a problem in the camera looking up light.
In spite of the scripts in the event directory it will not switch off.
my files, any hints?
logdata.txt

ma...@makr.zone

unread,
Dec 17, 2020, 10:22:52 AM12/17/20
to ope...@googlegroups.com

Hi Cees

it seems that this PR...

https://github.com/openpnp/openpnp/pull/999

https://github.com/openpnp/openpnp/pull/999/files?diff=unified&w=1

...somehow fell out of the branch after the merge.

Must have happened when I resolved some artificial conflicts that git sometimes seems to produce after merging PRs from multiple parallel running branches back. Some git expert out there know how to avoid that?

Will fix it ASAP.

Thanks for the report and sorry about that.

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/0a1bc788-aead-4128-9c2c-cfdd4671c200n%40googlegroups.com.

Marek T.

unread,
Dec 17, 2020, 10:30:11 AM12/17/20
to OpenPnP
Hi Mark,

Haven't you thought to create cofigurable actuator and place its control into the code? System control.
For bottom camera it seems be super-easy, a bit more complicated for the top (few more on/off points).

ma...@makr.zone

unread,
Dec 17, 2020, 10:33:35 AM12/17/20
to ope...@googlegroups.com

Yes, but Jason's got (much) more advanced plans for that.

_Mark

Marek T.

unread,
Dec 17, 2020, 12:15:52 PM12/17/20
to OpenPnP
Uuuu!
Looking on how busy is Jason - It does not bode too well ;).
Fortunately it's not something especially important...

But, Jason, some your thoughts on how you wanted to do it?
It seems be really nothing complicated (comparing to famous pick).

ma...@makr.zone

unread,
Dec 17, 2020, 12:32:35 PM12/17/20
to ope...@googlegroups.com

Ok, it is fixed.

You can download the new version:

https://openpnp.org/downloads/

_Mark

Jason von Nieda

unread,
Dec 17, 2020, 12:50:43 PM12/17/20
to ope...@googlegroups.com
Hi Marek and Mark,

For lighting, I think we can start small. Most people just need to turn lights on and off when they are needed without messing with scripts. So maybe a single actuator attached to a camera that takes a boolean would be fine, along with a few checkboxes that say when to trigger it.

Later on we can get fancy - I envision the user being able to create lighting profiles that are selectable from pipelines or parts. For instance, if you have a lighting system that supports side, angle, and ring lights, it would be cool if you could decide which of those gets used, along with their intensities, for each part. But again, that can all come later.

Jason


Cees Leeuwinga

unread,
Dec 17, 2020, 1:17:33 PM12/17/20
to OpenPnP
I have struggeld so much with this failure, that the programm OPENPNP was damaged and would not be running again.
I tried to reinstall it, but no chance it was screwed up., no movements anymore possible etc.
I cleand the hard disk and put a older version on it, and now im in the process to get it running again.
Cees

Op donderdag 17 december 2020 om 18:50:43 UTC+1 schreef Jason von Nieda:

Marek Twarowski

unread,
Dec 17, 2020, 1:46:26 PM12/17/20
to ope...@googlegroups.com
Hi Jason,

Sounds good.
I'm not sure if one common actuator is enough, but maybe it is.
In my custom I've made it super simple (IMO).
My bottom light is always on so I don't control it. 

Top light I'm turning on for:
- fuducial check
- job pause
- job finished 
- nozzle calibration (however there is actuator in the pipeline today)
And off for:
- job start
- fiducials done
- job "unpaused"
- nozzle calibration done

I haven't connected it to the picture capture to avoid blinking while the serie of pictures taking (repeated capturing or nozzle calibration or multi nozzles alignment). And it seemed to be more simple to me.


You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/j4ucK1JuyH0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CA%2BQw0jwb8xB2nxEvFOyysxLupES_rrMANbsAfUXGsEZ-HtyjDA%40mail.gmail.com.

Jason von Nieda

unread,
Dec 17, 2020, 1:47:53 PM12/17/20
to ope...@googlegroups.com
Hi Marek,

I didn't mean one actuator for the whole system - I meant one actuator per camera. So telling a camera to turn on it's lights would actuate the attached actuator.

Jason


Marek Twarowski

unread,
Dec 17, 2020, 1:56:05 PM12/17/20
to ope...@googlegroups.com
Ahh I didn't read it carefully enough, sorry. "single actuator attached to a camera".

So remains to decide when it's optimal to turn it on/off and where to put it into the Gui, actuators and checkboxes. A new config tab?

ma...@makr.zone

unread,
Dec 17, 2020, 2:05:33 PM12/17/20
to ope...@googlegroups.com

Hi Jason, Marek

I started thinking about how to provide a solutions that can be both simple to use for the ON/OFF use case and extensible.

But it always exploded into complexity that seemed over-the-top. The user would have to be able to define a number of custom labeled parameters (at least doubles and perhaps bools) that dynamically generate the GUI required to then define lighting parameter-sets (similar to the OpenPnPCaptureCamera's property sliders, perhaps). Then multiple such parameter-sets could be defined in the Machine Setup tree, one for each lighting solution. This requires quite a lot of GUI plumbing for adding/deleting etc. in the tree. Then these parameter-sets could be selected in the pipeline ImageCapture stage (Drop Down). For advanced use cases they only make sense there, IMHO, as they work hand-in-hand with the pipeline.

But how would the parameters out of the set be assigned to actuators? One actuator per parameter? This won't work, if multiple params must be specified in one command to the controller. And what if the params are somehow encoded? Such as an RGB code made out of a "Red", "Green" and "Blue" slider, for instance? Some RGB controllers probably require such composite params. Do we create a new multi-parametric actuator? With scaling/composing/hexcoding? It seems endless.

Then I remembered there is the ScriptRun pipeline stage.

https://github.com/openpnp/openpnp/blob/develop/src/main/java/org/openpnp/vision/pipeline/stages/ScriptRun.java

The user can choose a script file and set arguments. The whole complexity of the parameter-sets and their plumbing can either be packed into a script, so the user simply chooses "SideLightHard", "FrontalMidLevel", "DiffuseBright" etc. or she can use one Script and set the lighting solution as an argument (in the required syntax).

I think this is a much more versatile and reasonable solution for those with complex lighting setups.

And the rest of us can simply use a boolean actuator on the camera. This one can then also be used to switch OFF any ScriptRun driven lighting solution (by using {False} Gcode or by using a ScriptActuator if not single-driver/Gcode controlled).


If you're ok with that, I'll add the boolean actuator.

_Mark

Jason von Nieda

unread,
Dec 17, 2020, 2:11:55 PM12/17/20
to ope...@googlegroups.com
That sounds great to me, please go ahead Mark!

Thanks,
Jason


ma...@makr.zone

unread,
Dec 17, 2020, 2:49:49 PM12/17/20
to ope...@googlegroups.com

> I have struggeld so much with this failure,...

Have you seen my email that it should be fixed now?

> ...  that the programm OPENPNP was damaged and would not be running again.

I don't understand. Haven't you backed up everything, when I told you, after you had no backup the last time? Btw. your almost-good machine.xml is still somewhere here in the user group discussion.

_Mark

ma...@makr.zone

unread,
Dec 17, 2020, 2:50:35 PM12/17/20
to ope...@googlegroups.com
> Top light I'm turning on for:
> - job pause
> - job finished
> And off for:
> - job start
> - job "unpaused"

I would provide an enum selection of camera light mode:
  • Off
  • On
  • Capture
  • Automatic
In Capture or Automatic the actuator would be turned ON when
  • settleAndCapture is called.
plus in Automatic when
  • user moves the mouse over the camera view
  • Machine thread is finished with user action or Job Runnable and a camera was moved, or if a tool was moved over a bottom camera. This should cover both the Job end/pause/failure and all GUI camera positioning actions. It should only light up the camera that was moved (to), so the other lights don't blind the active camera.
In Capture or Automatic the actuator would be turned OFF when
  • settleAndCapture is done.
plus in Automatic when
  • the light is ON, after a configurable timeout. Using a short timeout this should also cover machines with LEDs that must not be turned on continuously (overheating etc.).

_Mark

Marek Twarowski

unread,
Dec 17, 2020, 3:24:16 PM12/17/20
to ope...@googlegroups.com
If it will be on when "settleAndCapture is called" it forces to use long settle time for the image stabilization, isn't it? 
Plus if the picture taking will be repeated (like in the mode when pixels changing is checked) it will blink like stupid. The same with no sense, except it looks funny, if you have more than one Nozzle and making alignment.  And while Nozzle calibration.
Am I wrong? 
It's convenient to connect it to capture but for me it makes no sense. 
I know it's Christmas, tree, lamps...

You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/j4ucK1JuyH0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/b6557b83-3e67-57fd-c9d4-1fef38c7f47d%40makr.zone.

Yevhenii Shcherbakov

unread,
Dec 18, 2020, 2:52:03 AM12/18/20
to OpenPnP
Hi Mark.
IMHO, It is great!


четверг, 17 декабря 2020 г. в 22:24:16 UTC+2, Marek T.:

ma...@makr.zone

unread,
Dec 18, 2020, 2:58:29 AM12/18/20
to ope...@googlegroups.com

Hi Marek

First: Personally I don't switch lights. They are just always ON. And on my machine I don't see a reason for them to switch. But....

Some machines may have the down-looking camera so close to the nozzles that its light blinds the up-looking camera, so it then makes sense to switch it ON only when used. (English speakers: or do you say "blinds" or "glares"?)

Some lights may overheat or damage the LEDs if continuously on.

> If it will be on when "settleAndCapture is called" ...

That's how it works today using scripts AFAIK.

> ... it forces to use long settle time for the image stabilization, isn't it?

In theory: No. If you switch off auto-exposure and all other "auto" settings, as is absolutely necessary anyway, it should not matter.

In practice: Maybe. Some cameras may still do some adaptive image processing, despite setting everything to manual.

I will think about one more setting:

  • Off
  • On
  • ExclusiveCapture
  • Capture
  • Automatic

In this setting it would keep all the camera lights always ON except when another camera that looks the other way (down vs. up) is in settleAndCapture().

Do you think that would help?

_Mark

Marek Twarowski

unread,
Dec 18, 2020, 4:50:43 AM12/18/20
to ope...@googlegroups.com
Sounds good, let's try, later improvement should be not complicated if needed.

I know which script (capture) is usually used by folks and IMO it's just mistake. 


ma...@makr.zone

unread,
Dec 18, 2020, 5:00:25 AM12/18/20
to ope...@googlegroups.com

The right way is using the CameraBeforeSettle and CameraAfterSettle scripts.

https://github.com/openpnp/openpnp/wiki/Setup-and-Calibration%3A-Camera-Lighting#add-lighting-control-scripts

That is no mistake, IMHO.

_Mark

Duncan Ellison

unread,
Dec 18, 2020, 5:02:53 AM12/18/20
to OpenPnP
_Mark,

It's  a very fine distinction, but ...

Most correctly, you would say that the "uplooking camera is 'blinded' by the downlooking light", especially if the blockout is total.

or you could say that the "'glare' from the downlooking light, upsets the image from the uplooking light"

'glare' implies a partial image with unpleasant bright sports, 'blinded' implies that the amount of light is overwhelming.

:-)

Duncan

Marek Twarowski

unread,
Dec 18, 2020, 5:09:51 AM12/18/20
to ope...@googlegroups.com
For me it's total mistake.
How many times the light will blink for one part if you need auto-repeat the image capturing and what for?
Before align and after align is much better IMO.
The same before nozzle calibration and after it.

Mike Menci

unread,
Dec 18, 2020, 6:13:48 AM12/18/20
to OpenPnP
My Up and down looking Coaxial light- cameras are always ON.
I would agree to switch Up-looking camera ON-OFF if Up-looking camera would turn on ahead of time, so that when nozzle reaches the camera it has time for light to settle down for vision to work properly.
Mike

ma...@makr.zone

unread,
Dec 18, 2020, 7:23:04 AM12/18/20
to ope...@googlegroups.com

Thanks Duncan :-)

Learning heaps of stuff in this community!

_Mark

ma...@makr.zone

unread,
Dec 18, 2020, 7:52:29 AM12/18/20
to ope...@googlegroups.com

> Before align and after align is much better IMO.
> The same before nozzle calibration and after it.

For the bottom camera, maybe.

But for the down-looking camera? Many DIY feeders use vision too. It is not generally foreseeable (i.e. a lengthy machine move in advance), when the next vision operation will take place.

But: having the enum, will allow us to add other patterns later. 

Btw, for pro machines the lights often act like a flash, LEDs can take much higher currents momentarily. So blinking is not per se bad.

https://youtu.be/tJ-N-6DMIe4

...is it just because I'm Swiss that I like these Swiss machines? ;-)

_Mark

Marek Twarowski

unread,
Dec 18, 2020, 8:59:08 AM12/18/20
to ope...@googlegroups.com
Of course Mark I've had bottom on my mind mainly.
But downlooking also has many repeated captures while fiducials checking. 
So IMO before/after entire fiducials processing is better than for each single capture.
Maybe while feeders feeding?
And while Pause is usable.
All the rest of time it's not needed to be on.
And my opinion is unchanged, for me attaching it to the capture is not good.

Flashing that you say is rather other cameras' story... Isn't it?

PS. Swiss chocolate is also great! :-).
Seriously, I'm whole pro to reach the speed like that :-). Vacuum checking is magic for me how quick it must work there - if they check it at all.
-


Jason von Nieda

unread,
Dec 18, 2020, 10:15:01 AM12/18/20
to ope...@googlegroups.com
I agree Marek - I think having a mode, or an option, where the lights turn on at the start of a repetitive vision operation and off at the end makes sense and is the use case that a lot of people run.

Mark - rather than an enum where single values encapsulate a number of operations, I think it would be better just to have checkboxes to turn on features. The features would be as you've described, and as I've mentioned above:

- When mousing over camera view.
- Before settle.
- Before vision.
- Always on.
- Etc.

I prefer the explicit options rather than hiding them in an enum because it makes it very obvious what will happen rather than having to go dig in the docs.

Jason

ma...@makr.zone

unread,
Dec 18, 2020, 12:57:40 PM12/18/20
to ope...@googlegroups.com
Important note first:

With Advanced Motion Control the light switching in the proposed settleAndCapture() trigger can now use the new uncoordinated actuation, by switching off the Before Actuation coordination on the Light actuator:

https://github.com/openpnp/openpnp/wiki/Motion-Planner#actuator-machine-coordination

The BeforeSettle trigger (as the Script trigger today) happens before the waitForCompletion() on the move to the camera or camera subject location:

https://github.com/openpnp/openpnp/blob/e6217a00f77ad2058d0c22683faa6ed205833406/src/main/java/org/openpnp/spi/base/AbstractCamera.java#L668-L680

This means the uncoordinated actuation will not yet trigger motion planning, execution and waiting, but it is immediately sent to the controller. This in turn means the light will already be switched ON, when the move is finally executed. If the move is substantial, even a bad camera has time to adapt. It will be more or less the same timing as BeforeAlignment. But it will equally work for calibration, fiducial recognition, any feeder vision etc. pp. provided there is a pending move in front.

Having said that, is can also make sense to keep the Before Actuation coordination switched on, if for instance the LEDs must not be ON too long due to overheating.

> Mark - rather than an enum where single values encapsulate a number of operations, I think it would be better just to have checkboxes to turn on features. The features would be as you've described, and as I've mentioned above

I agree for the mousing-over option and other timeout triggers to be separate checkbox options.

But otherwise, IMHO that's not so easy as it sounds ;-)

Some of these triggers may at one time be and at another time not be nested into one-another. E.g. a Capture can be nested inside an Alignment. Or not. So the user needs to switch on both triggers for proper operation. The ON triggers must also balance the OFF triggers (like parentheses in a math formula).

GUI/usability: Unless I'm mistaken you end up with a row of checkboxes that need to be switched on or off in a nested&balanced pattern to be correct i.e. you effectively end up with the same few valid choices that the enum provides. Only the user can shoot him- or herself in the foot much easier. :-/

Implementation: To cover the nesting of triggers, a naive approach could be to use a counter, increment it on the selected ON-triggers, decrement it on the selected OFF-triggers and switch the light according to the predicate counter > 0. In case of exceptions we can probably make this robust using finally {} i.e. make sure we balance the counter correctly in all cases.

Today, each PlannedPlacementStep only handles one placement i.e. one nozzle alignment per step and each one of these switches the light ON and then OFF.

@Marek, you have a multi-nozzle machine, so it will blink between nozzles! This may happen so fast, you don't notice it (look into the log). But if no blinking is noticeable there, the same is true for multi-pass alignment and fiducial drill-down. Blinking will be so short (few ms), humans don't see it and any camera auto-exposure will equally not be affected (if it was, it would be equally fast to recover).

If you wanted to avoid all blinking in a multi-nozzle machine, you'd need to introduce two new JobProcessor steps "preAligment" and "postAligment". And the same bracketing would be needed for fiducials, etc.

With the ON-trigger being in a different step in the JobProcessor than the OFF-trigger, plus exception handling and retries etc., it would become much more complicated to keep the counter balanced. Imagine what could happen to the rest of a Job if the JobProcessor ever got this wrong!

I'll also admit: I absolutely and resolutely hate any form of reference counting. If you ever "worked" with Microsoft OLE/COM, you know this is utter ***** ;-)

Frankly, this all seems too much. Even if the above-mentioned asynchronous operation of the lights is not enough (or not available), I see these valid scenarios:

  • User has the recommended camera (ELP) or better: AFAIK, it has no problems settling with rapid light changes when all the "auto" options are OFF, as recommended. Switching the lights can even be seen as an advantage, because it creates a definitive starting point for Camera Settling and especially the Debounce Frames logic. It is then hard-synced to the controller and therefore to motion and unaffected by any kind of video lag.
  • User has a crap camera with only auto-exposure: He/she can use always-ON or the new ExclusiveCapture (detailed earlier). Or live with longer settle times. Or buy a better camera and keep more hair down the road, because a crap camera will make vision unreliable as hell in more ways than this one. ;-)

Finally, the ExclusiveCapture option (anti-glare) does not fit into the multi-option checkboxes pattern, as it effectively inverts the logic.

So: I propose sticking to the enum, plus extra checkbox option for mouse-over et al, plus timeout.  :-)

 _Mark

Jason von Nieda

unread,
Dec 18, 2020, 1:47:17 PM12/18/20
to ope...@googlegroups.com
Hi Mark,

I think we're rapidly over-complicating this and I don't want all this complicated functionality in Camera. I propose we start small:

- Add the boolean actuator to Camera. This gives us a clear lighting API we can add on to. The existing scripts can be adapted to use it, which will simplify them.
- Add a checkbox to Camera to turn the lights on before settle and off after capture. This will support users with fast lighting and cameras who do not mind the blinking. This handles the vast majority of vision cases and for many people the scripts can go away entirely.
- Add a toggle button to the CameraView to turn the lights on/off for that camera. Being able to turn on the lights easily while performing setup tasks will be very helpful, and is common in other PnP software.
- (Optionally) Add a checkbox to CameraView for "auto" where mousing over will activate the lights for that camera view.
- (Optionally) Add a checkbox to ReferenceBottomVision to turn lights on at the start of the operation and off at the end of the operation.
- (Optionally) Add a checkbox to ReferenceFiducialLocator to turn lights on at the start of the operation and off at the end of the operation.

I think this will be a very small set of changes and will solve the problem for a majority of users. Advanced users can still use scripts to handle lighting if they need to and this gives us a path to grow the feature down the road.

Thoughts?

Thanks,
Jason


Marek Twarowski

unread,
Dec 18, 2020, 1:58:36 PM12/18/20
to ope...@googlegroups.com
Ech...
Mark, whatever you do it will be a step forward and 90% of people will be glad of it.
I said what I think about connecting it to the capture and I still think I'll be between those 10% :-).
But I don't worry at all, I'm used to thought that will never get free from customizations :-). So you can easy ignore my doubts and fears ;).

Marek Twarowski

unread,
Dec 18, 2020, 2:05:29 PM12/18/20
to ope...@googlegroups.com
I confirm that "Add a toggle button..." is very usable. I didn't do it at the beginning and we have found it pretty quick as the mistake.
I've added the button to the jog.panel to have right quick access to it, but CameraView sounds also not bad.

ma...@makr.zone

unread,
Dec 18, 2020, 3:29:03 PM12/18/20
to ope...@googlegroups.com

Hi Jason

I feel my message was misunderstood. I must have formulated it badly, and I agree it was too long and too "twisted". I was speaking out against over-complication and what I documented was in-deed the complication that I feared, as a justification for not doing it that way!

Therefore, I humbly ask to re-read my message and try to understand why I still want the simple enum instead of a growing number of checkboxes. Why I can't implement what you ask for (especially the "Optionally" points).

_Mark

Jason von Nieda

unread,
Dec 18, 2020, 3:47:38 PM12/18/20
to ope...@googlegroups.com
Hi Mark - I've re-read your previous message, and your other messages in the thread, and it is still not clear to me. Leaving out the (Optionally) points, perhaps you could explain briefly why these three items would cause problems?

- Add the boolean actuator to Camera. This gives us a clear lighting API we can add on to. The existing scripts can be adapted to use it, which will simplify them.
- Add a checkbox to Camera to turn the lights on before settle and off after capture. This will support users with fast lighting and cameras who do not mind the blinking. This handles the vast majority of vision cases and for many people the scripts can go away entirely.
- Add a toggle button to the CameraView to turn the lights on/off for that camera. Being able to turn on the lights easily while performing setup tasks will be very helpful, and is common in other PnP software.

I think these three items would cover the vast majority of use cases, and would be a marked improvement over what we have now. Additionally, these changes are very small and easy to add on to later.

Thanks,
Jason


ma...@makr.zone

unread,
Dec 19, 2020, 5:47:53 AM12/19/20
to ope...@googlegroups.com

Hi Jason

90% of what you say is equivalent to what I meant to describe. And if we assume, we can "forever" leave it at the functionality listed in the last mail, I'd be OK with it.

But with the Optionally points and one of my ideas this is no longer the case. I don't know if I can describe it better this time. I'll try with a more concrete example:

  1. We start with the "before/after settle & capture" checkbox.
  2. In my view, this is already not such a clear user interface: If the checkbox is off, does that mean that OpenPnP will turn the light always ON with the machine enable? Or do we leave that to the ENABLE_COMMAND, effectively having to duplicate the actuator Gcode and changing the actuator state behind its back?
  3. Or do we add another checkbox "enable with machine"?
  4. If yes, the user is given the impression that she has 4 options now (binary: two bools). But it is easy to see that there are only three sensible choices.
  5. Now we add the mentioned (Optionally) ReferenceBottomVision "before/after alignment" checkbox.
  6. The user is given the impression that she has 4 [or 8] options now (binary: two [or three] bools).
  7. But that's wrong: before/after settle must remain switched on, for those cases of bottom vision that don't run through alignment, like e.g. calibration.
  8. Now you could argue, that we need to add a further "before/after calibration" checkbox.
  9. But again, the extra freedom of degree makes no sense: I see no realistic scenario, where one wanted to switch the light on in alignment but not in calibration or vice versa. There is no real choice here, either you have the lights always on, or you switch them in camera settle or you absolutely need to switch all the operation-level "outer bracket" checkboxes on.
  10. Plus this add an eternal burden onto every future bottom vision application, to add one more checkbox, to handle the trigger etc. And this would be even worse for all the down-looking (e.g. feeder) vision applications. Plus one for pipeline editing...
  11. Now you could argue, OK, let's only add the most important/performance critical vision operation checkboxes and let the rest be handled by the "before/after settle & capture" checkbox.
  12. Having only some of these "outer bracket" checkboxes means we need to keep track of trigger nesting. If the user selects both the outer bracket "before/after alignment" and the inner bracket "before/after settle" options (as she must), then the "after settle & capture" trigger must not switch off the light!
  13. I won't start the "over-complication" essay again. In my previous message I tried to explain why there are very valid reasons why truly eliminating blinking with multiple nozzles in the JobProcessor is not so easy as it sounds and if you really wanted to implement it (by adding extra state machine steps), then keeping track of the nested trigger brackets is very hard in the face of exceptions and retries.

Having many checkboxes seems to add much freedom of choice. But if you really think about it, only a few "semantic" combinations make sense. You're effectively back to these exclusive choices:

  1. Don't switch at all (leave it to Gcode or Scripts etc.)
  2. Switch ON with machine enable.
  3. Switch early/late, bracketing whole vision operations.
  4. Switch before/after camera settle & capture.

I detailed in my message, how I can effectively cover option 3 with uncoordinated actuation, sneaking in the actuation command before the motion command, as this is typically still queued in the motion planner. This will work for all vision applications that I can think of, both bottom and down-looking. This will also effectively eliminate blinking due to correctional moves between vision passes of both alignment and fiducial location. Blinking will technically still be there (as it is today!), but it will be so short neither a human, nor the auto-exposure of a camera will be affected.

Plus I proposed an extra ExclusiveCapture choice. For this one the trigger is kind of inverted towards other cameras. Therefore it doesn't naturally mix with checkbox options.

All in all I'm back to an enum choice. Plus the mouse-over and other GUI related checkboxes.

Hope it makes more sense this time.

_Mark

Marek Twarowski

unread,
Dec 19, 2020, 7:12:56 AM12/19/20
to ope...@googlegroups.com
Hi Mark,

These four options sound good to me. In fact they're two to me as first is nothing and the second is almost nothing (if it is just connecting the lights to the enable_command).
Option 3 in your meaning is maybe good to test (maybe in test release) and replace with some simpler if won't work reliably for users?
Or it could be opt3 ("bracketing" in simple Jason/mine meaning) and opt3a (Mark proposal) ? 
And with time 3 or 3a could be removed like it was with different Nozzle calibration options.

ma...@makr.zone

unread,
Dec 19, 2020, 8:48:35 AM12/19/20
to ope...@googlegroups.com

Hi

> Option 3 in your meaning is maybe good to test (maybe in test release) and replace with some simpler if won't work reliably for users?

This topic somehow seems to escape succinct writing down by me. Sorry about that. :-/

When I said "I can effectively cover option 3 with uncoordinated actuation", I meant there is no option 3 to choose after all. Instead, you switch off the Before Actuation coordination checkbox on the Light actuator:

https://github.com/openpnp/openpnp/wiki/Motion-Planner#actuator-machine-coordination

In combination with option 4 this then acts like option 3.

Adding the "anti-glare" option (what I called ExclusiveCapture before) we have:

    1. Don't switch at all (leave it to Gcode or Scripts etc.)
    1. Switch ON/OFF with machine enable/disable.
    2. - ? -
    3. Switch ON/OFF before/after camera settle & capture.
    4. Switch ON/OFF with machine enable/disable, but in addition switch OFF while another camera pointing into the other direction is inside settle & capture (anti-glare).

    > Or it could be opt3 ("bracketing" in simple Jason/mine meaning) and opt3a (Mark proposal) ?

    Nested bracketing has the problems I described. I see no robust way.

    But I could simply add the ON-trigger and ignore nesting in the OFF-trigger, so the light would be switched ON earlier, but switched OFF after the first settle & capture and then switched back ON before the next settle & capture. Again the blinking is probably so fast, it does not matter. Tried to visualize it, with the "Nested" showing the theoretical optimum that I believe can't be implemented easily:

    If this is all bad: The scripts still exist and you could run your lights at whatever script trigger you like. I guess you learned to arrange yourself with the nested bracketing problems, like having to switch on the lights manually for pipeline editing etc. And I guess your lights blink more than you think, but so fast you don't notice. A log would show this.

    _Mark

    Marek Twarowski

    unread,
    Dec 19, 2020, 10:45:37 AM12/19/20
    to ope...@googlegroups.com
    Mark, 
    I don't use light scripts at all. I've added one fixed name actuator for the downlooking lighting and on-off points of him to the job.processor (mainly).
    - off at job start
    - on at fiducials start
    - off at fiducials done
    - on at pause on
    - off at pause off
    - on at job finished.
    No chance that anything can blink there, I don't even need to check it in the log...
    As I said I don't use feeders with vision so I didn't care about it, but don't think it's problem to add there similar on-off "bracketing", most primitevily to the feed of pick at least.

    Sorry, I'm pragmatic and can't see any reasons to complicate the simplest things if the solution works.

    Blinking can be still done with the scripts (which I personally have cut out from the code naively believing it will save some ms per job :-)).

    ma...@makr.zone

    unread,
    Dec 19, 2020, 11:31:54 AM12/19/20
    to ope...@googlegroups.com

    Yes I see. With no bottom camera light switching in play and only fiducial vision (no feeder or any other down-looking vision), your solution can be quite simple.

    If my previous explanation was not understandable, then I guess you just have to trust me when I say it is not so simple for the universal general case.  :-)

    _Mark

    --
    You received this message because you are subscribed to the Google Groups "OpenPnP" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

    Marek Twarowski

    unread,
    Dec 19, 2020, 12:18:14 PM12/19/20
    to ope...@googlegroups.com
    I think I moreless understand your approach. 

    But I just have never liked using camera.settle, and even if blinks would be invisible I'm not convinced to it. I don't like executing many redundant operations. 
    I always prefer to hammer the nail in with one hit instead of ten ;). I believe it's faster...

    So we (you) wait for Jason's final decision...



    Jason von Nieda

    unread,
    Dec 19, 2020, 3:12:22 PM12/19/20
    to ope...@googlegroups.com
    Hi Mark,

    Thanks for the explanation - it makes more sense now.

    Let me clarify what I meant:

    I think the difference between what I am describing and what you are describing has more to do with where the control happens. For my (Optionally) settings, I meant that the control (and the setting) would be on the calling object, primarily meaning the vision operation itself. In other words, ReferenceBottomVision, ReferenceFicudialLocator, ReferenceStripFeeder, etc..

    For instance, ReferenceBottomVision could have "Control Camera Lighting?" and it would just turn on the lights before the operation starts and off when it ends. ReferenceFiducialLocator could do the same. The idea is that the individual vision system knows better about when it needs lights than anything else.

    Now, I think you can (successfully) argue that the lights just simply need to be on when taking an image and off when not. That works fine if you assume that every vision system is going to want the same lighting, and maybe for a first pass it's okay, but my ultimate goal with lighting is to support multiple lighting systems on a camera. This is common in commercial machines, and I have seen its effectiveness in person.

    Here is how that could look:

    (Someday):
    - A Camera has user definable lighting profiles. As an example, let's say: Ring, Coaxial, Angle. These are probably different Actuators.
    - On ReferenceBottomVisionWizard you can choose a profile to be used as the default: Coaxial.
    - On ReferenceBottomVisionPartWizard for the part "Annoying White Body IC" you choose profile: Angle.

    If instead we use an enum we end up making a bunch of choices for the user, and then each time we need to make any kind of change we can't change existing enum values, cause that breaks it for everyone, so we have to add ever more complex and specific enum values.

    I don't think we need to worry very much about the bracketing, as you call it, if each vision system is responsible for making sure the lights are correct for its purpose. When ReferenceFiducialLocator needs lights, it calls for them, and when ReferenceStripFeeder needs lights, it calls for them. Each operation will make sure the lighting scenario fits its needs.

    The one caveat to that is that if we provide an option to simply turn on the lights before settle, and off after capture, then we immediately have a system that will work for many people with a single checkbox.

    So, if I were to propose the full end to end system, it would look like this:

    - Each Camera has a set of named, user defined lighting profiles. A profile is simply a name and an actuator.
    - Each vision performing class has a dropdown to select a lighting profile from the associated camera.
    - The CameraView has a toggle button to turn lights on and off manually for the user to use as needed.
    - The CameraView could also have a checkbox for "auto" which would turn the lights on when moused over

    Now that I spell it all out, it's probably really not that much work to do it this way right now, but my thought was that we could start small with my previous proposal which is just adding "before settle, after capture" and not make the changes to the various vision systems. This would work for almost everyone and it would be easy to add profiles and individual vision system control later by removing the checkbox and adding the profiles.

    Thanks,
    Jason




    ma...@makr.zone

    unread,
    Dec 20, 2020, 10:37:18 AM12/20/20
    to ope...@googlegroups.com

    Hi Jason

    Think I'm starting to see where the differences in our judgment originate. Especially if we start talking about custom lighting profiles again.

    (I'm not sure I get this out properly in English).

    In my judgment the correct entity that should determine the lighting profile is the Pipeline. It often contains hard-wired properties that only work with a specific lighting setup e.g. a specific threshold setting vs. the brightness of the shot. If you go further with "Ring, Coaxial, Angle" and start customizing detection methods between pipelines e.g. canny detecting edges in conjunction with "Angle" lighting for a particularly difficult low-contrast subject, as an alternative to the usual threshold detection in conjunction with "Coaxial" lighting, then this is even more true.

    This is also usability oriented: Imagine you're editing the Pipeline, trying to get a grip on that "Annoying White Body IC". Wouldn't it be super nice to be able to go to the ImageCapture stage and adjust the lighting profile interactively, in tight interplay with the other stages of the pipeline?

    Let's say it bluntly: If I ever build adjustable lights into my machine, I would absolutely want it that way!

    That's why I'm so obsessed with settle & capture as the one and only trigger. It is called right from the Pipeline ImageCapture stage and it could carry the lighting profile as a parameter in the future.

    Of course one could assign the lighting profile outside the Pipeline on the Wizard where the Pipeline is actually managed. But IMHO, this would just complicate editing. It would also require a lots of work for the extra GUI plumbing for each and every pipeline carrying Wizard. From a design stand-point, to burden and duplicate this responsibility onto that many different objects also feels wrong for me.

    Btw. if one day we rework Pipelines to be managed globally and referenced rather than copied, i.e. you select one from a drop-down of known good ones, rather than edit/copy/paste, this solution would be compatible. 

    > For my (Optionally) settings, I meant that the control (and the setting) would be on the calling object, primarily meaning the vision operation itself. In other words, ReferenceBottomVision, ReferenceFicudialLocator, ReferenceStripFeeder, etc..

    For ReferenceBottomVision, ReferenceFicudialLocator I disagree. It is the Part/Fiducial [should be Package] in question that carries the knowlegde whether it is the "Annoying White Body IC" in BottomVision or not, whether it is a Homing Fiducial, a PCB fiducial (HASL or ENIG?) or a SlotSchultzFeeder fiducial. And for the ReferenceStripFeeder I'd again prefer to handle it as a tight interplay of  lighting and detection in the pipeline, e.g. how to light and detect black/transparent plastic or white paper carrier tape. The same for the countless other feeder classes where we can adjust the Pipeline individually.

    Sometimes the information about the specifics (Part/Fiducial/Feeder) and even the question whether Vision is needed at all, are not known where I think you would place the lighting trigger. For instance, if you want to bracket the lighting for the whole of locateBoard(), you don't know which type of fiducials there are, if any. Plus you need to care about bracketing, because getFiducialLocation() can be called directly (homing/SlotSchultzfeeder) or indirectly by locateBoard().

    For feeder vision it is particularly bad: some feeders (mine and others) perform location calibration (and OCR) as a side effect of getPickLocation(). Whenever some calling code wants to know for the first time where exactly the pick location is, it will effectuate the vision (btw. that's why I introduced the job preparation cycle, to make this less awkward and more efficient). In the Job this can happen from feed or pick step or both. Often as a lazy evaluation one-time per feeder operation.

    The example shows how unpredictable vision calling can be. A solution that works in all cases, without posing any burdens on the calling code, has a huge advantage in my view.

    Of course we could also refactor half of OpenPnP and make Lighting a first class citizen. With up-front planning, querying the lighting profile(s) from the upcoming Part/Fiducial/Feeder, carefully anticipating lazy evaluations, and storing all those in the Placement. Then adding separate steps in the JobProcessor to trigger them. We'd need to touch most of the GUI triggered vision too ... I don't think so.

    Finally I'd like to ask, whether my explanation about the uncoordinated actuation is understandable and whether you agree that it solves most of the "proper-time-for-auto-exposure" problem.

    _Mark

    Marek Twarowski

    unread,
    Dec 20, 2020, 11:12:54 AM12/20/20
    to ope...@googlegroups.com
    I've the only one question.

    ''That's why I'm so obsessed with settle & capture as the one and only trigger. It is called right from the Pipeline ImageCapture stage and it could carry the lighting profile as a parameter in the future.".
    Do you mean the first stage turning an actuator on? So what is turning it off?

    Jason von Nieda

    unread,
    Dec 20, 2020, 3:22:45 PM12/20/20
    to ope...@googlegroups.com
    Hi Mark,

    I agree - Pipeline is the right place and settle and capture is the right call. If we think of this in terms of Pipeline this even gives us a logical path to adding “no blink” by passing a flag to the pipeline to say whether it is the final call in a sequence. Not needed now though.

    So, would you agree with:

    - Add a Boolean actuator to Camera.
    - Add a checkbox to the ImageCapture stage to control lighting, defaulting to on.

    I think this is a very simple,
    clean change that leaves a lot of room for expansion later, and works for most people right now.

    Thanks,
    Jason
    --
    Sent from my BeOS enabled toaster

    Jason von Nieda

    unread,
    Dec 24, 2020, 12:46:37 AM12/24/20
    to ope...@googlegroups.com
    Hi Mark - just wanted to see if you had seen the below message and whether you think it's acceptable?

    Also, I forgot to answer:

    Finally I'd like to ask, whether my explanation about the uncoordinated actuation is understandable and whether you agree that it solves most of the "proper-time-for-auto-exposure" problem.

    Yes, I think I understand. By turning off coordinated motion before actuation it means the actuator will be turned off after capture, before the next movement, and then immediately back on or in parallel with the next movement. So as long as communication and actuation is fast, there should be very little or no "blink". Is that right?

    Thanks,
    Jason

    ma...@makr.zone

    unread,
    Dec 24, 2020, 3:14:16 AM12/24/20
    to ope...@googlegroups.com

    Hi Jason

    yes, I've seen the answer, and of course I'm fine with it. I just haven't found the time to implement it yet.

    And yes, your description of uncoordinated actuation is correct. The remaining blinking will be irrelevant for most machines (FET switching over USB: I estimate a <1ms glitch i.e. completely invisible).

    I have some ideas about how to eliminate even this blinking in the future. But let's not complicate it now.

    _Mark
    --
    You received this message because you are subscribed to the Google Groups "OpenPnP" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
    Reply all
    Reply to author
    Forward
    0 new messages