Camera fps, CPU load and lighting/exposure

[ma...@makr.zone]

Hello everybody

I've just confirmed an old suspicion of mine. Camera frames per second (fps) depend on lighting or rather the exposure value you need to set for the given lighting situation.

This at least is true for the ELP FullHD camera and the new 720p 120fps global shutter camera I'm evaluating.

If you increase the exposure, the cameras seem to accumulate multiple frames into one, the frame rate drops in turn. In fact, the exposure time is being increased to decrease noise, which would be inevitable if the sensor signal was just boosted. A higher exposure time also results in stronger motion blur, which is a bad thing, of course.

This can now also explain what Marek T. observed: CPU load increases, if exposure is lowered. With a higher fps being streamed from the camera, it is only natural, that the CPU load from frame decompression/handling increases.

Btw. 1: we established that (at least on Windows) this CPU load happens inside the Windows DirectShow qcap.dll, so it is not a Java or OpenPnP Problem.

Btw. 2: you won't find the Capture FPS Test button on your Device Settings. It's a gadget I added, that I will PR soon. In fact, I just wanted to make sure the new camera really delivers 120 fps... and look what I found ;-)

With exposure all dialed down I can get ~93fps in camera settle, all unique frames, including the settle detection algorithms. Don't underestimate Java:

So ... what can we do?

We must put stronger LEDs on our machines, so we can dial down the exposure to the minimum!

  :-)

_Mark

[ma...@makr.zone]

Google swallowed the GIF animation. See here:

https://makr.zone/camera-fps-cpu-load-and-lighting-exposure/519/

_Mark

[Marmelade]

On Thursday, 27 August 2020 15:47:02 UTC+2, ma...@makr.zone wrote:

We must put stronger LEDs on our machines, so we can dial down the exposure to the minimum!

Correct.

And the strong leds need to be switched to prevent heating up.

I'm using some strong duris led's (4000K) - working like a charm for years - fixed settle time is 50-100ms, depending on elp module.

[bert shivaan]

Got a link for them? Like from digikey or Mouser?

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/07a5fdd7-3a9b-4ee0-a4b1-7821f5dc00c6o%40googlegroups.com.

[ma...@makr.zone]

https://www.aliexpress.com/item/32916630999.html

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CA%2BKNHNyQWLh9tnyFhohhQD9XWn3wuZc7te4GFCmhSUFDtVgomg%40mail.gmail.com.

[Marek T.]

@Bert, I use something like this:

https://www.micros.com.pl/mediaserver/info-oly.plcc4.4200.pdf

and keep them turned on continously.

@Mark, thx for the link with camera, just wanted to ask you about it :-).

[Marmelade]

unfortuneately not available anymore at mouser

https://www.mouser.co.uk/ProductDetail/OSRAM-Opto-Semiconductors/GW-JDSRS1CC-FRFT-5L7N-L1N2?qs=%2Fha2pyFaduhR6gyTppAmiSAMrCH8jHvV9186GPFyHtU87wDh8mrYZlgnKZnTjELEb8Z9aKCb87A=

On Thursday, 27 August 2020 17:00:15 UTC+2, cncmachineguy wrote:

Got a link for them? Like from digikey or Mouser?

On Thu, Aug 27, 2020 at 10:32 AM Marmelade <ooyh...@gmail.com> wrote:

On Thursday, 27 August 2020 15:47:02 UTC+2, ma...@makr.zone wrote:

We must put stronger LEDs on our machines, so we can dial down the exposure to the minimum!

Correct.

And the strong leds need to be switched to prevent heating up.

I'm using some strong duris led's (4000K) - working like a charm for years - fixed settle time is 50-100ms, depending on elp module.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ope...@googlegroups.com.

[Mike Menci]

I found some on eBay Osram GW JDSRS1.CC-FRFT try it 

Mike

[ma...@makr.zone]

Made the Pull Request:

https://github.com/openpnp/openpnp/pull/1053

_Mark

[Marek T.]

Mark, when you dig in it all, could you try to consider whether it's possible to display some other ID of camera together with the one like "RYS HFR USB 2.0..." ?

I know it comes from camera driver but maybe there is some option to present something more?

I mean the situation that there are two identical cameras in the system and missing possibility to differ them somehow.

[ma...@makr.zone]

AFAIK there is no such thing, other than what Jason already displayed in the Combobox. This is an often-discussed Problem of USB devices. They clearly designed this bus thinking "people just have one of each device". If you think about keyboards, mice etc. and webcams in the usual use cases, it is mostly true.

_Mark

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/d9848434-7179-4e3d-8cd7-e5f52e67a344o%40googlegroups.com.

[Jason von Nieda]

There has been some discussion in the past of trying to use frames from the camera to create some type of fingerprint as an ID. I've never tried it. I think it would be pretty tricky.

I have generally found the IDs to be stable, at least on Mac, as long as the USB ports and hubs don't change.

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/5d8fc0dd-a68e-6047-7228-ca766f8b5fa0%40makr.zone.

[ma...@makr.zone]

> There has been some discussion in the past of trying to use frames from the camera to create some type of fingerprint as an ID.

Once installed on the machine, we could identify them by flashing the lights.

We must finally move the lighting actuator control to OpenPnP. :-)

Jason, please detail what you envisioned. Last time I we talked about a solution, you said you wanted more than just Boolean control (brightness, color etc.).

Do you mean like adding up to four Double Actuators R, G, B, W and then setting the RGBW channel settings in the pipeline Capture stage?

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CA%2BQw0jxP3%3DKuGkr%3D6H6W_WcWuARCF5B1Vk726XtpMeUf9a29yw%40mail.gmail.com.

[Jason von Nieda]

Lights would work, as would movement for differentiating between a single top and bottom camera, and for multiple cameras there could be QR codes or fiducials or whatever.

For camera lighting, here are rough thoughts:

I think it would be fine to start with a single actuator assigned to each camera with just a boolean on and off. That would be a huge step improvement over what we have now.

For a more flexible system, I had envisioned something where you could create any number of lighting profiles with names. The profiles would probably be defined on the camera and there would be an API for getting the list of profiles and for setting the current profile on the camera. There would be a default profile of Off and then the user could define other ones. For example: Off, Side, Coax, Angle, Red, Green, etc. The ultimate point of all that is that you could do things like set a default profile for vision operations and jogging, and set specific profiles from either pipelines or on a part or package basis.

How the profiles actually control the hardware would of course be via actuator, and we could get as flexible as we want with that, but I think having a profile with a name and a boolean actuator would be enough for the first pass. The actuator could, of course, send multiple commands to set light levels, colors, etc.

Finally, for an all singing, all dancing version, instead of fixed profiles we could instead let the user add one or more actuators to the camera and specify metadata about the actuators such as name, min, max, and step. Then, in places where a user would want to set the lighting profile for an operation, they would be presented with the actuators and sliders and those values would be saved with that profile.

So, let's say you have a very nice Juki RS-1 that can do all kinds of neat lighting tricks. You could set the top camera lighting for a specific fiducial package like:

- Side [0 - 255]: 0

- Angled [0 - 255]: 255

- Coax [0 - 255]: 128

Thanks,

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/d2b980f8-b6f5-ba47-50a5-fa45c4b11be3%40makr.zone.

[sebastian...@gmail.com]

Hey,

your suspicion and conclusion is kind of correct.

Let me drop some comments:

 

"If you increase the exposure, the cameras seem to accumulate multiple frames into one, the frame rate drops in turn."

This is only true for HDR imaging. Usually cameras do not accumulate multiple frames into one.

The sensor elements (pixels) are exposed to the lens's image for a time interval (exposure time), limited either by using a mechanical or electronic shutter.

In that time the sensor elements convert any incident photons to electrons and doing so accumulate a charge over time.

I'll spare you the sensor specific details on how they turn that charge into a digital number but after exposing the sensor it is "read out".

So the minimum that defines the frame rate is exposure time + read out time. Then you have some camera and interface specific compression and transfer times that

ideally should not affect fps.

"In fact, the exposure time is being increased to decrease noise,"

This is incorrect. Noise increases with exposure time. The longer your pixels accumulate photons the longer they accumulate electrons from various sources of noise.

Cover your lens and check the noise level (dark noise) for various exposure times.

In practice you always try to minimize exposure time by using brighter illumination.

Of course that brightness is limited by things like power, heat, eye safety, hardware availablility...

Sebastian

[ma...@makr.zone]

Thanks Sebastian

Instead of "accumulate multiple frames into one", I should have written "accumulate multiple frame times into one".

I wrote this, because I had the impressions that fps followed rational numbers i.e. multiples of the nominal frame time. But maybe I'm wrong.

Btw. I would not exclude the possibility that accumulation actually happens digitally, i.e. the sensor is read at fixed intervals and the obtained pixel values just digitally added. It would simplify the whole DSP real-time system design a lot, running everything at fixed intervals, guaranteeing real-time constraints, including the interlock with USB packet synchronization and bandwidth considerations.

>> "In fact, the exposure time is being increased to decrease noise,"

> This is incorrect. Noise increases with exposure time.

This was intended to make sense together with final, crucial part of that sentence "... if the sensor signal was just boosted". What I'm saying is that if you take the low exposure and boost it to full range, then noise is very strong. It is like a high ISO setting in a digital cameras.

But I agree I could have formulated that better. English is a foreign language ;-)

Made some adjustments on the Blog, do you think its better?

https://makr.zone/camera-fps-cpu-load-and-lighting-exposure/519/

Thank you.

_Mark

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/b0926400-b151-4275-95df-b1f6922aa3e3n%40googlegroups.com.