Issue with frame rate Basler USB.

[Dorian Battivelli]

Dear Bonsai users, 

I'm having some issue to change the frame rate of acquisition with my Basler usb. 

When I change the fps on the feature file via PylonView, and then modify consequently  the fps in the VideoWriter node, the resulting video is not saved with the good length. 

For now, it seems that the only acceptable fps to record with my workflow is 20 fps. If I increase or decrease it, the resulting video is not reliable anymore in term of duration. 

Someone would have any suggestion to fix this issue? 

Thank you, 

Best, 

Dorian 

[Gonçalo Lopes]

Hi Dorian,

Have you confirmed the value of the exposure setting in the camera (or whether exposure time is set to Auto)? Often this is the root of these problems, as acquisition will be limited by how long it takes the sensor to grab an exposure, regardless of FPS. For example, if your exposure time is 20ms you will never get an acquisition rate above 50Hz.

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/917d13d3-31a2-4816-bb04-113edac8e3dfn%40googlegroups.com.

[Dorian Battivelli]

Hello Goncalo, 

I changed the exposure time on Pylon software, so that I can acquire 40 fps (cause indeed both fps and exposure are interdependent) . And when I run the video live on Pylon, indeed, 40 fps are acquired (attached screenshot). 

So I don't understand why, when I set the VideoWriting node at 40 fps it does not work. For now 1 min of recording with the 40 fps configuration results in 24 sec videos. Would you have some suggestion to solve this? 

I also attach my parameters for time exposure, hoping that it can help to understand the root of the issue, 

Thank you, 

Best, 

Dorian 

[Dorian Battivelli]

To further precise the workflow I'm using, I had to set the buffer "false" in the VideoWriter node, in order to run the 40 fps pylon configuration. Otherwise, I reached RAM saturation (of 64 gb) after a jigsaw increase that finally saturated also the CPU and made the overall workflow crash. With buffer "false" for each VideoWriting node (I'm using 4 cameras simultaneously), it solves this crashing issue, but I end up with shorter videos, as described above. 

Thank you, 

Best, 

[Dorian Battivelli]

As last precision (I'm trying to give you all the information that could help), when I look the details of my video file, it is a 40 fps video. So it's not clear for me why it is shorter than the real duration of recording. Could that be due to a delayed onset of the recording? 

[Gonçalo Lopes]

Hi Dorian,

To further precise the workflow I'm using, I had to set the buffer "false" in the VideoWriter node, in order to run the 40 fps pylon configuration. Otherwise, I reached RAM saturation (of 64 gb) after a jigsaw increase that finally saturated also the CPU and made the overall workflow crash.

This is the crux of the matter. From what you describe it looks like your computer is struggling to run the live video compression fast enough. When Buffered is set to True, the frames are pushed into a queue and compressed in parallel so as not to disrupt the video acquisition process (i.e. to avoid frame drops). However, if your computer cannot encode frames at around the same rate as they are acquired on average, then you will end up putting many more frames in the queue than you are taking out. This is why you saw the RAM increasing, and eventually this will crash the computer.

If, on the other hand, you set the Buffered option to False, then Bonsai will indeed block the pipeline until each frame is completely written to the file. However, this means that while it is waiting for the video frame to be written it is not receiving any new frames from the camera, so you end up dropping frames which leads to the shorter durations in the video. The FrameRate you specify in VideoWriter is only really used for decoding of the video, the encoder doesn't know anything about whether you dropped frames or not, i.e. if in reality you acquired 20 frames / second over 2 seconds, then when playing back you get only 1 second at 40 fps.

If you don't want to have these gaps in the video, you need to turn Buffered to True. If your computer is crashing due to the RAM, then you have 3 options to try:

  1) get a faster CPU / more RAM;

  2) reduce the resolution of the video;

  3) change the video codec

Which option is best depends on a number of factors including which exact hardware you have and what is the resolution of your video. There are a few posts in the forums already discussing this problem so it would be best to go over them and follow-up if there are questions.

Hope this helps.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/2a6ca309-bdb3-410c-938d-ad762bc0deben%40googlegroups.com.

[Dorian Battivelli]

Thank You Gonçalo.

Indeed, I tried with another computer, and I manage to avoid the memory leaking issue. 

Concerning the timing that was not real time, it was a mistake in the recording setting on PylonViewer app, 

Thank you for the help, 

Best, 

[Laurens Witter]

Dear all,

I'm having very similar issues with running three IDS cameras at 100Hz. It works when the resolution is low enough, or when running in black and white. As soon as I get color involved at HD resolution it turns to ±30Hz, but encodes at 100Hz (so a sped up movie). Strange thing is that it doesn't scale with resolution or computational load: It is either running at ±30Hz or at 100Hz, nothing in between.

The computer seems fine when encoding: i7 11700K, 16Gb Ram, 3x Samsung 980 1Tb SSD. It runs at ±25-50% processor load when acquiring and ±1% SSD load. Should be headspace to run at least a bit faster..... When I run it buffered it fills the memory in seconds and crashes the computer.

Another weird thing is that the 'standard' encoder only accepts a very particular setting on the camera (24bits RGB encoding). Anything else and it returns an empty movie file of 6Kb. I tried other encoders but can't seem to figure out how to set up these encoders. I've heard FFMPEG is great and fast, but I can't get it to work. Does anyone have a link with clear steps how to set this up on a windows PC?

Another question here is whether I would need a GPU to get this to work. When naively calculating the datastream I came to the conclusion the direct stream to disk should work without any issue if I would install an SSD per camera, but apparently it doesn't. So, where is my bottleneck and how could I work around this? It's been driving me crazy.....

Many thanks in advance!

Laurens

details:

3x IDS UI-3060CP-C Rev2 HQ via USB 3.0 (running at 100Hz, prefer to run them at 160Hz)

i7 11700K @3.6Ghz

16Gb ram @3200MHz

3x 1Tb Samsung 980 M2 SSD

no GPU (except the integrated one)

Op maandag 7 juni 2021 om 14:52:58 UTC+2 schreef dorian.b...@gmail.com:

[Gonçalo Lopes]

Hi Laurens,

Video compression can be very quirky unfortunately, and such non-linearities are sadly not uncommon. It might indeed be worth trying FFmpeg. You can now find a working example on the Bonsai Gallery (Tools > Bonsai Gallery) and search for FFmpeg writer.

Hope this helps and let us know if you get stuck using the example.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/98629978-4a46-45f3-ac79-8d747660b7e5n%40googlegroups.com.

[Laurens Witter]

Hi Goncalo,

Many thanks for pointing me to the Bonsai Gallery, it really helped getting ffmpeg to work! But unfortunately, it's not solving my problem yet. In my naivety I assumed that I could easily just stream data to disk if the SSDs were fast enough.

I'm now down to recording 720x390 pixel movies @100Hz that I can stream and save from 3 cameras. Anything higher and memory starts to fill up within seconds to minutes crashing the computer. I'm amazed by this because in principle this should just generate less than 100MB per minute per second per movie (720*390*24b*100Hz/8bit/1e6), but this seems to be the max for my system.

The really weird thing is that when I keep the bitrate for FFMPEG encoding the same, memory fills up with a higher res movie, even though the processor is only used ~60-80%. So, the bottleneck I think is not in writing to disk, but in handling the movie (which my non-dedicated GPU build does poorly).
I guess I'll keep tweaking a few things and see whether I can squeeze out a bit more? Are the above settings for a 3-camera setup reasonable, or should I be able to get more out of it? If you say I should be able to get more out of it (I hope so!), then I'll go through this google group and FFMPEG documentation in more detail again.
Anyway, many thanks for your help pointing me in the right direction! And many thanks for developing Bonsai!

Kind regards,

Laurens

Op zaterdag 28 augustus 2021 om 17:10:13 UTC+2 schreef goncaloclopes: