Framerate issue with synchronizing three cameras

[Chelsey Linnenbank]

Hi,

I'm a new user of Bonsai and have some troubles with my workflow. 

We have a experimental set-up that consist of three cameras (IDS) that needs to be synchronized , therefore we wanted to use Bonsai. 

However, with the combinator ''CombineLatest'' we have runned into framerate issues. Despite the set values for the framerate in the configuration file and for the videowriter (ffmpeg) the video ends up in slowmotion, because it speeds up due to the three images. 

I've also tried other combinators like ''Zip", which solves the framerate issue but not synchronize the cameras. 

So do I miss something?

How do you synchronize more than two cameras with Bonsai?

Thank you!

Chelsey

[brunocruz]

Hi Chelsey,

I don't have experience with IDS cameras, but I thought they could be synchronized with an external trigger signal (?). If that is the case, I would advise you to go for that solution instead of relying on Bonsai to synchronize the incoming stream of data. Alternatively, you could also just use a small LED that turns on/off and it is visible by all 3 cameras simultaneously.

If you really have to rely on Bonsai to synchronize your frames, CombineLatest is probably the best you can do (check another post here, it might be clearer). 

For each sample any of the inputs to this node emits, CombineLatest will produce a package of the latest samples of each of the inputs. Considering the simplest scenario wherein all cameras are acquired at the exact same frame rate, for each frame arriving, it will "reproduce" the previous frame of the other 2 cameras. This will result in 3x as much data and hence giving the appearance of a 3x slowed down video.  Zip "fixes" it by applying a different combination logic. Zip "waits" until all 3 sources have emitted a sample and only then outputs a package. 

Now, why use CombineLatest? As long as you can live with a temporal jitter in the order of ~10ms, this node will give you, for each incoming frame, the *latest* frames available from the other two streams. This is probably the best you can do to synchronize them relying on bonsai alone at the moment. Essentially, when analyzing your data, later on, all three videos you are saving will have the exact same number of frames. The way to interpret these videos would be something like: 

frame #100 of video A has the corresponding closest frames in frame #100 of video B and video C.

In practice, if you just want to do post-hoc analysis, and if you don't want to deal with duplicating your data and having videos with different durations you could simply save the indices of the frames I believe. This will give you a lookup table that tells you for frame #100 of video A, the closest frames are #X and #Y in videos B and C, respectively.

It would look something like this:

Hope it helps,

B