Videowriter Delay

[Fitz Sturgill]

Hi,

I'm using Bonsai to record ~10s video snippets in response to a trigger during a behavioral experiment. To do this, I'm using a Point Grey Flea3 camera and feeding into the GPIO pin a TTL pulse which I'm then reading out within Bonsai from the PointGrey camera source metadata. There is a experimentally imposed variable delay in between each ~10s snippet. The issue is that sometimes Bonsai fails to record a video snippet so that I"m dropping videos across behavioral trials. I think this has to do with the length of the delay, the shorter it is, the more often my Bonsai workflow fails. Another detail is that I'm I'm running matlab on the same computer (although the CPU and memory usage don't appear to be anywhere near maxed out). I'm also using Windows 7 64bit (it's a core i7 processor with a bunch of RAM). Would it help to use a camera-independent trigger? Say a serial command or have an arduino serving as a sort of DAQ?

I've attached a screenshot of my workflow as well as the bonsai code itself. The DistinctUntilChanged and ElementCountWindow nodes are my kludgy way of extracting just the rising edge of a TTL pulse.

Any help would be greatly appreciated,

Fitz

[goncaloclopes]

 Hi Fitz,

I currently don't have access to a PointGrey camera so I can't try out the workflow myself. However, I've made a number of improvements to the logic to try and streamline the processing.

First, I've replaced the direct metadata comparison with the GpioPinState node. This node automatically splits out the four bits of GPIO pin state into a Scalar structure for easier access. I've assumed your trigger is in Pin 0, but if this is not the case, you can change which pin you want to test by double clicking on the selector node following GpioPinState and changing the selected member.

I've also replaced the ElementCountWindow by a Condition in order to isolate only the rising edges. As you've realized, DistinctUntilChanged will compute both the rising and falling edges of the signal. An empty Condition basically works as an "if" clause in traditional programming languages, letting only elements that are "true" go through. In this case, rising edges will be "true" and falling edges will be "false".

I'm curious if this fixes your problem, otherwise the workflow looks fine. Is every package up to date in your Bonsai install? If this is the case and the problem persists I will have to debug this workflow myself in a few days when I have access to a PointGrey.

Hope this helps,

[Fitz Sturgill]

Hi Goncalo,

I definitely learned something from your answer but I'm still having an issue. When I hooked up my TTL pulse to an oscilloscope and watched the oscilloscope and the Bonsai greater than node at the same time I could see that sometimes the trigger was being missed. So on my Bpod (which I'm using to control my experiment and send TTL pulses) I made a test protocol that sent a super wide 100ms trigger pulse regularly.  

Here is a video of what I see in Bonsai:

https://www.youtube.com/watch?v=GQQaF-NEDQI

Randomly, the 100ms pulse appears shorter. The frame rate of the Flea3 camera is 120fps or 8.3ms/frame so it doesn't make sense that I'm severely undersampling the TTL pulse. Anyway, to me, this is now more complicated and I'm not sure really related to Bonsai anymore- it could have something to do with the camera and I had one BNC output on my Bpod that seemed to be acting flaky. Intermittent hardware/software problems are the worst...

SO I figured I'd try a totally different approach:

One option is to just use serial communication. I tried something ridiculously simple as a first test. I wrote an arduino program that just alternately prints ones and zeros to the serial port (COM15 in this case). While inelegent, a dedicated arduino could detect a TTL pulse and then transmit it to Bonsai via serial communication:

void setup() {

        Serial.begin(9600);

        pinMode(13, OUTPUT);

}

void loop() {

                Serial.write(B00110001); // 1, UTF-8

                Serial.write(B00001010);// linefeed character: UTF-8 (binary)

                digitalWrite(13, LOW);

                delay(1000);

                Serial.write(B00110000); // 0, UTF-8

                Serial.write(B00001010); // linefeed character: UTF-8 (binary)

                digitalWrite(13, HIGH);

                delay(1000);

        }

When I run this and then use the arduino serial monitor I can see a 1 or 0 printed on a new line every second. The question is how do I pull this into Bonsai? I quit the arduino sketch but left the arduino plugged in (and blinking). Then I created a serialread node in Bonsai and set it to COM15. But when I run just this single node and inspect it I just don't see anything. I initially tried using Firmata but was a bit confused (I'm a reformed molecular biologist) so tried the aforementioned approach just to try and understand how pulling serial commands into Bonsai works.

Thanks,

Fitz

[goncaloclopes]

Hi Fitz,

Sorry to hear about your troubles. Given your report, I agree that the problem seems to be in the detection of the TTL pulse from the camera. I have a couple of questions to try and debug the situation:

1. Which GPIO pin are you using to sample the TTL pulse? Is it pin 0?

2. Could you try saving the embedded hardware frame counter to make sure you are not missing camera frames? That is one reason I can imagine for seeing shorter pulses. To do this, you first need to enable the frame counter under "Advanced Settings" in the FlyCapture camera control dialog. Then you should be able to right-click on the FlyCapture source node in Bonsai and select the frame counter from Metadata->embeddedFrameCounter. Try saving that into a file using CsvWriter and check if the numbers are always consecutive.

3. Can you write the GPIO pin state to a file as well and plot it? I would be curious to know how much shorter the pulse really is. The Flea3 GPIO pins are sampled only in s single instant at the end of frame acquisition. This means that your TTL pulses should be always at least double the inter-frame interval, to make sure you don't lose the pulse. I would expect observed pulse durations to be off by at most 2 frames (one frame at onset, and one frame at offset) which in this case would be 16ms, but I agree that the observed pulse seems to be shortened by more than that. Again, curious to see how big is the difference.

Regarding synchronization through the serial port, it should work, but be aware that using Serial over USB will introduce a latency of at least ~10-20 ms because of protocol buffering. Anyway, the sketch you made seems to make sense. However, maybe the newline separator is not being correctly specified. Can you try using Serial.println() instead just so we can make it a bit more standard? Later you can set a custom newline separator in SerialStringRead in order to customize your own separator. Also, make sure that the baudrate of the port is set to identical values on both Arduino and Bonsai. In Bonsai you can set it by clicking in the "Manage Ports" button on the COM port selector.

Hope this helps making more progress,

[Fitz Sturgill]

Switching to Serial.println worked-   using a parse node (%i) I'm now reliably triggering acquisition of my frames.    I was/am indeed using GPIO pin 0 (opto-isolated). To chase down the issues that remain I plan to use a separate TTL trigger source (NIDAQ) combined with recording the frame counter, etc. as you've suggested below. I will get back to you....