Recording TTL

[Lilliana Sanchez]

I have a number of questions arising from difficulties getting the TTL signal into OpenEphys. With my current setup, I don’t see a spike/line coming from BNC 4 in the LFP viewer. I do, however, see a `synchronized_timestamps.npy` output within the `RecordNode_108/experiment1/recording1/continuous/Rhythm_FPGA-101.0/synchronized_timestamps.npy`. I am using the `binary` recording mode to get python-friendly output, hence the `npy` extensions.

I want to make sure I have everything hooked up the best way: 

- In the Hiroshi cable (7-pin from the Basler Camera), which lines did you use for the TTL?  We are currently using 3(+) and 5(ground) which correspond to the yellow and white wires. 

- Once you have captured data, where is the TTL recorded in the open ephys output  (binary or OEphys format)? 

- Is there something specific I need to do in the camera settings (I assume within the pylon capture software) or open ephys to enable the ttl?

- Below in the image bottom left in the FPGA controller there are buttons that can be turned on or off. Such as the DAC ttl and the ttl settle, do you know if these need to be on off or have you had to mess with these at all?

- Are there instructions on recording ttl that you have found somewhere?

Thanks a bunch in advance!

[Josh Siegle]

Hi Lilliana,

I’m not familiar with Basler cameras, but I can try to help.

- Do you have a wiring diagram or user manual for those cables that indicates the function of each of the 7 pins? If you have the correct pins connected to the acquisition board, then you should see vertical bars for each event overlaid in the LFP Viewer. Getting these events to appear is the first step in troubleshooting the issue. You may need to use a very small timebase for visualization (e.g. 0.25 s) to see the events at high frequencies.

- There may be some setting in the Pylon capture software that toggles the TTL output when acquiring frames. If that doesn’t work, you could alternatively use an external TTL generator (such as an Arduino or Pulse Pal) to trigger the acquisition of frames. Then you could copy that signal to one of the acquisition board digital inputs, so there is an event corresponding to each frame.

- The TTL events will be recorded in the events/Rhythm_FPGA-101.0/TTL_1 directory. The timestamps in the “continuous” directory are for the continuous (neural) data from the headstages. Documentation about the various files in the Binary format is available here.

Hope that helps!

Josh

<Capture.PNG>

- Are there instructions on recording ttl that you have found somewhere?

Thanks a bunch in advance!

--
You received this message because you are subscribed to the Google Groups "Open Ephys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/6bd733ea-06ea-467f-a5b5-b3a2f848f702n%40googlegroups.com.
<Capture.PNG>

[Lilliana Sanchez]

I was able to find the wiring diagram for out Basler camera, and for future reference, at least our camera, had TWO ground wires, one for GPIO (output) including TTL signals and one for opto-coupled (input) signals. 

What I'm now seeing is that the TTL array is very different in size from the `timestamps` file in the `tracking_data` (and one is not an integer multiple of the other). Not only is the length different, but the duration is also different. Do you know why there would be a discrepancy? How can the TTL and tracking data timestamps be synced? 

N.B. We have two record nodes: one for the tracking data and one for the continuous data. Does that have any bearing on the TTL/timestamps issue?

[Josh Siegle]

Hi Lilliana,

I assume the Basler camera frames are being saved by Bonsai – do you see the same number of timestamps in the TTL array as frames that are saved by Bonsai?

Can you upload the TTL array and tracking_data events somewhere? I can take a closer look to try and figure out where the discrepancy is coming from.

Having two Record Nodes shouldn’t make a difference, but if you can send your signal chain settings.xml, that would also be helpful!

Josh

To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/b07097b1-3302-417c-8fde-73ddeea27a14n%40googlegroups.com.

[Lilliana Sanchez]

I am not saving the frames via Bonsai at this time. 

The attached `timestamps.py` file is the TTL data and `data_array.npy` is the tracking port data. 

Let me know if you need anything else and thanks!

P.S.

The fact that there are two record nodes (107 and 108) prevents `pyopenephys` from reading the data correctly. 

[Josh Siegle]

Thanks for sending this!

The pyopenephys library was developed prior to the addition of Record Nodes to the GUI, which explains why it doesn't work with your recording.

For loading the Rhythm FPGA data saved by Record Node #1, I'd recommend using the open-ephys-python-tools library, which is maintained by the Open Ephys team.

For the Record Node saving the tracking plugin data, you’ll have the load the data manually because the event data is saved in a format that's specific to the Tracking Plugin.

I was able to read in the data array like this:

da = np.load(‘data_array.npy’)

x = np.frombuffer(da.flatten(), dtype=np.single)[::4]

y = np.frombuffer(da.flatten(), dtype=np.single)[1::4]

width = np.frombuffer(da.flatten(), dtype=np.single)[2::4]

height = np.frombuffer(da.flatten(), dtype=np.single)[3::4]

The last four lines are converting the byte buffer into float values, then extracting the x and y position, as well as the image width and height values (which are constant). It would be great to have a more elegant solution for this, but this will require some additional work.

I'm seeing a total of 601 values in the Tracking Plugin data array, and 3110 TTL frame trigger timestamps. Because the TTL line is recording both "on" and "off" events, the actual number of triggers is 1/2 of the total recorded events, or 1555.

There are a few reasons why there could be more timestamps than tracking values:

1) The Bonsai workflow was started after the frame triggers started, or stopped before the frame triggers ended. To help troubleshoot this, it would be helpful to save the camera frames in Bonsai using a SaveImage operator, to confirm how many frames are actually being captured.

2) Not all frames have events that are received by Open Ephys. It possible that, due to the way OSC messages are being buffered by the Tracking Plugin, information about some frames is being lost. Do the Tracking Plugin events have timestamps associated with them? If so, those could be used to determine whether the events are being received at a constant rate, or if there are gaps.

Let me know if that makes sense!

Josh

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/ece3a2cf-6d9b-401a-b2fe-e5816dc109c5n%40googlegroups.com.

[Lilliana Sanchez]

Thanks for taking a look at this! I understand what's going on with the tracking data now, but interpreting the timestamps is another matter. 

First, the manual (https://open-ephys.github.io/gui-docs/User-Manual/Recording-data/Binary-format.html) states that the timestamps are 64bit floats as NPY format files. When reading the TTLs using numpy `load` command (or the `open_ephys` analysis package) I find that the data are integers. 

Does the data need to be scaled by the sample rate to get the time (in seconds)? If I do that, the TTL elapsed time is reasonable for how long I ran the test recording. 

However, the timestamps from the tracking plugin do not span the same time frame as the TTL. The tracking timestamp interval varies from 77,000 to 336,000, whereas the ttl timestamp intervals are consistently 150 and 220 (for on/off intervals). Again, I don't know what the units are. The TTL could be in ms, but the tracking plugin time can't be in ms or s. If one assumes that the tracking plugin timestamps are in units of the sample rate (30kHz), the total time still doesn't make sense interpreting as µs, ms, or s (it is 25,850).

I'd like to use the tracking plugin. The reason I'm not using Bonsai to track the position of the animal is that I don't think it can be synced with the TTL as that signal is going straight to BNC connector board attached to the open ephys acquisition system. Thus, finding the position of the animal will always be difficult as the bonsai tracking and open ephys electrophysiology recording won't start at the same time. 

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/ece3a2cf-6d9b-401a-b2fe-e5816dc109c5n%40googlegroups.com.

[Josh Siegle]

Hi Lilliana,

I'll send a more detailed response later this week, but here are some quick points:

- The timestamps you recorded are actually in units of sample numbers since the start of acquisition. In version 0.6.0, which was just released, we changed the convention so "timestamps" are always in units of seconds, and "sample numbers" are always integer counts. But this doesn't apply to your data, which was recorded with an earlier version of the GUI.

- If video frames are recorded by Bonsai and frame triggers are recorded by Open Ephys, you can still perform synchronization by aligning the Bonsai frames to the trigger TTL timestamps. Everything doesn't have to go through the same piece of software.

Josh

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/ece3a2cf-6d9b-401a-b2fe-e5816dc109c5n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Open Ephys" group.

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/20c5be91-d955-48be-9f0c-3dc47a4e0bd7n%40googlegroups.com.

[Lilliana Sanchez]

Thanks again for your advice. 

My attempt to use the `tracking plugin` is preventing me from updating OpenEphys. I don't know how to recompile it on windows or use visual studio (I'm a linux programmer). 

I was back in the lab today, and I found that the tracking plugin is not the reason for the inconsistent timestamps interval in the tracking data. It appears that Bonsai is having trouble recording and processing data (even at 50Hz). I know this as output directly from bonsai of the LED coordinates is fragmentary (using the CSV writer) where many of the frames are reporting NaNs or zeros. The position is only irregularly reported. In addition, the video visualized directly within bonsai is lagging by a few seconds and the image has lines. I've tried another Basler camera from a neighboring lab, and the same result occurs. 

In short, I think something is desperately wrong with Bonsai or its interface with Basler cameras, via the Pylon capture plugin. 

Can you recommend where I might get help diagnosing my problem?

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/ece3a2cf-6d9b-401a-b2fe-e5816dc109c5n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Open Ephys" group.

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/20c5be91-d955-48be-9f0c-3dc47a4e0bd7n%40googlegroups.com.

[Josh Siegle]

We are planning to update the Tracking Plugin to work with the latest version of the GUI in the near future, so you shouldn't have to compile it yourself!

To get help troubleshooting your Bonsai workflow, you should post on the Bonsai GitHub Discussions page.

Josh

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/ece3a2cf-6d9b-401a-b2fe-e5816dc109c5n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Open Ephys" group.

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/20c5be91-d955-48be-9f0c-3dc47a4e0bd7n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Open Ephys" group.

To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-ephys/c60a0700-a719-4066-84e2-a29edb20e53fn%40googlegroups.com.