External TTL input

[Nikolas Karalis]

Is it possible to record external TTL inputs via the AUX Input port?

[Nikolas Karalis]

Perhaps more generally:

What is the suggested way to sync the miniscope with other events happening?

It seems the easiest would be if TTLs could somehow be recorded by the miniscope DAQ, but it is not clear to me if this is possible.

Thanks in advance! :-)

[Daniel Aharoni]

Hey Nikolas,

While the hardware is capable of recording external TTL inputs the USB data protocol (UVC) we currently use between the Miniscope DAQ and computer isn't really made for this sort of thing. For this reason, we don't currently support this functionality in our DAQ/Software. With that said, we are working on switching over all Miniscope devices to use a more generic vendor class USB protocol which would then totally facility this sort of thing. Once switched over the DAQ and other future DAQ devices will all be able to do this.

To your more general question, there are 2 ways to sync the Miniscope with other devices. One is to use the Sync output of the DAQ and read that into another device. This output toggles a 3.3V signal high and low on ever frame acquisition. The other option is to use the TTL input trigger on the DAQ to externally trigger the Miniscope recording. You can then sync the start of recording with other devices.

[Nikolas Karalis]

Hi Daniel,

thanks a lot for the info.

The reason I was asking is that for simple synchronization of events, a single TTL recorded in the miniscope software would be a very simple and efficient solution not requiring any other device/synchronization.

But in this case, I will go for the more general frame-out synchronization.

Best,

Nikolas

[kim...@gmail.com]

Hi Nikolas and Daniel,

I am actually using miniscope V3 in sync with Neuralynx using TTL toggle signals, and I found some difference between TTL timestamp in the Neuralynx and miniscope timestamp.dat file, which makes me less confident about sync 2 data. Maybe my problems are also helpful for Nikolas to set up experiments.

Question 1. In the attached picture 1, I plotted all miniscope TTL signals sent to Neuralynx from miniscopeDAQ board, which are 30Hz (and 15Hz toggle) and I found there is 1 "extra" signal sent to Neuralynx around 1850second and 2 another between 2200-2400second. How can I interpret this? I noticed this does not happen always, but quite often. I don't see it this signal from the miniscope timestamp.dat file. Should I ignore it, or count as a frame? 

Questions 2. In the attached picture "first_frame", I see sysClock of the very first frame is a big number and I read from other conversation that it is meaningless number. 

But then, the 2nd frame start from 4ms. Then when was the 1st frame recorded? Should I deduct a time of 1 frame, so (4 -33)ms something?  

Questions 3. In the attached picture "first_frame", I noticed that each frame was not recorded at a constant speed. For example, it is 34ms between frame 2-3, 39ms between frame 3-4, 74ms between frame 6-7, and 5ms between frame 7-8. 

And when you look at TTL signals in the Neuralynx (attached picture 3), what Neuraylnx got from TTL toggle signal is very constant, always  66.7ms  (at 15Hz) between each signal.

So I am wondering if the TTL signal is actually from separate system, independent from the frame acquisition? I noticed this problem while sync my two data, at the end the total miniscope frame was always less than 2* TTL signal in the Neuralynx, and I dont know how to understand this gap between two numbers.  

Hopefully you have any idea what is happening here? Looking forward to hearing any suggestion!

Best,

Chae

2021년 1월 28일 목요일 오전 11시 28분 48초 UTC+1에 nikolas...@gmail.com님이 작성:

[kim...@gmail.com]

Hi Daniel, 

About my problems that I posted, I was thinking what could cause this.

It is possible that when I press stop to end the acquisition, that the miniscope software stop collecting images immediately, but there is some delay before the camera stops producing images or that there are already images in the buffer that the software then skips? In this case, can I crop the last few frames from Neuralynx as they may not have a corresponding image?

Or, if something similar happens at the start, i.e. the camera starts producing images, but the software is not quite ready yet to receive the images and thus the first few timestamps in Neuralynx do not have a corresponding image frame. How likely it is that this happens?

Thank you for your help!

Best regards,

Chae

2021년 2월 26일 금요일 오전 2시 29분 25초 UTC+9에 kim...@gmail.com님이 작성:

[kim...@gmail.com]

Hi again,

I noticed an unusually long gap between some timestamps of miniscope frames (attached picture: image.png), but the buffer (column D in the attached Excel file and image.png) is not incremented. 

I am afraid that there are actually drops of frames. Isn't the buffer supposed to store up to 256 frames before starting to drop frames?

PS: I already checked that the sync signal from miniscope to Neuralynx is constant at 30Hz.

For information, this is the specification of my hard drive (laptop):

• 1 x Samsung MZ-YLF1280 128GB SATA 6GB/s 2.5" SSD Drive MZYLF128HCHP-000L2

Specification

• Brand: Samsung
• Series: 830
• Model No: MZ-YLF1280
• P/N: MZYLF128HCHP-000L2
• Device: Solid State Drive(SSD)
• Capacity: 128GB
• Interface: SATA
• Data Transfer Rate: 6GB/s
• Sequential Read: 520 Mb/s
• Sequential Write: of 320 Mb/s

Thanks a lot for your help.

Best regards,

Chae

2021년 2월 26일 금요일 오전 6시 34분 40초 UTC+9에 kim...@gmail.com님이 작성:

[Daniel Aharoni]

Hi Chae,

These are all great quesitons.

• Question 1. In the attached picture 1, I plotted all miniscope TTL signals sent to Neuralynx from miniscopeDAQ board, which are 30Hz (and 15Hz toggle) and I found there is 1 "extra" signal sent to Neuralynx around 1850second and 2 another between 2200-2400second. How can I interpret this? I noticed this does not happen always, but quite often. I don't see it this signal from the miniscope timestamp.dat file. Should I ignore it, or count as a frame? 

These "extra" toggles seen by your Neuralynx system being sent out by the Miniscope DAQ are indeed frames acquired by the Miniscope DAQ but dropped on their way to the Miniscope software. You should effectively ignore these toggles as the acquired frame that generated them is not saved in the recorded .avi files. As you mentioned, the timestamp file generated by the software suggests this as it will show a larger inter-frame interval when this happens.

• Questions 2. In the attached picture "first_frame", I see sysClock of the very first frame is a big number and I read from other conversation that it is meaningless number.  But then, the 2nd frame start from 4ms. Then when was the 1st frame recorded? Should I deduct a time of 1 frame, so (4 -33)ms something?  

So the Miniscope, Miniscope DAQ, and Miniscope software are continuously acquiring frames once everything is powered and booted up. When you hit record, the next frame received by the software becomes the first saved frame. Since the Miniscope might have just started acquiring that frame or might already almost be done acquiring that frame when you hit record, the actually timing of the first frame is a bit undefined. Generally speaking though, subtracting the expected inter-frame interval time (~33ms for a 30FPS recording) from the second frame's timestamp should give you an accurate acquisition time for the first frame.

• Questions 3. In the attached picture "first_frame", I noticed that each frame was not recorded at a constant speed. For example, it is 34ms between frame 2-3, 39ms between frame 3-4, 74ms between frame 6-7, and 5ms between frame 7-8.  And when you look at TTL signals in the Neuralynx (attached picture 3), what Neuraylnx got from TTL toggle signal is very constant, always  66.7ms  (at 15Hz) between each signal. So I am wondering if the TTL signal is actually from separate system, independent from the frame acquisition? I noticed this problem while sync my two data, at the end the total miniscope frame was always less than 2* TTL signal in the Neuralynx, and I dont know how to understand this gap between two numbers.  

The Miniscope software uses the high-resolution CPU timer to timestamp the arrival of new frames. This timer, along with variable USB latency, causes the inter-frame timing saved in the timestamp file to vary a bit (usually by a few ms). In actuality, the Miniscope acquires frames at a very stable interval and this stability is reflected in the toggling Sync output of the Miniscope DAQ as you have seen. 

So to summarize,

• The frame rate of the Miniscope itself is super stable. If you need better accuracy of frame timing than what the Miniscope software gives you then you should use the timing of the outputted toggle signal of the Miniscope DAQ.
• The Miniscope DAQ's toggle signal happens every time a new frame arrives to the DAQ from the Miniscope. Unless there is an issue with your coax cable, this toggle signal should basically always shows a consistently timed square wave with no dropouts.
• Frames being sent from the Miniscope DAQ to your computer have a chance of being dropped. This happens due to USB bandwidth and timing constraints and is mainly dependent on your USB port, USB drivers, and what other devices are sharing the USB bus. While the software does have a large frame buffer, dropped frames happen before ever making to the software's frame buffer. On a decent setup you likely will see a dropped frame every few thousand frames or so.
• Dropped frames never make it into the timestamp file or the saved avi files so you basically don't have to worry about them as long as they are not common. The result is just a slightly larger inter-frame interval when this happens.
• If syncing this using the Miniscope DAQ's toggle output, toggles will show up even from frames that end up getting dropped over USB (the Miniscope DAQ has no way of knowing when this will happen). You can compare the timing of the toggles with the timestamps of the software to find and throw out toggle events that ended up being for dropped frames.

The Tank Lab published a paper using the Miniscope hardware and describes how to address toggles of dropped frames. This might be of use to you: https://www.cell.com/neuron/pdfExtended/S0896-6273(18)30852-3

[kim...@gmail.com]

Hi Daniel,

Thank you very much for your kind explanation, it helps me a lot.

Have a good weekend :)))

Best regards,

Chae

2021년 2월 26일 금요일 오후 8시 7분 57초 UTC+1에 dbah...@gmail.com님이 작성:

[Eric Melonakos]

Hi Daniel,

We are wrestling with similar issues to those in this thread, trying to sync Miniscope V4 frames (DAQ V3.3) via TTL toggles to a Neuralynx system. Similar to the Tank Lab paper you shared, we look for dropped frames by comparing the Miniscope timestamps to the timing of the TTL toggles. However, using this method, there is no way to know if the first frame was before or after the first TTL toggle (which affects the TTL toggles I compare all subsequent frames to). I think this can be addressed using your suggestion to trigger the start of recording with a TTL input and then using the time the TTL input trigger was sent to the Miniscope DAQ to translate the Miniscope timestamps into Neuralynx time. I have a few questions about how that would work:

Here's an example recording's timestamps:

Questions:

1. Does time 0 in timeStamps.csv indicate the time that the TTL input trigger arrived (or the software button was clicked) to start the recording?
2. I'm trying to understand how the order of TTL input trigger, first TTL toggle, and first frame acquired (first timestamp in timeStamps.csv) in time. Here's two options that I can think of:
   
   or, alternatively,
   
   Obviously, if Option A is what happens, then there is no TTL toggle corresponding to the first acquired frame.
3. If the two systems (Miniscope/Neuralynx) shared a common clock, would TTL toggles *always* happen in time before their corresponding timestamps in timeStamps.csv, or can they happen after? I.e., are TTL toggles triggered prior to timestamping the frame acquisition?

Thanks for your help!

-Eric

[Eric Melonakos]

Hi all,

Just an update on the above issue:

In applying the methods described in the Tank Lab paper (pages 48-50), we were surprised to see that sometimes a timestamp was advanced in time relative to the corresponding TTL. Upon further investigation, we found out that TTL pulses were also occasionally dropped (29 out of a total of 276080 TTLs in our ~2.5 hr test recording at 30 Hz). It's not a lot of dropped TTLs, but it could throw off time syncing if not accounted for. One possible reason is that the Neuralynx DAQ we have (Digital Lynx SX) is made to receive TTLs that are 5V, and the Miniscope DAQ outputs 3.3V pulses. On the other hand, the vast majority of the Miniscope DAQ’s 3.3V TTLs are recognized by the Neuralynx DAQ, so who knows. Since the timing of them are normally so regular, it was easy to identify where they were missing and insert guessed timestamps, but I thought it would be good to report this possible issue.

As a separate question, can the Miniscope DAQ hardware safely receive 5V TTLs to trigger the start of miniscope recording?

Thanks,

Eric

[Daniel Aharoni]

Thanks for the follow up Eric. This is a lot of interesting information. I think if a frame is fully received by the Miniscope DAQ it should be very rare, if not impossible, for the DAQ not to output its sync toggle signal. Maybe there is a chance the frame data had an issue going between the Miniscope and the DAQ and therefore never hit the part of the DAQ firmware code that toggles the sync output.

As for the DAQ being 5V tolerant, you will need to use the DAQ sold by OEPS, https://open-ephys.org/miniscope-v4/daq, as they have modified the DAQ to have 5V tolerant digital inputs. Other Miniscope DAQs might also handle 5Vs ok but we have seen mixed results.

[Eric Melonakos]

Thanks Daniel! That's the DAQ version/vendor we chose. Do you know the answers to my original 3 questions? I.e., (1) what is the relationship between the start of recording and timestamp = 0, (2) where is the first TTL relative to the start of recording and first acquired frame, and (3) what is the general timing of TTLs relative to the associated timestamps in timeStamps.csv? (see above for details) Any help you can give is appreciated!

[Daniel Aharoni]

Hi Eric,

1) I think I answered that previously in more detail but the relationship of the first recorded frame's timestamp is relative to when the software loggest the "start recording" trigger. 

2) This is a tricky to answer due to how triggers and commands get buffered. The best thing to do would be to directly test this. Theoretically the first TTL toggle should match with the first acquired frame but this might not always be the case due to how the command buffers work and if you are triggering recording manually or remotely.

3) The TTL toggle happens the moment the DAQ PCB receives the end-of-frame signal from the Miniscope which occurs right as the last pixel data gets sent. This data then gets packaged over USB and received by the software before the software records its own timestamp. So the TTL toggle signal will ALWAYS occur before the software timestamps the frame.

[Michael Graupner]

Hello, 

I would like to follow up with a related question in this very instructive exchange. 

I am externally triggering  the DAQ to launch a miniscope recording (Input Trigger on the DAQ). Simultaneously, I am recording the "Sync Output" of acquired frames from the DAQ. All this is done through a NIDAQ board controlled by ACQ4. From the relative time between trigger and the first TTL toggle of the sync output, I compute that the first frame is acquired ~110 ms after the trigger start. However, the miniscope timestamp file tells me that the first frame occurred 12 ms before the trigger, i.e., the  time stamp (ms) of the first frame is negative. How can this discrepancy be explained? 

Based on what Daniel explained before, I trust the sync output in order to determine the exact frame times. However, why would the logged "start trigger" time occur after the first frame arriving on the computer if the delay between trigger start and first frame is >100 ms? 

Thank you for your help in advance. 

Best,
Michael 

[Daniel Aharoni]

Hi Michael,

Thanks for the detailed investigation into triggering and timing of the Miniscope DAQ. The simple answer is the way I implemented external triggering is non-optimal and likely can result in a timing discrepancy of 10 to 100 ms. I think this is mainly due to how the DAQ and software handle this trigger. Roughly speaking, the DAQ needs to talk with the software to tell it it received a trigger, then the software needs to send the DAQ the actual signal to start recording. We limit these sorts of communications to happen between individual USB frame transfers which can mean that 1 to 3 frames go by as this communication is sent back and forth.

It still should be the case that the "Sync Output" gives proper timing of frames being acquired and also that the software generated timestamps will be accurate to within a few ms between different data types streaming through the DAQ software.

--
You received this message because you are subscribed to the Google Groups "Miniscope" group.
To unsubscribe from this group and stop receiving emails from it, send an email to miniscope+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/miniscope/cf4270f2-eb3a-481e-9a25-d7982241e13bn%40googlegroups.com.

--

Daniel B. Aharoni, PhD

Assistant Professor
Dept. of Neurology, UCLA

DBAh...@gmail.com
DAha...@mednet.ucla.edu

[Eric Melonakos]

Hi,

I'm seeing huge jumps in timeStamps.csv that do not correspond to dropped frames. I have a Miniscope + Neuralynx setup like Chae describes above, and I am comparing the TTL event timestamps Neuralynx creates from the Miniscope DAQ's Sync Output with the timestamps in timeStamps.csv in order to look for the timing of dropped frames in my recording (similar to the method described in the Tank Lab paper, pages 11-13 of Methods S1). Briefly, I first compare the number of timestamps between Sync Output and timeStamps.csv to see how many dropped frames to expect. Then, I look at the difference between the time of each Sync Output and its corresponding timestamp in timeStamps.csv. When there is a dropped frame, there is a "step" in the difference between them, and the total size of the steps across the recording should equal the number of dropped frames * the size of a timestep (e.g., for a 30 Hz framerate with 3 dropped frames, I'd expect 3 * 33.3 ms = 100 ms total of steps). Here is an example, where there is a single dropped frame at frame number 4219, and two dropped frames at frame number 267942:

In the above and most other recordings, the steps are integer multiples of the timestep size, and the total size of all the steps matches the expected number of dropped frames. However, in about a third of my recordings, there are huge jumps in timeStamps.csv (multiple seconds) that are way longer than expected from the number of dropped frames. For instance:

In this recording, there were 7 dropped frames expected, and there are three steps: one 100 ms step at frame 28496, one 133 ms step at frame 28857, and one 11.4 second(!) step at frame 9322. Since the 7 missing frames are explained neatly by the other two jumps, I am inclined to think that the 11.4 second step is an error. Importantly, as seen below, the large step is not due to any irregularity in the Sync Output signal (left), but is only due to a large timestep in timeStamps.csv (right).

I also have another recording with a 3 second step with zero expected missing frames. Has anyone else seen this issue? If so, do you know the cause, or if the huge steps can safely be ignored when using Sync Output to time calcium events as I have described?
-Eric