Handling PointGrey timestamps

[goncaloclopes]

Hi everyone,

Happy New Year!

In light of some offline discussions, there is some useful information to share regarding handling PointGrey's binary timestamp format. You can find the description of the format and how to interpret it in the reference manual, section 8.15 (page 69). In short, you cannot interpret the numbers coming out of the embedded metadata timestamp field directly as timestamps as there are a number of issues (see below).

It turns out the format is cyclical: part of the number keeps track of how many 1 second cycles the camera has done. In fact, there's two cycles embedded inside (one for each 1 second interval and another for each 125 microsecond interval). Unfortunately, the seconds cycle counter only goes from 0 to 127, meaning that the timestamp will effectively reset every 128 seconds. I guess they were focusing more on keeping track of times for ultra-high speed cameras than for long recording periods...

So, how to convert this format to total seconds? First, if you look in the format, the last 12 bits of the representation are supposed to cycle every 125 microseconds. This is a MUCH faster timescale than what a regular USB camera is sending out (e.g. 120 Hz effectively means you get a frame every 8 milliseconds). Also the exact representation of those 12-bit seems to be camera-dependent (they even say you should discard the last 4 bits for USB cameras), so what can be done typically is simply dropping those 12-bits entirely!.

The following Python routine will translate a PointGrey timestamp to seconds with sub-millisecond resolution (make sure that time is an integer data type or integer NumPy array, or else the bit manipulation operations will not work):

def converttime(time):
    #offset = time & 0xFFF
    cycle1 = (time >> 12) & 0x1FFF
    cycle2 = (time >> 25) & 0x7F
    seconds = cycle2 + cycle1 / 8000.
    return seconds

Now, we still have the problem that this representation will cycle every 128 seconds. To uncycle it, we need to find where it's cycling. Fortunately, time only moves forward :-) so we can get where the cycles are by looking for the points where the difference in time is negative. Then you only need to accumulate those cycles and shift the cyclic time accordingly:

def uncycle(time):
    cycles = np.insert(np.diff(time) < 0, 0, False)
    cycleindex = np.cumsum(cycles)
    return time + cycleindex * 128

Hopefully this will be helpful information for anyone trying to interpret the PointGrey timestamps.

Cheers!

[Richard Warren]

Thanks for this! Here's a matlab version that appears to work if anyone needs it:

function times = timestampDecoder(timeStamps)

    % converts metadata timestamps recorded in bonsai from point grey camera into seconds

    %

    % input:     timeStamps: metadata timestamps recorded in bonsai

    % output:    times: seconds, normalized such that first time is 0

    

        

    % extract first cycle (first 7 bits)

    cycle1 = bitshift(timeStamps, -25);

    % extract second cycle (following 13 bits)

    cycle2 = bitand(bitshift(timeStamps, -12), 8191) / 8000; % 8191 masks all but but last 13 bits

    

    % account for overflows in counter

    times = cycle1 + cycle2;

    overflows = [0; diff(times) < 0];

    times = times + (cumsum(overflows) * 128);

    

    % offset such that first time is zero

    times = times - min(times);

    

end

[Mike Gacho]

Thanks Gonçalo! :)

I have some naive questions regarding this. I've now started dealing with pgrey timestamps and I've changed my Bonsai workflow according to what it has been suggested in this post: 

https://groups.google.com/forum/#!searchin/bonsai-users/timestamp%7Csort:date/bonsai-users/Wq2Bo1DnCD8/jb0BfvIVAgAJ

But I'm still not sure how to read these numbers correctly. In the example csv file attached, I understand that in the column the first number before the space is the frame counter, while the second number after the space is its timestamp, right? 

What do timestamps in this format represent? Is there any reason why the timestamps have 9 or 10 digits and no more in my example? I noticed that at row 3556 of the csv file the timestamp decreases from 10 digits to 6 digits, and then starts incrementing gradually again. Is this happening for the cycling of the format? Which part of the number corresponds to each of the two type of cycles here? 

Maybe these are obvious questions, but I'm struggling  to figure out how it works. Let me know if I need to explain it better in case.

Meanwhile I will try to see if with the Python routine I will get things more clear

I've attached also the workflow that I'm using to trigger the pointgrey camera from digital input of Arduino and that generated this csv file in the recording example.

Thanks!

[Gonçalo Lopes]

Hi Mike,

Yes, it can be hard to wrap your head around the native timestamp format from PointGrey. I had to struggle with it myself, so don't worry, these are all natural questions.

First of all, the easiest way to start understanding what these numbers mean is to forget about their "text" representation, and just interpret them directly from their binary representation. Did you take a look at the reference manual? I just realized that in the latest version you should look into section 8.16 (page 91).

In short, the number you are seeing is a 32-bit number, split into 3 components: the seconds (7 bits), the cycles (13 bits), and the offset (12 bits). These numbers will all cycle throughout a continuous recording, and as I mentioned in the original post, the seconds component can only represent 128 seconds of data, so you need to be clever about how you convert the data in order to detect when there is cycling and keep time moving forward.

Did you try using the Python code examples I included in the original post? Before using them you should first convert the second column of the CSV text file into an array of 32-bit integer numbers (int32). Once you have these numbers, you can call the function converttime to convert them back to fractional seconds. For example, looking at your first 3 timestamps, you have:

319619333
320710480
321802659

converttime(319619333)
=> 9.538
converttime(320710480)
=> 9.57125
converttime(321802659)
=> 9.604625

So it looks like you are running a ~30Hz camera capture.
After you have converted the whole array, you should be able to call uncycle on the result, so you make sure to keep accumulating time even after the absolute timestamp limit.

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/81870ac9-93f9-4fa7-9496-e3991d355512%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Mike Gacho]

Hi Gonçalo,

Thank you for the reply. I tried the python codes and they work properly, I got the same numbers you wrote. So great!

One more question for me to understand. I guess the first timestamp (9.538 sec) represent the time that passes between the moment I start the workflow in Bonsai and the moment in which I send the trigger and the camera start to acquire frames, right?

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Gonçalo Lopes]

Hi Mike,

Actually, not exactly. These timestamps are computed internally in the camera, and they start counting time from the moment the camera is first powered. If you want it to represent relative time, the easiest way is to probably subtract the first timestamp from all the calculated timestamps.

Hope this helps.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/6300a898-51ca-4502-96e6-c0fe931d98fb%40googlegroups.com.

[Nick E]

Hi Goncalo,

Perhaps this is a silly question, but are these python scripts intended for post-processing of the timestamps, or within a PythonTransform node between the timestmap node, and the Zip (or whatever output).

Cheers,

Nick

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Gonçalo Lopes]

Hi Nick,

These were intended as post-processing, but I guess you could adapt them to work online as well. I am planning to potentially include a timestamp extraction operator directly in the PointGrey package in the future to decode these timestamps.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/92de67a9-5b2d-4fa0-b778-ef21d1315d51%40googlegroups.com.

[Alex Fanning]

Hi,

I have a metadata source node for timestamps related to each frame for fluorescence traces, but I do not have one for TTL pulses sent via an arduino using DigitalInput. How can I include a metadata source node for timestamps for the workflow related to TTL pulses?

Alex

[Gonçalo Lopes]

Hi Alex,

Can you elaborate a bit on what you mean? Are your TTL inputs coming through the Arduino related in any way to the video timestamps?

On second thought, this looks like a potentially unrelated question. Would you mind starting a new thread describing your problem? If you can share some example workflow, that would be extremely helpful.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/297bd857-cbbe-4cde-a0bc-29a0708cf711%40googlegroups.com.