problems with point grey embedded data being saved by Bonsai, as well as acquisition delay

[jt2...@columbia.edu]

Hi! 

I am having trouble making sense of the data that is saved by Bonsai.  I have a point grey Flea3 FL3-U3-13Y3M camera and am acquiring at 30fps on Bonsai.  I am trying to save the embedded Timestamp/Frame Counter from the camera as a Zipped Csvwriter file.  See the screen shot image for the setup on Bonsai.

Sometimes, I get:

1. Columns of meaningless data being inserted into the csv file, resulting in shifts in the columns. For example, random meaningless values are inserted in a column between column 1 and 2 as you go down the file in  VideoMetadata20171216T204811.mat. The first column is timestamp from camera.  The second column column is frame counter from camera. you can ignore the other columns.

2. The timestamp seems to reset to a smaller value in cycles.  It always go from 1e+08 to 4.29+e29 and reset to approximate value. Is this value unix time? I am not sure if I should be using the following code that I found in group discussions here to convert the values:

function timestampDecoded = timestampDecoder(timeStamps)

 

    % converts metadata timestamps recorded in bonsai from point grey camera into seconds

    %

    % input:     timeStamps: metadata timestamps recorded in bonsai

    % output:    times: seconds, normalized such that first time is 0

    

    timeStamps(isnan(timeStamps(:,1)))=0;    

    % extract first cycle (first 7 bits)

    cycle1 = bitshift(timeStamps, -25);

 

    % extract second cycle (following 13 bits)

    cycle2 = bitand(bitshift(timeStamps, -12), 8191) / 8000; % 8191 masks all but but last 13 bits

    

    % account for overflows in counter

    timestampDecoded = cycle1 + cycle2;

    overflows = [0; diff(timestampDecoded) < 0];

    timestampDecoded = timestampDecoded + (cumsum(overflows) * 128);

    

    % offset such that first time is zero

    timestampDecoded = timestampDecoded - min(timestampDecoded);

   

    

end

3. Sometimes, between these reset cycles of timestamps, the corresponding frame count is also affected.  Some timess the frame count would jump to a smaller value with less digits and at the same time the timestamp would go to a stretch of NaN. If one looks closely, the small digits seems to be continuation of the count that was happening before the timestamp cycle reset.  ex. 105144 —> 145 —> 5146 —> 5147 —> 5148

After about 90 frames, the frame count would jump back to the previous frame count, and the timestamp would reset to ~1e+08.  

See bl6a3acc5id3greyofVideoMetadata20171214T224242.mat.  The first column is timestamp from camera.  The second column column is frame counter from camera. you can ignore the other columns.

4. I also notice sometimes a 2 second delay between start of acquisition (play button and key down on bonsai) of wear device data and the video frame saved data.  

I am not sure if any of this is normal.  For example, can I rely on the incomplete frame counter data that seems to occur whenever the timestamp resets.  Tonight, I recorded from the computer a few more files and they seem ok now.  But not sure what the change was. I am not confident that this would not show up again.  Please advise. Thanks

Jonathan

[Gonçalo Lopes]

Hi Jonathan and welcome to the forums!

1. and 3. Can you please attach the .bonsai file so I can check the properties of the nodes? Also, can you please attach the raw .CSV files recorded by Bonsai, rather than the post-processed MATLAB files? I just want to make sure that we constrain the issue to the raw data files themselves.

2. Regarding timestamps: yes, different parts of the timestamp value cycle in different timescales, as explained in the technical reference manual section 8.16 (page 79) and in a previous forum post. In general, I tend to prefer simply using the hardware frame counter and computing the timestamp myself, assuming the hardware frame rate will be approximately constant down to a few microseconds.

4. How are you measuring this delay? Are you comparing timestamps from the same clock? If so, which clock are you using as the basis for comparison?

Finally, just a minor suggestion. You don't need to use a combination of branches and Zip to select which fields you want to record on the CsvWriter. There is a Selector property embedded directly in the node which allows you to pick which fields will be recorded and in what order. You can condense the top part of the workflow to something like:

​

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/142f3a98-eb60-41bb-8ed2-d606beaf57c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jt2...@columbia.edu]

[Gonçalo Lopes]

Hi Jonathan,

Thank you for sharing the CSV files. I have manually scrolled down the lines of the full metadata file and I don't seem to notice any random characters. Could it be that there is an issue with character encoding in the computer on which you are doing the analysis? Can you point to a specific line number where you notice this issue?

What I do notice is the behavior of timestamp cycling, which is indeed normal behavior for PointGrey cameras.

Another possibility is that the MATLAB code for parsing and converting the files is having difficulties dealing with line separators or the strange timestamp format?

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/fce9c616-1f2d-4d53-9f3c-334d90716f22%40googlegroups.com.

[jt2...@columbia.edu]

Thanks Goncalo! I attached the files you asked in the above post. 

To measure the delay, I compared the length of time elapsed for the video itself, as well as the length of time calculated from the start and stop of the camera timestamp, to the corresponding values calculated from the accelerometer base station timestamps recording the trigger pulses.  I also recorded in later tests the camera strobe output signal, which gets a timestamp at the base station.  The result is that the video time and camera timestamps agree (for example, 6 secs) and the basestation events recorded longer pulses and camera output events (for example, 8 secs).  The strobe time length seems to be aligned well to the camera pulse signal. I also noticed a delay in the video coming on after start of acquisition relative to the pulse signals and accelerometer data stream. This suggest to me that the camera is responding to the pulses at the beginning of bonsai acquisition, but somehow there is a problem saving the frames in the beginning.  Would you agree with this assesement? How would you fix this issue?

On Saturday, December 16, 2017 at 9:29:37 PM UTC-5, jt2...@columbia.edu wrote:

[Gonçalo Lopes]

Hi Jonathan,

(Please see above for my answers to the CSV files)

Can you tell me which version of the PointGrey driver you are using? I have heard reports from previous users that version 2.10 was having issues with initialization delays. They solved it by reverting to version 2.9 or possibly upgrading to later versions.

Are you configuring the camera to report a strobe output, or are you driving it with a synchronization signal coming from elsewhere?

Be aware that if the camera is working in free-mode, you will get strobe outputs whenever the camera is connected to the computer, whether or not you are actually collecting frames from it, so it is possible the lengths don't match if the driver is taking additional time to start (or indeed if you are manually waiting to start acquisition at a specific time). If you need to control this, you have to trigger the acquisition start by hardware, or trigger every frame. You can also do it in the FlyCapture UI by manually firing a software trigger.

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/90667b12-86c4-496e-8b0c-30edaa449696%40googlegroups.com.

[jt2...@columbia.edu]

Thanks Goncalo about this. You are right. I checked again and indeed there is no shifting in the raw csv file. When I imported the files into MATLAB I chose the fixed width option rather than delimited.  Now the shifting and weird incomplete values for timestamps problem is gone.  The issue that remains is the delay which I will respond in a bit

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[jt2...@columbia.edu]

Hi Goncalo, the driver is 2.7.3.93. I am configuring it to report a strobe output. . I attached all the options in the flycapture2 software.  GPIO 1 (or pin 2 white color) is supposed to be the one that is reporting the strobe. Can you detect any problems?

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Gonçalo Lopes]

Hi Jonathan,

It seems like you are using a hardware trigger to drive the camera capture (on a frame-by-frame basis). Can you tell me how fast are you sending trigger pulses to the camera? Your shutter seems to be fixed at 57ms, which means you wouldn't be able to collect frames faster than ~17 Hz, which seems a bit on the slow side.

If the exposure time is longer than the pulse intervals, then any pulses arriving while the exposure is underway will be ignored, which would cause dropped frames. This would show up as a shorter video length, since video length is calculated by multiplying number of frames by the expected frame rate.

Looking at the file it looks like you are indeed not dropping any frames (i.e. the frame numbers are all consecutive), so if this is not the problem, I guess there is indeed some delay at the beginning.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/9c7a3262-d5e7-44a5-8e8a-dbd53db1077f%40googlegroups.com.

[jt2...@columbia.edu]

Hi Goncalo,

I am triggering with accelerometer base station.  I configured it to trigger on a 30fps basis. The shutter I kept on auto but perhaps I should specify the time to be less than half the time between camera triggers (so ~14-16ms for 30 fps acquisition).  Previously, the shutter speed was at around 14ms so I thought it was fine.  If this is indeed an initial delay problem, what would you recommend trying to solve the issue? Thanks!

Jonathan

[jt2...@columbia.edu]

Hi Goncalo, I attached here the settings for the base station in case you can detect any issue. 

[Gonçalo Lopes]

Hmmm, I'm not sure how the auto shutter would work in the triggered case. You would imagine that they would have to wait for the next pulse to arrive before ending the exposure, but then you can get delays that have to do with data acquisition and transmission before you would be ready to start the next exposure.

Maybe they have some clever solution to this problem, but I find it safer to set a fixed exposure with enough of a comfortable margin. You mentioned that you recorded both the trigger signals and the strobe outputs from the camera and that they seem to align. Can you share a picture of the oscilloscope trace?

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/1168b5d2-40b0-4a50-9cd4-39f34944f9e1%40googlegroups.com.