Different amount of data recorded for common source with MatrixWriter

[Jonathan Newman]

Hi,

The following graph produces samples from my ephys hardware and saves timestamps (1 x sample block-size) and data (64 channels x sample block-size) using MatrixWriters.

The timestamps and data are produced synchronously and have the same number of samples (columns):

However, when loaded, the timestamps always have less samples (columns) than the data does. Is this because the amount of data being pushed to the MatrixWriter handling the timestamps is less than the one handling data, and the file is file stream is not completely flushed on close? Or do am I actually losing timestamps somehow?

- Jon

[Jonathan Newman]

Well -- I just ran this graph, which was a minimal example of this phenomenon, and now I'm getting the correct amount of samples. Let me see if I can add some of the complexity back to reproduce my issue.

[Jonathan Newman]

Ok, here is graph that actually reproduces the issue. I'ved added an extension onto the data path for audio monitoring:

On Thursday, July 19, 2018 at 10:22:59 AM UTC-4, Jonathan Newman wrote:

[Gonçalo Lopes]

Hey Jon,

This seems to be an issue with AudioPlayback indeed. There is an initialization side-effect that is delaying subscription to your top branch. This means the bottom branch will actually subscribe first to the data stream. This is annoying and will be fixed.

In the meantime, can you try the following workaround?

Basically I've added a PublishSubject to decouple the effects of AudioPlayback from the data branch (you can think of it as an opto-coupler :P) This way the delayed subscription should not affect the bottom branch and the number of samples should be correct in both.

This issue only affects AudioPlayback specifically when you have branching before. I basically fumbled the implementation in one small, but important detail.

Thanks for the report and let me know if the workaround works.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/7a742c9e-445b-4797-ac40-972c3b16df52%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gonçalo Lopes]

Actually, looking again at the implementation, it should work... let me know whether the workaround does anything, I'm curious now.

[Sarah P]

Hi Goncalo,

I was wondering, if I have two input sources that are recordings at >1kHz sampling rate, is it possible to use CombineLatest and Zip to send both data along with the corresponding timestamps to MatrixWriter?

Looking at this conversation, it seems like I would have to send each data set and the timestamp to their separate MatrixWriter.

I was thinking of trying to write the code to test this out, but since this conversation uses a similar coding scheme, I thought I'd ask before I tested it out.

Thank you!

Best regards,

Sarah

On Thursday, July 19, 2018 at 10:54:07 AM UTC-4, goncaloclopes wrote:

Actually, looking again at the implementation, it should work... let me know whether the workaround does anything, I'm curious now.

On 19 July 2018 at 15:46, Gonçalo Lopes <goncal...@gmail.com> wrote:

Hey Jon,

This seems to be an issue with AudioPlayback indeed. There is an initialization side-effect that is delaying subscription to your top branch. This means the bottom branch will actually subscribe first to the data stream. This is annoying and will be fixed.

In the meantime, can you try the following workaround?

Basically I've added a PublishSubject to decouple the effects of AudioPlayback from the data branch (you can think of it as an opto-coupler :P) This way the delayed subscription should not affect the bottom branch and the number of samples should be correct in both.

This issue only affects AudioPlayback specifically when you have branching before. I basically fumbled the implementation in one small, but important detail.

Thanks for the report and let me know if the workaround works.

On 19 July 2018 at 15:29, Jonathan Newman <jonatha...@gmail.com> wrote:

Ok, here is graph that actually reproduces the issue. I'ved added an extension onto the data path for audio monitoring:

On Thursday, July 19, 2018 at 10:22:59 AM UTC-4, Jonathan Newman wrote:

Hi,

The following graph produces samples from my ephys hardware and saves timestamps (1 x sample block-size) and data (64 channels x sample block-size) using MatrixWriters.

The timestamps and data are produced synchronously and have the same number of samples (columns):

However, when loaded, the timestamps always have less samples (columns) than the data does. Is this because the amount of data being pushed to the MatrixWriter handling the timestamps is less than the one handling data, and the file is file stream is not completely flushed on close? Or do am I actually losing timestamps somehow?

- Jon

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Gonçalo Lopes]

Do you mean having two independent AnalogInputs? If they come from the same board, the preferred way is to have a single AnalogInput with multiple channels, so that all data is already sampled synchronously in hardware, and there will be no need to timestamp or zip independent streams in Bonsai.

In this specific example the two MatrixWriter nodes were used simply to logically separate data with different binary formats (i.e. 8-bit vs 16-bit) into different files.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/65110916-8b87-4c2d-a3b4-a93de9f6d75a%40googlegroups.com.

[Sarah P]

I see. That makes sense, to use one AnalogInput with multiple channels!

I would like to align the video recording from CameraCapture and data from the AnalogInput using timestamps, as a post-processing step. This is why I was thinking about saving the timestamps for the data acquired with AnalogInput and sending them to MatrixWriter using CombineLatest and Zip. Is this possible, or is there another approach to do this?

[Gonçalo Lopes]

Hi Sarah,

It is still not clear to me what your desired output data format would be. How are you timestamping the input signals? Do you have a timestamp sample-by-sample, or buffer-by-buffer? To record timestamps generated in Bonsai together with binary array data you would need to either find a way to spread out or interpolate the timestamps for every sample and save everything together, or store a timestamp for every buffer. But again, it is not clear how this would align with the video, since I'm not sure which time basis you are hoping to use to interrogate your signal.

Above you mentioned two analog input sources sampling at 1kHz, but now you are mentioning there is a video stream, are they the same data stream? It would be easier if you can describe a specific setup in a contained way (maybe in a new post?), otherwise it will be hard to guess what would be most appropriate here given there are so many different ways to do it.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/CALqeRca%2B15eY7rwYAQwUg_S9%2B34%2Bvoank9mvBqGmg5kVpaA63A%40mail.gmail.com.

[Sarah P]

Hi Goncalo,

Here is a description of my setup.

I was thinking of timestamping the analog input data sample-by-sample, but that may be too much and possibly not possible with Bonsai. This was what I was asking.

I think if I know the start and stop times of the recording, I could figure out the approximate times of each sample, based on the sampling rate. In the attached code, I put the Timestamp node to save the start and stop time of the recording. Is there a Bonsai operator that saves just the start and stop times of the recording? It would simplify the saved data.

I have a question about MatrixWriter. When there are 2 channels setup for one AnalogInput node, what are the dimensions of the array in which the data is saved? Will the data save as a 2 x N (columns x rows) array or 1 x 2N array? In the example code, when I open the binary data in MatLab with A = fread(fileID,'int16'), I only see 1 column of data and wondered if this 1 column contains the data for both channels and I should organize the data as such in MatLab.

Thank you!

Sarah

[Gonçalo Lopes]

Hi Sarah,

Thanks, this is more clear to me knowing your description. You can choose the layout with which MatrixWriter saves the data in the file (ColumnMajor or RowMajor), but bear in mind that only ColumnMajor makes intuitive sense for arbitrarily streaming long files (i.e. where you don't know the length of the time axis apriori). The reason is that if you wanted to store the data the other way around, one channel at a time, you would have to know the total number of samples beforehand before jumping to the next channel, if that makes sense.

This is why in general the default assumed by MatrixWriter is ColumnMajor, where rows represent time, and columns represent channels. Depending on how you load data in matlab, you will indeed have to reshape the data appropriately.

Regarding timestamping, I wouldn't advise just working out sample timing from start and stop of recording as there might be non-negligible clock drift, especially if you go for long recordings. It might be more helpful to just use CombineLatest again in this case to work out timing.

One trick to do this is to generate a "virtual channel" which will contain the latest frame number and which will be appended to the data coming from the NI-DAQ. The way to generate this is to use ScalarBuffer to generate a fake buffer every frame with a value set to the frame index. That way you will end up with a "staircase" like pattern in your binary files where that channel will tell you for every sample what was the latest frame that Bonsai held in hand at the time of buffer acquisition. The logic for this can be simplified into an example with AudioCapture, like so (also attached):

In this example, AudioCapture is a stand-in for my DAQ, and CameraCapture is a stand-in for your combined video source. CombineLatest is simply taking the latest frame counter buffer from the camera whenever there is more data, and appending (Concat Axis:0) that channel to the data and dumping everything into a file.

If you plot that data afterwards, you will have two channels in your file which look like this:

The left side is the "analog" raw data and the right side is our fake frame counter data, which has the staircase pattern, but basically tells me for every recorded sample what was the latest buffer in memory. As we discussed, this might have a deterministic jitter (which hopefully is negligible) but there should be no drift over time.

Hope this helps.

[Sarah P]

Hi Goncalo,

This is a great suggestion! Thank you so much!

To make sure I understand the code, may I ask some questions?

- What is the difference between Dsp nodes and Reactive nodes? I noticed Concat (and other nodes) come in both formats.

- It looks like the ElementIndex reads frames from CameraCapture. Is the Value node sending the frame number as an array of 4 elements, hence the "Index. Index. Index. Index" under the Value property? So, the frame number is repeated 4Xs?

- For ScalarBuffer, is Channels  property = 1 because there is 1 camera? Why is the Size property 441 (cols) and 1 (rows)? What do the numbers mean? I can see that the Value property is storing the frame number in quadruplicate.

Thank you!

Sarah

[Sarah P]

Hi Goncalo,

I have been trying to combine your suggestion with the rest of the code that I'm developing. I'm attaching my code and your example code for easy review.

Here is a description of my setup for easy access.

For all the different configurations I've tried, I am getting the error message, "Input arrays must have the same element type."

I tried the following configurations:

- I tried adding ElementIndex > Value > ScalarBuffer after DrawCanvas. Or, after CombineTimestamp. Or, after MakeTimestampedImage.

- I tried adding WithLatestFrom > Concat after AnalogOutput. Or, after AnalogInput > SkipUntil.

- I tried different combinations of the above node series but am getting the same error message stated above.

I think I'm missing some piece of understanding about something. Any help would be appreciated!

I have a side question. I will be expanding the AnalogInput node to have 2 channels. Since the fake frame counter data channel is sent to MatrixWriter along with the AnalogInput channels, will this mean I will reshape the binary data in MatLab as a N x 3 array, where N = number of samples and 3 = number of channels? (I ask, to make sure I understand how the binary data will be saved by MatrixWriter.)

Thank you!

Best regards,

Sarah

[Gonçalo Lopes]

Hi Sarah,

I think we are deviating from the topic of this question. Would you mind opening up a new question for this with a more suggestive title and we can take it from there? It looks like you are trying to record the combined timestamp and DAQ data as a video, which might not be possible?

Let's discuss in a new thread. Hope this helps.

[Sarah P]

Sure thing! I created a new post. Please take a look.

Thank you,

Sarah

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/65110916-8b87-4c2d-a3b4-a93de9f6d75a%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.