Timestamp and keydown

[Margaux Silvestre]

Hi,
I would like to detect an object in a ROI Activity during a certain amount of time. For now, I play my Bonsai workflow, then press a Keydown when I want to start the detection and finally the workflow stops after 3 minutes (for example). The Timestamp starts as soon as I press play and so I don't have exactly 3 minutes of detection. How can I do to make the Timestamp starts when I press the Keydown and stops 3 minutes later? I joined my Bonsai file.
Thank you very much in advance for your help,
Best,
Margaux

[Gonçalo Lopes]

Hi Margaux,

Sorry for the late reply. It all depends a bit on what you want to do. If you need a branch to start exactly when you hit a key, you can use SubscribeWhen instead of SkipUntil to make sure that all initialization effects (e.g. starting a camera) happen only when you hit the key.

Another way, if you are interested in comparing relative timestamps that are generated only after you hit the key, you could move the Timestamp node to be after the SkipUntil. Critically, I think you want to branch the Timestamp only after SkipUntil, to make sure that the Zip pairings make sense (otherwise, you are pairing timestamps from the beginning of when you started Bonsai with centroids that are recorded only after you hit the key).

I see that now you are using a PythonSource to determine when the interval is over. However, in that way the python timer will count time from the beginning of start rather than from when you hit the key. An easier way to solve this would be to simply set up a Condition on the output of the RelativeTime node that you have already defined. You can simply test in a branch after Zip whether the total number of seconds is larger than your desired time and then feed that into the TakeUntil instead of the python timer.

If you want to take multiple of these intervals, you could use a Repeat node at the end to start the whole process again. However, using Repeat will save all the data into the same file. If you want multiple files, then you would need to use TriggeredWindow to cut the data into pieces instead of SkipUntil, and then group all of the recording logic inside a SelectMany.

I don't have time to sketch up an example right now, but let me know if you would like me to do it later.

Hope this helps,

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/7d9f2302-f811-42b8-94ec-a36dd54aa0a2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Margaux Silvestre]

Hi Goncalo,

Thank you very much for your answer.

I finally use the "Timestamp after the SkipUntil node" solution because when I was using SubscribeWhen, the workflow was not calculating the Background Subtraction before pressing the Keydown.

You anticipated my thoughts with the Python transform :). So now I am using GreaterThan and Condition to stop the recording. It is working perfectly, thank you very much.

For the Repeat node, I have few questions.
Is it possible to define a number of repeats I want to do? I tried with Take but it didn't work.
Is it possible to repeat only the part after the KeyDown? Or is it complicated?
Last question, is it possible to repeat the sequence but with different amounts of time each time?

Thanks again,
Best regards,

Margaux

[Gonçalo Lopes]

Hi Margaux,

You can use the node RepeatCount in order to limit the number of repetitions.

If you want to repeat only the part after KeyDown, you need to use a TriggeredWindow node instead of the SkipUntil. You should connect the KeyDown as the trigger exactly the same way as with SkipUntil. What this node does is to slice the entire sequence of frames coming from the camera into multiple sub-sequences, separated by when you hit the key.

You can then specify what you do with each of these "sub-sequences" by using a SelectMany node (this node literally reads as "from each sub-sequence, select many outputs - or tracking results in this case). In this case, you can simply group all of the nodes you had after SkipUntil inside the SelectMany node. The input to the nested node represents all the frames coming in from one sequence - you can just think of it as the frames coming through each trial, Bonsai will manage everything else.

If you want, inside this sequence you can also use a PythonSource to generate a random number and pass it as a parameter of the Condition.

Let me know if the explanation is not clear and I can try and sketch a quick example.

Hope this helps,

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/2e53df4d-7bf8-416e-86da-7b90a4280f78%40googlegroups.com.

[Margaux Silvestre]

Hi Goncalo,

I am sorry to disturb you again but I tried to follow your instructions and it didn't work. Probably because I didn't understand... So I replaced the SkipUntil by TriggeredWindow and then I added a SelectMany where I grouped all my nodes and I finally I added a RepeatCount node at the end of the sequence. So the workflow repeats the sequence the number of time I want to and without pressing the KeyDown between each sequence, this is great. But for the beginning, it is starting the sequences right away, without taking care of the KeyDown and never really stop at the end. Ideally, I would like to play the workflow, wait few seconds to calculate the Background and then press the Key to start the 3 or 4 repeated sequences. I probably didn't understand what TrigerredWindow is doing. Could you help me with that?

I don't really need a random number because I want to do sequences with a defined amount of time. For example, I would to start with 15 min, the 5 min, then 5 min and finally 10 min. Do I still need a Python transform to do that? If yes, I am not sure how to do it.

Thanks again,
Best,

Margaux

[Margaux Silvestre]

I forgot to join you the Bonsai file. Here it is.

Best,

Margaux

[Gonçalo Lopes]

Hi Margaux,

TriggeredWindow splits the sequence into sub-sequences with the boundary between each sequence determined by KeyDown. This means that the start of the sequence is also a sub-sequence itself (which will end on the first key press).

However, you can easily get rid of it by adding a Skip(1) after the TriggeredWindow node. This will drop the first sequence and only accept data coming in after the first key press.

In order to solve the different times problem, I guess you could Zip each sub-sequence with a number that specifies how long the Condition will wait. You could load them from CSV or from a PythonSource, Zip them with the output of TriggeredWindow (this will pair each sub-sequence with the interval time, in order).

You will then get inside the SelectMany a tuple pair (sub-sequence, number). Because the input is not simply a sequence, you now need to select the sub-sequence explicitly, and use Merge to get all the frames from it. You can then on the other branch feed the number into the Condition and it should work.

This way you also don't need the RepeatCount, since the number of repetitions will be implicitly determined by the number of time intervals you feed into the Zip (i.e. the Zip will complete as soon as all the numbers have been consumed).

Let me know if something is not clear, it is good to practice these explanations in order to make them more intelligible :)

Hope it helps,

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/98107118-0ce2-4aed-9808-74b7abcaeda7%40googlegroups.com.

[Margaux Silvestre]

Oh, I didn't know that the start is also considered as a sub-sequence. It makes sense now.

Skip(1) is working perfectly. Thank you very much.

I tried to solve the different times problem but for now, I don't have a lot of success. So I zipped a CSV file with the output of TriggeredWindow and it is working. In the SelectMany, I know that Source.Item1 is my sub-sequence and Source.Item2 is the time from my CSV file. But Bonsai is not happy with the Merge node after the sub-sequence. Moreover, I really don't know what to do with my Source.Item2 because I can't give it directly to Condition or GreaterThan and I have to link it to my Timestamp.

Many thanks,
Best,

Margaux

[Margaux Silvestre]

I forgot the file AGAIN.

[Gonçalo Lopes]

Sorry for the delayed reply. Were you using the Merge node from Dsp package? That is used for multi-dimensional buffered data. You should instead use Merge from the Reactive package, which should work.

Regarding what to do with Item2, if you are using CsvReader you can specify that the lines in the CSV file should be parsed as real numbers by specifying "%d" in the scan pattern (you can pick it from the drop down editor). In that way, instead of the output from the CSV being the lines of text, you get a sequence of numbers.

Then you could use CombineLatest to combine the RelativeTime node with this input. GreaterThan will work directly on the result of CombineLatest, by checking whether the first input is greater than the second input. That should be all you need.

Hope this helps!

On 19 May 2017 at 18:30, Margaux Silvestre <silvestr...@gmail.com> wrote:

I forgot the file AGAIN.

--

You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/8acb9934-70ab-40b1-9535-fb7a7adadeea%40googlegroups.com.

[Margaux Silvestre]

Indeed, I used the Merge node from Dsp package instead of the Reactive package. Now, it is working perfectly.

I think all your instructions are working fine. But now, I have a problem with the Timestamp. It is staying at zero all the time. I tried to put it after the Merge node, before the Merge node and even outside of the SelectMany node. I do not understand what's going on? 

Sorry for bothering you and thank you for your help,

Best,

Margaux

[Gonçalo Lopes]

Hi Margaux,

That's strange. Could you attach your current workflow so I can have a look?

--

You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/59026413-2ceb-4f59-a49d-ba0a4c3109c4%40googlegroups.com.

[Margaux Silvestre]

Here it is.

Best,

Margaux

[Gonçalo Lopes]

I think you may need to reverse the order of connections in the CombineLatest before the GreaterThan node, since you want to check when RelativeTime is greater than the maximum interval, and not the other way around, right?

I've gone through the workflow and fixed that along with two other minor things that may help as well:

 1) Instead of comparing only the time of day you can actually compare the full timestamps directly and get the total number of seconds in the delta

 2) RepeatCount should not be needed now, since the number of repetitions is given simply by the number of lines in the CSV file

Hope this helps. You may have to insert all the correct filenames in the workflow again.

On 20 May 2017 at 19:24, Margaux Silvestre <silvestr...@gmail.com> wrote:

Here it is.

Best,

Margaux

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/44e0427d-b52d-4be1-8636-d24ba14b8177%40googlegroups.com.

[Margaux Silvestre]

Thank you very much for your precious help.

So I tried the workflow you sent me and it is stopping right after the first sub-sequence and do not do the other ones. I am feeling vey stupid because it is supposed to work now ...

Best,

Margaux

[Gonçalo Lopes]

Strange, it did work when I tried it. What exactly is stopping, is it the entire workflow, or just the part inside SelectMany? This workflow is designed to stop after all the lines in the CSV file have been consumed, but maybe this is not what you wanted.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/16605132-290d-4277-a3fe-87cdc8d8bd14%40googlegroups.com.

[Margaux Silvestre]

It is only the SelectMay that stops. 

Oh I was maybe not clear. As soon as I press the Key, I would like to do the sub-sequence one after another without stopping. I am sorry for the confusion. 

[Gonçalo Lopes]

Ah, that explains it! I thought you wanted to trigger every sub-sequence with a KeyDown.

That turned out to be slightly more involved, but I have the feeling there must be an easier way that I'm just not seeing right now.

Basically I decoupled the two problems: on one hand you have the video source; on the other you have the sequence of triggers and slices that run one after the other, the next slice starting when the previous one stopped.

I solved this by concatenating a series of windows defined by the number of lines in the CSV file.

I can explain it better later, but let me know first if this solves what you wanted.

On 20 May 2017 at 21:23, Margaux Silvestre <silvestr...@gmail.com> wrote:

It is only the SelectMay that stops. 

Oh I was maybe not clear. As soon as I press the Key, I would like to do the sub-sequence one after another without stopping. I am sorry for the confusion. 

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/3e605954-ed83-4fbd-ba2d-118d20cd22c3%40googlegroups.com.

[Margaux Silvestre]

What you explained is exactly what I would like but when I played the file you wrote, the workflow it-self is stopping before the end of the the first sub-sequence ... 

We can discuss that on Monday if you want to enjoy your week-end. 

Thanks!

[Gonçalo Lopes]

Hmmm, I just downloaded and tried it again and it seems to work for me. Could it be because of the video or the CSV?

How are you determining that the video stops before the end of the first sub-sequence? I'm asking because when I run it here it's hard to see anything given that the sequences just flow one into the other without ever stopping, i.e. to the eye it just looks like one big sequence, but in fact they are running correctly. The way I found to determine this was by adding all the times from the CSV together and measuring the total time of processing and verifying that they are the same.

Maybe the reason you say this could be due to the behavior of the CsvWriter inside the Nested? It is currently set to Overwrite the final data file, which is probably not what you want, right? If you want a different file for each "run" you should instead set the Suffix to FileCount or Timestamp. Alternatively, if you want everything to concatenate in one big file, you should set Append to True.

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/e08eab37-dd45-4112-a569-93ae310c38eb%40googlegroups.com.

[Gonçalo Lopes]

Also I realized that CsvWriter is not happy saving the entire tuple from Zip because there is an image in there. If you only want to save the tracking information, you should set the Selector property to Item1.

[Margaux Silvestre]

Sorry for the delay, I didn't see your e-mails. 

I joined you the CSV file I used and the video is a .avi file, so it should fine. 

I determined that the video stops before by looking at the RelativeTime node and it is stopping after 20 seconds (which is the first line of my CSV file).

I also realised that the CSVWriter is empty ... I would like to save the time, the tracking (centroid coordinates and the false and true of the detection) and the coordinates of my ROI Activity in a single big file or several ones (the solution which is the easiest). What should I do then? I didn't get the set the Selector property to Item1? 

Best,

Margaux

[Gonçalo Lopes]

The attached file seems to work for me with the "Times.csv" file that you provided. I modified CsvWriter to avoid overwriting the same file and instead create different files for each "trial". I also removed the image itself from the group of items to save, which makes CsvWriter happy without having to specify a selector.

Can you test whether this works for you?

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/479bb221-2e50-4c08-b42e-5e95722c0ffc%40googlegroups.com.

[Margaux Silvestre]

It's perfect and exactly what I wanted. Thank you so much!

I have a very last demand. I would like to save a specific time when I am pressing a key (different from the first one of course). I was thinking about doing as on the picture below but it's not working as I have a RelativeTime node. How could I link the Keydown to the RelativeTime?



Thank you very much again,
Best,

Margaux

[Margaux Silvestre]

I found a way. I used SkipUntil and TakeUntil nodes linked to KeyDown and I saved it in a CSV file.

This one is the last question I'll ask. Is it possible to save all the video? I tried to do it by inserting a VideoWritter before the Grayscale node but it's saving only the first sequence (15s).

Best,

Margaux

[goncal...@gmail.com]

That is strange. Can u send me your current workflow?

--

You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/ef0a0c60-9e66-424e-91bd-2d59bbf0c6cc%40googlegroups.com.

[Margaux Silvestre]

Today, the VideoWritter is working for a mysterious reason. I joined you the workflow to be sure but now it should be fine.

Thank you for all your help. The workflow is perfect now.

Best,

Margaux