Playing one of two sounds randomly only once when a ROI is activated

[lebert...@gmail.com]

Hi,

I am trying to make bonsai play randomly one of two sounds when my ROI is activated. Moreover, I cannot find how to play a sound only once. The ON/OFF solution makes the sound play constantly as long as the ROI is activated. As my sounds are sweep sounds, it only plays the first frequency of the sweep indefinitely as long as the ROI is activated, and I would need bonsai to play the full sound only once when the ROI is activated. Does anybody knows how I could do that?

Thanks in advance!

Jules

[lebert...@gmail.com]

Hi again.

I am a beginner in programming and I am struggling with this simple bonsai workflow. I managed to create a workflow with a random inside it. But as the workflow seems to be constantly read it only jumps from the beginning of one sound to the other. How could I generate my sweep sound only once and then go to the rest of the workflow? The rest of the workflow would be that, depending on which branch of the workflow the random chose to go, one other ROI would give a reward and restart the workflow, and the other just play an error sound and also restart the workflow.

Basically, it is a 2AFC task where I learn a rat to go either on the right or the left of my track depending of the sound played. And if he makes the good choice, bonsai will send a message to an Arduino which will give him a reward.

I join my workflow if it can help.

Thanks!

[Gonçalo Lopes]

Hi Jules,

Sorry for the delayed reply. I am attaching an example that will hopefully be of use to you:

Basically, the idea is that you need to store each of your WAV audio data in a variable (e.g. a BehaviorSubject).

Once the data is stored, you can replay it using the AudioPlayback node. The trick is how to control when the replay is done.

In this case, I am using a SelectMany node. This node can be used to trigger stuff on-demand. Inside each SelectMany, I simply have a SubscribeSubject that reads from one of the stored audio buffers and sends it out to the AudioPlayback node. This way you can control exactly when you want to play each sound, and it will play it only once to the end. Here I am using a different KeyDown event to control the different sounds, but you can use other things.

Make sure that your event source fires only once at the beginning of the event, and that it doesn't keep firing. For example, if you want ROI entry to activate the sound, you can't have the event firing for every frame the ROI is activated, but only the first time there is an entry. If you have a Boolean source, for example from the PythonTransform, you can use DistinctUntilChanged to make sure events are only fired when the value changes.

Play around with the attached workflow a bit and hopefully it will make things more clear.

Hope this helps!

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/e1173f7b-9967-4004-b5e1-437ed1cf2b61%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[lebert...@gmail.com]

Hi Goncalo,

Thank you for your answer!

I have been trying to make it work but it i am struggling with two problems.

In my workflow, the activities of three ROIs are stored in three different PublishSubject nodes.First, I track the activity of the centre ROI. If the centre ROI detects activity, the program goes either on the branch tracking the left ROI or the right ROI with a 50% chance for each, and plays the corresponding sound.

Here occurs the first problem: the sound is either not played for the right branch, or played very weirdly for the left branch, but the corresponding AudioReplay node shows activity. I tried to test a simpler workflow to see if there were a problem with how I am coding the AudioReplay (see below) but the sound is perfectly played in this case:

The second problem occurs with the Repeat node. After the activation of the Central ROI, the activity of one of the other ROIs is tracked. Once this ROI is triggered, Bonsai sends a message to Arduino to give the animal a reward and activates the Repeat node. The Repeat node should theoretically allows to repeat the workflow, which means restart the tracking of the central ROI and generate another random value. But, even if it seems that Repeat is activated, activating the centre ROI  does nothing after that.

Could you help me with these issues?

I have attached my workflow to the message.

Many thanks,

Jules

Le lundi 5 février 2018 20:56:58 UTC, goncaloclopes a écrit :

Hi Jules,

Sorry for the delayed reply. I am attaching an example that will hopefully be of use to you:

Basically, the idea is that you need to store each of your WAV audio data in a variable (e.g. a BehaviorSubject).

Once the data is stored, you can replay it using the AudioPlayback node. The trick is how to control when the replay is done.

In this case, I am using a SelectMany node. This node can be used to trigger stuff on-demand. Inside each SelectMany, I simply have a SubscribeSubject that reads from one of the stored audio buffers and sends it out to the AudioPlayback node. This way you can control exactly when you want to play each sound, and it will play it only once to the end. Here I am using a different KeyDown event to control the different sounds, but you can use other things.

Make sure that your event source fires only once at the beginning of the event, and that it doesn't keep firing. For example, if you want ROI entry to activate the sound, you can't have the event firing for every frame the ROI is activated, but only the first time there is an entry. If you have a Boolean source, for example from the PythonTransform, you can use DistinctUntilChanged to make sure events are only fired when the value changes.

Play around with the attached workflow a bit and hopefully it will make things more clear.

Hope this helps!

On 5 February 2018 at 14:58, <lebert...@gmail.com> wrote:

Hi again.

I am a beginner in programming and I am struggling with this simple bonsai workflow. I managed to create a workflow with a random inside it. But as the workflow seems to be constantly read it only jumps from the beginning of one sound to the other. How could I generate my sweep sound only once and then go to the rest of the workflow? The rest of the workflow would be that, depending on which branch of the workflow the random chose to go, one other ROI would give a reward and restart the workflow, and the other just play an error sound and also restart the workflow.

Basically, it is a 2AFC task where I learn a rat to go either on the right or the left of my track depending of the sound played. And if he makes the good choice, bonsai will send a message to an Arduino which will give him a reward.

I join my workflow if it can help.

Thanks!

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Gonçalo Lopes]

Hi Jules,

I need a bit more time to go over this carefully, but two quick thoughts:

1) Before I suggested using a BehaviorSubject for sound, but it may be better to use a ReplaySubject to ensure that all the audio buffers are definitely stored and played back. Another way is to make sure that BufferLength matches the exact number of samples in your WAV file. Otherwise, sounds that are played back may sound weird if samples are missing.

2) The Repeat operator only repeats execution of the current branch, it says nothing about what is going on in other branches. Specifically, Repeat does not interrupt any branch that may be going on in parallel. If you want to do this, you need to merge all the branches into a single stream (using Merge, Amb, or other combinators) and then place Repeat at the end of that. The idea is to make sure your trial completes entirely before you call Repeat.

Hope this helps for now. I will take a closer look later.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/a2b08555-f96e-4e61-87a3-567c039ada47%40googlegroups.com.

[lebert...@gmail.com]

Hi Goncalo,

Thanks for replying. I am trying to understand how the combinators work, but I am not sure about how to use them here. Should I also merge the branches with AudioPlayback with the rest?

[Gonçalo Lopes]

Hi Jules,

Yes, that would be the case. If you have issues because of type compatibility, you can use the Unit operator before Merge so that all inputs have the same type.

Another way to do it is to use the Sink operator to play the sounds, instead of creating a dedicated branch (i.e. all the nodes in the branch go inside the Sink). That will allow you to create a side-effect (playing the sound) without changing the main processing data stream.

Hope this helps.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/16412868-6a6f-4b7d-bc8c-98419ad7d1e0%40googlegroups.com.

[lebert...@gmail.com]

Hi Goncalo,

I used the combinator nodes and it seems to work better. I can now alternate between 2 ROIs and repeat the workflow. But the problem occurs when playing the sound.

If I use a ReplaySubject to store the sound, AudioReader seems to play only the very first part of the sound instead of the entire sound (even with my buffer set at 800ms, exactly the length of the sound) for one of my sounds, the SoundB. The other AudioReader (SoundA), even if showing activity when the random chose to go on its branch, does not play anything.

I also tried to use again BehaviourSubject instead of ReplaySubject. In this case, the SoundB is effectively played, but not the SoundA. Furthermore, with BehaviourSubject, the workflow does not seem to go back to the start after the Repeat node as it does with ReplaySubject.

I tried to play the sounds in a simpler workflow, with only one ROI triggering the sound, and the sounds are played perfectly in this case.

I wonder why this problem occurs when I use a more complex workflow.

Do you have any idea of why I cannot play properly the sound in this workflow?

Many thanks,

Jules

[lebert...@gmail.com]

I don't know how to edit my previous message. I forgot to mention that I have try to use Sink operator, but no sound is played with this solution. 

[Gonçalo Lopes]

Hi Jules,

I believe the problem is the two different AudioPlayback nodes which are being activated and de-activated every trial. Some sound cards don't like the quick connect and disconnect. I am attaching a modified workflow where I use a single AudioPlayback node to bottleneck all the sounds into the same sink. This way there is a single connection to the sound card and hopefully it should be enough to play all the sounds successfully.

To do this, I've created an extra variable "ActiveSound", which represents the shared audio channel which will be used for every trial. This way we should be able to get rid of any initialization issues. Let me know how this works.

By the way, you seem to be using some nifty USB DACs to output high-frequency signals. Would you mind sharing the references to the hardware? It sounds very useful.

Hope this helps.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/c344ebbc-0958-4eec-870c-ec0e7ccd9cc3%40googlegroups.com.

[lebert...@gmail.com]

I am currently using a M-Audio Super DAC II with a DAP-Audio Palladium 500 Vintage Amplifier.

The workflow is working. Thanks a lot for everything!

[lebert...@gmail.com]

Hi Goncalo,

The workflow is working perfectly, but I would like to introduce sounds only played by the right speaker or the left speaker. To do that, I have generated with matlab stereo wave file with one track empty and the other with the sound. The files are perfectly read with a classic player (Only one of the two speakers is playing a sound). But when I try to make bonsai play the files, it doesn't play any sound, even if AudioPlayer shows activity. I have also tried with stereo wave file with the same track on left and right, and again no sound is played. 

Can bonsai play stereo files? And if not, how could I generate a sound in only one speaker with my workflow?

Thanks!

Jules

[Gonçalo Lopes]

Hi jules,

That is strange, Bonsai can most definitely play stereo WAV files. Can you attach a short sample of the WAV file you are generating with MATLAB that we could test?

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/a43adbbc-29b3-4bfb-ac71-1f0dc98f8958%40googlegroups.com.

[lebert...@gmail.com]

Please find attached one of the stereo files I am trying to play with bonsai. It has a duration of 800ms and I set the frequency of the AudioPlayback in bonsai at 192000 to play correctly the mono version of the file.

[lebert...@gmail.com]

Hi Goncalo,

I tried other stereo .wav files, with something in the right track and in the left track, but it is also not working in bonsai. Is there a solution for that? Or should  find another idea to play a sound only in one of the two speakers during the task?

Thanks,

Jules

[Gonçalo Lopes]

Hi Jules,

Sorry for the delayed response. I just tried playing the file on Bonsai in my computer, and it seems to work fine, and the output is indeed a 2-channel stereo waveform. The playback sounds identical to what I get from VLC media player (a kind of decaying bird whistle sound).

However, when I tried your workflow, it also didn't work for me. Working through it I realized the source of the problem: the FunctionGenerator node that is used to initialize the ActiveSound keeps spitting out waveforms. You need to stop them by inserting either a Take(1) or an IgnoreElements, like so:

After this, the sounds should play fine. I've attached the modified workflow as well.

Hope this helps!

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/2c11d825-260c-4938-af22-d478d7ba578a%40googlegroups.com.

[lebert...@gmail.com]

Hi Goncalo,

Thank you, the stereo sound are played correctly now. But another problem occurs. With the solution of Take(1) or IgnoreElements, each time bonsai plays a sound, it also plays all the other sounds played before. In the latest version of my workflow I use 2 other sounds, but the architecture is similar to the workflow I've send previously. If we call the sounds used A, B, C and D this is what happens:

If A is triggered first, bonsai will play the sound A.

If B is triggered afterwards, bonsai will play A then B (and not only B).

And then, if C is triggered, I will hear A, then B, then C, instead of only C.

Etc...

I am not sure but I think each time the workflow stores a sound in ActiveSound with the MulticastSubject node, it adds it in the subject instead of replacing the previous sound stored. I don't know why adding Take(1) or IgnoreElements causes this. I've tried few things to correct it but it is not working, I am not sure of what I am doing.

Do you have any idea on why this occurs and how avoid it?

Thanks again for taking time to answer, you are really doing an amazing job with bonsai, and with this forum!

Jules

[Gonçalo Lopes]

Hi Jules,

Your intuition is correct, the buffers are being added to a queue for playback, which is why you hear them in sequence as you describe.

If you want to be able to interrupt the playback of cached sounds, you actually need to get rid of the shared AudioPlayback since there is no way to cancel the playback of already queued buffers unless you interrupt the entire playback node itself.

Instead, you need to create independent playback nodes that can be destroyed on a playback by playback basis. To do this, you can replace the Play Sound A/B (SelectMany) nodes with a WindowWorkflow operator that creates the individual playbacks, like so:

The Switch operator makes sure that only one of the workflows is playing at a time. Every time a new playback workflow is activated, any currently playing workflow is interrupted and replaced by the latest one.

I am attaching also a modified version of the task to achieve this.

Hope it helps!

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/b6c1bca6-0688-4d5a-817b-163dcd1e4996%40googlegroups.com.

[Gonçalo Lopes]

Also, please bear in mind that interrupting sounds this way may not actually be what you want, since the sudden setting of an audio source to zero will generate very audible "clipping". In this case you would have to ramp the sound volume to zero before interrupting it, but this will get slightly more involved.

[Christopher Machle]

Hello Gonçalo,

I am experiencing difficulty in playing the same randomly selected tone at two different stages of an ROI activation sequence. Attached is the toy example I created to discern how best to implement this audio cuing for my task. However, before explaining my problem/confusion, it might be helpful for me to take a step back and explain what my task is.

I am adapting the “Restaurant Row” paradigm outlined in Sweiss et al 2018, as pictured below.

 

In my version of the task, mice will run in either clockwise or counterclockwise circles, sampling the food pellet “offers”  they receive from each restaurant over the course of a 1 hour trial (during which they will collect all of their food for a given day). If a mouse enters the T-shaped junction of a restaurant from the correct direction, an audio cue corresponding to one of four reward probabilities (1, 0.8, 0.2, or 0) will be played until a decision is made. If the mouse rejects the offer and turns left toward the next restaurant (as depicted above) , the cue silences immediately once the offer zone is exited. Alternatively, if a mouse accepts the offer, the same tone will play for a fixed wait time of 5 seconds, which constitutes the cost of an offer. If a mouse enters the offer zone but quits before the full wait time has elapsed, the tone will be silenced immediately.

Though my current workflow (which I am happy to attach if it would be helpful) accurately tracks all of the directionally sensitive ROI activation, I have not been able to fully simulate this cue contingency. In my toy example below, everything seems to work exactly as I need up until the Audio Reader node. Even though the input mapping node sends the relevant string value for the appropriate file to the Audio Reader, the first entrance of the ROI makes no noise (and the Audio Reader appears to be receiving no input at all despite getting an input from input mapping). Additionally, if the ROI that is simulating the offer zone is activated repeatedly without activating the T-junction ROI (so that no tone is generated), the first proper ROI activation sequence will generate the same problem, where the appropriate string gets processed through the input mapping node but not by the Audio Reader node. (To be clear, the stream emanating from the video capture node in this toy example does not explicitly capture the contingencies I have in the ‘master’ workflow, but if you manually trigger the first stream then the second, you can simulate the relevant contingencies).

Any intuitions as to what is going on? I am pretty thoroughly confused. And thank you in advance!!

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Gonçalo Lopes]

Hi Christopher,

Based on your description of the task, I have attached a mockup of a sound management system based on the idea of a state machine and the new bonsai 2.4 audio management system. Modelling complicated sequential dependencies can be hard if you try to feed all the audio streams explicitly.

This workflow requires two WAV-files (Offer.wav and Paid.wav). I've simulated the relevant events as key presses, but because they are declared as named variables, you should be able to replace them with your custom event detectors based on video. Here is the full workflow (also attached):

Basically the idea is that every time you enter a restaurant you start a decision process, with a tone. That decision can result in rejection or acceptance of the offer. Each result has its own branch: the reject branch just stops the sound; the accept branch goes into the Offer state, which starts the timer and keeps the sound running while keeping an eye out for exit events. At the end, if the cost is payed, another sound is played just to signal the positive outcome. I've also included a tally of all positive outcomes to show how you can keep a running accumulation over trials.

Hope this is close to what you were after so you may be able to adapt it to your specific rig.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/516dc48c-b94b-4742-bd24-e4b790b2ee5a%40googlegroups.com.

[Christopher Machle]

Hi Gonçalo,

Thanks for getting back to me. After playing with your very sleek solution (and tweaking it a bit to match some of the nuances of the task that I did not explain well), I have a workflow that does almost everything that I am looking for as far as audio cues are concerned! Thank you so much for the help here!

The final component I have not been able to figure out is how to play a "reward delivered" tone when an accept decision is rewarded, and a "reward not delivered" tone when an accept decision is unrewarded. Because the reward/no reward signal will also be sent through the arduino board to modulate the actual deliverance of the pellet (in the full workflow), it seems like that boolean value needs to be published in the global workspace. So, I attempted to publish the boolean value of reward/no reward globally, then tried to condition what reward outcome tone is played from the "Offer" select many node through this published boolean. But, when I do that, I cannot get the subscribed value to fire when inside the select many "Offer" node. I am thinking that I probably do not fully understand how select many nodes work, and I am probably using a condition node when another node would work best. However, I am still very confused as to why the reward-associated boolean will not display its value inside the select many node.

I've attached my workflow below, and appreciate any insight you may have as to what I am doing wrong!

Thank you in advance,

Christopher

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

[Gonçalo Lopes]

Hi Christopher,

I suspect this has to do with the fact that you haven't fully stopped your Offer state. Because you are using QuitOffer, which is now a global source which is always active even outside the logic of the task, you need to use the First operator to make sure that you only want the first event to generate an output, and no others after that.

This is similar to what I used in the WaitForDecision node after each of the global accept/reject events (the First node after each variable subscribe).

Another wait to make your states bullet-proof in this respect is to place the First operator just before the WorkflowOutput node inside the SelectMany. That way, no matter what your state does inside, the first output will always shutdown the state and move on.

Hope this helps.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/4bf127a6-b7e0-4aab-a44e-aa1f1efe9e04%40googlegroups.com.

[Christopher Machle]

Hi Gonçalo,

Your above suggestion helped significantly. I ultimately got the workflow to run exactly how I wanted it to.

However, after running some mice in the rig, I realized that the structure of my workflow allows mice to collect rewards in a way that discourages some from running proper laps. Because offers are conditioned on the direction of entry and the previously entered reward zone, mice can run "L" shaped paths instead of proper laps in the workflow I've attached below. Using the diagram from a couple posts above as an example, a mouse can receive an offer from Plain, run clockwise (backwards) through Banana and Chocolate to Grape, then clockwise to through Chocolate to receive an offer from Banana (satisfying the conditions by having entered from the Banana hallway and having previously visited Chocolate) . Instead, I want to place tighter restrictions where offers are only presented in a strict counterclockwise sequence. For example, if a mouse gets an offer from Plain, then runs backwards to Grape, the next offer they can receive should only be presented at Grape when they run counterclockwise through Chocolate then Banana then Plain to Grape. As such, the offer history would always follow a strict [Banana, Plain, Grape, Chocolate] order, forcing mice to fully correct their lap path before being able to receive an offer. 

I have struggled significantly to implement this structure. The only ideas I have been able to come up with either involve defining publish subject variables in terms of themselves or writing python transform nodes with non-local variables. Both routes appear not to be supported by Bonsai.

Do you have any suggestions? Let me know if my explanation of the problem is not clear.

Thank you!

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/4bf127a6-b7e0-4aab-a44e-aa1f1efe9e04%40googlegroups.com.

[Christopher Machle]

After going back to the drawing board and reading about recursion in Bonsai, I realized that I was trying to reference nonlocal variables when in reality I needed to reference global variables within the python script. I will continue working toward a solution tracking reward history using a python transform.

[Gonçalo Lopes]

Thanks Chris, glad to hear you were able to figure out the variable management issue.

Regarding managing the tighter restrictions you mentioned, the way I would do this is to use the Permutation operator from the Numerics package. With that you could generate repeated blocks of shuffled conditions (blocks of 4), and that feed that into CreateObservable > Concat > Repeat. That way you can get repeated sequences of 4, which are shuffled. Because you are using shuffling, you know that you are not repeating trial types in any one block.

Hope this helps, let me know if more clarification would help!

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/0eb49d31-ac6a-4d3e-8986-ed6d521f53fb%40googlegroups.com.

[Christopher Machle]

Hey Gonçalo,

Thanks for the reply! And sorry I did not update my solution sooner. What I ended up doing is very similar to what you described above, but with a series of python transform nodes, global counting variables, and some modular arithmetic. From what you described, it seems like I used some simple python code to achieve the same end. 

Thanks again for all the help.

Jules

</d