Random sampling from a condition file (select 50 out of 1000 conditions)

[Kam Kwok]

Hi all,
My experiment requires me to randomly select 50 conditions out of a 1000 conditions from a condition file. How do I do this with the Builder View?
Thanks for your help.
Kam

[Michael MacAskill]

Hi Kam,

This should be straightforward. See example 2 at <http://www.psychopy.org/recipes/builderTerminateLoops.html>

In essence, you create a loop pointed to your 1000-line file, with random selection. But you also add a code component which terminates that loop after 50 trials.

Regards,

Michael

[T.O.]

Hi Michael,

I am a new PsychoPy user and am using v1.81.03 (the latest version).

I am having the same problem (or issue) as Kam so I followed your suggestion.

However, even though I put the suggested codes to Begin Experiment and Begin Routine tabs, the loop still plays all the possibilities in the condition file.

I am trying to program a rapid auditory stream experiment.

I put all the .wav files (letters) in one folder and made the condition file like below.

I made the loop like below.

Lastly, the code was made like below (I noticed that the word "True" is not colored in green as it should be).

I set myCount = 0 in the Begin Experiment section as well.

With the above setting, PsychoPy still plays everything in the condition file list.

How could I make it pick just one letter out of 26 letters?

Thank you very much for your time.

Sincerely,

Takashi

2013年7月31日水曜日 6時03分23秒 UTC+8 Michael MacAskill:

[Michael MacAskill]

Dear Takashi,

Thanks for an excellently documented question, which makes it easy to give a precise answer.

To fix your immediate problem, you could change this line in your code component:

sound_1.finished = True

to:

trials.finished = True

i.e. you aren’t wanting to terminate a sound playing (which has presumably already happened). It is the *loop* which you want to terminate (which in your case is called “trials”), so that the next sound isn’t played.

However, since the messages below were posted there is now a much easier way of implementing what you want. In the loop dialog, see how there is now a field called “selectedRows”? You can just specify a value in there to run just one row from your file. That way you don’t need any code component at all.

e.g. to get a single random row between 0 and 19, simply enter this in the selectedRows field:

randint(20) # random integer between 0 and 19

(NB Python counts from 0 so the first row in your csv file is row 0).

Hope that helps,

Michael

> On 24/12/2014, at 21:29, T.O. <takash...@gmail.com> wrote:
>
> Hi Michael,
>
> I am a new PsychoPy user and am using v1.81.03 (the latest version).
> I am having the same problem (or issue) as Kam so I followed your suggestion.
> However, even though I put the suggested codes to Begin Experiment and Begin Routine tabs, the loop still plays all the possibilities in the condition file.
>
> I am trying to program a rapid auditory stream experiment.
> I put all the .wav files (letters) in one folder and made the condition file like below.
>
>
>
>

> I made the loop like below.
>
>
>

> Lastly, the code was made like below (I noticed that the word "True" is not colored in green as it should be).
>
>
>

> I set myCount = 0 in the Begin Experiment section as well.
>
> With the above setting, PsychoPy still plays everything in the condition file list.
> How could I make it pick just one letter out of 26 letters?
>
> Thank you very much for your time.
>
> Sincerely,
>
> Takashi
>
> 2013年7月31日水曜日 6時03分23秒 UTC+8 Michael MacAskill:
> Hi Kam,
>
> This should be straightforward. See example 2 at <http://www.psychopy.org/recipes/builderTerminateLoops.html>
>
> In essence, you create a loop pointed to your 1000-line file, with random selection. But you also add a code component which terminates that loop after 50 trials.
>
> Regards,
>
> Michael
>
>
> On 31 Jul, 2013, at 09:26, Kam Kwok <kwok...@gmail.com> wrote:
>
> > Hi all,
> > My experiment requires me to randomly select 50 conditions out of a 1000 conditions from a condition file. How do I do this with the Builder View?
> > Thanks for your help.
> > Kam
>
>

> --
> You received this message because you are subscribed to the Google Groups "psychopy-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-user...@googlegroups.com.
> To post to this group, send email to psychop...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/053df54b-2b50-430c-92b6-d90318d167fc%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Michael R. MacAskill, PhD 66 Stewart St
Research Director, Christchurch 8011
New Zealand Brain Research Institute NEW ZEALAND

Senior Research Fellow, michael....@nzbri.org
Te Whare Wānanga o Otāgo, Otautahi Ph: +64 3 3786 072
University of Otago, Christchurch http://www.nzbri.org/macaskill

[T.O.]

Hi Michael,

Thank you so much for such a useful advice and very encouraging feedback about the documentation.

I followed your suggestion to change the code in a below fashion and it solved the problem!

sound_1.finished = True

to:

trials.finished = True

Furthermore, I really like the new option of "selectedRows" and I tried your below suggestion.

  

e.g. to get a single random row between 0 and 19, simply enter this in the selectedRows field: 
randint(20) # random integer between 0 and 19

Unfortunately, I am encountering a new issue here when I implement this method.

I have such a condition file to create my loop (it is identical as the previous document).

In addition, I have created such a loop.

However, when I run the experiment, I get the below error message.

I'm sorry about the bad resolution but the last row says "TypeError: 'None Type' objects is not iterable".

Just to make sure, I have also tried running "randint(20)" to see whether 25 was exceeding the number of rows in the condition file or "randint(1,20)" to see whether assigning lower and higher limits (to exclude the first row which is just a name of a variable) will solve the issue but they returned the same error messages.

My understanding about the "Selected rows" function is that we can specify the range of rows (by numbers) which will be used as stimuli in a loop.

Moreover, I read that randint() function commands PsychoPy to pick up any integers randomly.

Therefore, in my case, I thought that typing "randint(25)" in the "Selected rows" section would command PsychoPy to randomly pickup any single row (or cell) from the "Letters" column (ranging from the 1st to 26th rows) per stimulus.

But unfortunately, I can't find out the way to fix this problem.

I would really appreciate it if you could point out where I made a mistake.

Thank you so much for your time.

Sincerely,

Takashi

[Michael MacAskill]

Hi Takashi,

> On 28/12/2014, at 21:07, T.O. <takash...@gmail.com> wrote:
> I followed your suggestion to change the code in a below fashion and it solved the problem!

Glad that worked, even though the other solution should be preferable.

> Furthermore, I really like the new option of "selectedRows" and I tried your below suggestion.

> However, when I run the experiment, I get the below error message.

> I'm sorry about the bad resolution but the last row says "TypeError: 'None Type' objects is not iterable”.

Sorry this is quite a new feature and I haven’t yet gotten used to how to specify the expressions in this field. PsychoPy can be a bit fussy about how they are entered. Try this:

$[randint(25)]

We need to use a $ prefix to indicate that we are entering some code rather than a literal value, and we also need to use square brackets so that the resulting integer gets packaged within a list, even though it is only a single number.

Best wishes,

Michael

[T.O.]

Hi Michael,

Thank you very much for such a quick reply!

I tried the suggested method to solve my problem (cited below).

$[randint(25)]

This worked perfectly and my problem is solved very nicely!

I deeply appreciate your kindness and support!

Sincerely,

Takashi 

[T.O.]

Hi Michael,

I'm so sorry for bothering you again!

I have a further question about the "Selected rows" function.

As stated earlier, the $[randint()] function worked perfectly for choosing specific single row (i.e. stimuli) from condition file with multiple conditions.

However, when I ask the loop to repeat the randomly picked stimuli for "multiple" times (e.g. nReps = 3 in my case shown below), it keeps playing the same stimuli for multiple times until the loop finishes.

This is the properties of my sound stimulus.

"$Letters" signify the name of the column in my condition file (shown in our earlier correspondence).

(By the way, I initially thought that "set every repeat" function would "renew" the stimuli every time the loop is repeated but I was wondering why it is not the case this time.)

Just to see what happens, I have also inserted ISI before the presentation (please see below) of the stimuli (sound_1) and used the "set during ISI" function but it only gave me error message.

Getting back to the main topic, below is the properties of my loop.

"Letters.xlsx" is the name of my condition file.

Is there any function to ask PsychoPy to pick different stimuli each time the loop is repeated?

I would deeply appreciate it if you could teach me whether there is such a function or not.

Thank you so much for your patience and continuous support!

Sincerely,

Takashi

2014年12月28日日曜日 18時07分11秒 UTC+8 T.O.:

[Michael MacAskill]

Dear Takashi, 

As stated earlier, the $[randint()] function worked perfectly for choosing specific single row (i.e. stimuli) from condition file with multiple conditions.

However, when I ask the loop to repeat the randomly picked stimuli for "multiple" times (e.g. nReps = 3 in my case shown below), it keeps playing the same stimuli for multiple times until the loop finishes.

Yes, it isn’t intuitive, but in this case, the loop is created only once (with a single random row chosen) and then that exact same loop runs three times.

You can achieve what you want, however. 

(1) Set the nReps of your loop called PreX_loop to 1

(2) Insert a new loop around PreX_loop (so that PreX_loop is “nested” within it).

(3) Set the nReps of the outer loop to 3.

(4) This outer loop is not attached to a conditions file (it doesn’t need to access any variables).

Having a nested situation like this means that the inner loop (PreX_loop) is re-created from scratch on each iteration of the outer loop (effectively, three loops are created and run only once each). Therefore, a new randint() is chosen on each iteration.

By contrast, setting the nReps of a loop itself just results in that same loop being run multiple times (i.e. one loop running three times). 

Hopefully I’ve explained that meaningfully?

(By the way, I initially thought that "set every repeat" function would "renew" the stimuli every time the loop is repeated but I was wondering why it is not the case this time.)

The “set every repeat” option does renew the stimulus value on each iteration. But your loop is only accessing one row of the file, so the effect of it is not obvious in this particular situation because there is only a single iteration. If it ran through all 26 rows, then you would certainly hear a different sound being played on each trial.  

Best wishes,

Michael

[Michael MacAskill]

> Having a nested situation like this means that the inner loop (PreX_loop) is re-created from scratch on each iteration of the outer loop (effectively, three loops are created and run only once each). Therefore, a new randint() is chosen on each iteration.

NB in this case, it is still quite possible that duplicate rows could be selected across three trials (e.g. rows 6, 23, 6) because each random selection is independent (i.e it is sampling with replacement).

If this is a problem, then you need a slightly different approach so that you can keep track of what rows have already been selected. e.g. insert a code component, and in the “Begin Experiment” tab, put this:

randomRowList = range(25) # create a list of the numbers 0 to 25
shuffle(randomRowList) # put them into a random order

Then in your loop dialog, replace the $[randint(25)] expression with this:

$[randomRowList.pop()]

The pop() function of a list removes and returns the next item of a list (being randomly shuffled, you will get a random number even though we are proceeding in order through the list).

You will always get a unique row number because previous ones disappear as the list shrinks by popping entries off the end (i.e. you are now sampling without replacement).

Cheers,

Michael

[T.O.]

Hi Michael,

Thank you so much for your thorough advice.

Both suggestions you have given me worked perfectly!

Again, I deeply appreciate your sincere support!

Sincerely,

Takashi

2015年1月1日木曜日 9時41分45秒 UTC+8 Michael MacAskill:

[T.O.]

Hi Michael,

I really appreciate your time spent on my recurring questions.

I have devoted on programming my rapid auditory presentation (RAP) experiment incorporating your suggestions.

The program is running and everything looks great!

However, unfortunately, there is one big issue with the presentation of the auditory stimuli.

Each stimulus is presented quite fast (0.09ms with ISI of 0.01ms).

One trial consists of 30 items and each items (i.e. recorded letters read aloud by a person) are drawn randomly from the list of txt files (e.g. folderName\A.wav).

When I play the stream, it has periodic "breaks" (i.e. silent lapses about 0.1 ms) and it sounds like the stream is stopping every 5 or 6 items.

I doubted whether this could be caused by overloading the PC since I have so many constraints added to the randomization (i.e. I wrote a code to create csv condition file specifying the order of items in the RAP trial as well as the order of RAP trials themselves within the whole experiment).

However, this seems not to be the case since when I created a very simple RAP stream for testing, it still had this periodic lapses.

I have also tried running the RAP using longer stimulus presentation time (e.g. 0.3 ms) but the stream still does not seem to be smooth.

(Relating to this note, I also found that when the actual wav file has shorter presentation time compared to the assigned presentation time (e.g. wav file length 0.2 ms vs. stimulus presentation time 0.3 ms), the routine keeps repeating the sound like B,B,B,B, instead of just playing plain B. This phenomenon is inconsistent with the PsychoPy manual's description since it says that the sound should stop when the wav file happens to be shorter than the assigned presentation time. I am not sure whether this is a bug of the latest PsychoPy version and is somewhat indirectly causing the above mentioned issue).

I would very much appreciate it if you could share with me your assessment whether this problem could be caused by the some bug or some inevitable issue when programming RAP experiments (e.g. PsychoPy jams when too many wav files are loaded at once).

Thank you so much for your time!

Sincerely,

Takashi

2015年1月1日木曜日 9時41分45秒 UTC+8 Michael MacAskill:

[Michael MacAskill]

Hi Takashi,

This is getting out of my area of expertise now, as I don’t know much about the details of sound stimuli.

But would you mind checking through your message and re-posting? Your stimuli details seem unfeasibly brief. Did you mean to say seconds instead of milliseconds throughout?

Regards,

Michael

On 15/01/2015, at 20:38, T.O. <takash...@gmail.com> wrote:

Hi Michael,

I really appreciate your time spent on my recurring questions.

I have devoted on programming my rapid auditory presentation (RAP) experiment incorporating your suggestions.

The program is running and everything looks great!

However, unfortunately, there is one big issue with the presentation of the auditory stimuli.

Each stimulus is presented quite fast (0.09ms with ISI of 0.01ms).

One trial consists of 30 items and each items (i.e. recorded letters read aloud by a person) are drawn randomly from the list of txt files (e.g. folderName\A.wav).

When I play the stream, it has periodic "breaks" (i.e. silent lapses about 0.1 ms) and it sounds like the stream is stopping every 5 or 6 items.

I doubted whether this could be caused by overloading the PC since I have so many constraints added to the randomization (i.e. I wrote a code to create csv condition file specifying the order of items in the RAP trial as well as the order of RAP trials themselves within the whole experiment).

However, this seems not to be the case since when I created a very simple RAP stream for testing, it still had this periodic lapses.

I have also tried running the RAP using longer stimulus presentation time (e.g. 0.3 ms) but the stream still does not seem to be smooth.

(Relating to this note, I also found that when the actual wav file has shorter presentation time compared to the assigned presentation time (e.g. wav file length 0.2 ms vs. stimulus presentation time 0.3 ms), the routine keeps repeating the sound like B,B,B,B, instead of just playing plain B. This phenomenon is inconsistent with the PsychoPy manual's description since it says that the sound should stop when the wav file happens to be shorter than the assigned presentation time. I am not sure whether this is a bug of the latest PsychoPy version and is somewhat indirectly causing the above mentioned issue).

I would very much appreciate it if you could share with me your assessment whether this problem could be caused by the some bug or some inevitable issue when programming RAP experiments (e.g. PsychoPy jams when too many wav files are loaded at once).

Thank you so much for your time!

Sincerely,

Takashi

[T.O.]

Hi Michael,

Thank you so much for your prompt reply.

I'm sorry I made a mistake.

It should be 90 ms and 10 ms (for the presentation stimulus time and ISI respectively) or 0.09 sec and 0.01 sec just as you have mentioned.

Yes, I will re-post the issue and hopefully, the solution could be found.

Anyways, I really appreciate your time reading the long questions and answering them!

Sincerely,

Takashi

2015年1月15日木曜日 17時49分37秒 UTC+8 Michael MacAskill:

[Michael MacAskill]

Dear Takashi,

OK, that makes more sense.

Are you running your screen at 100 Hz? Builder operates on a cycle which is slaved to the refresh rate of your screen. If you are using a standard LCD, for example, running at 60 Hz, you might see such timing glitches, because PsychoPy’s timing (running on 16.7 ms intervals) won’t be able to keep in sync with your intended stimulus intervals.

Just a thought.

Regards,

Michael

On 16/01/2015, at 01:31, T.O. <takash...@gmail.com> wrote:

Hi Michael,

[T.O.]

Hi Michael,

Thank you so much for giving me your idea again.

Yes, I'm using 60 Hz monitor.

As you have speculated, this could be causing the problem I mentioned.

Since there are bunch of PCs with better refresh rate at my lab, I will try my code out in those PCs and see what happens.

I deeply appreciate your kindness :)

Sincerely,

Takashi

2015年1月16日金曜日 13時25分07秒 UTC+8 Michael MacAskill: