Does counterbalancing work well on MTurk

16 views
Skip to first unread message

Jun Lyu

unread,
Jul 19, 2021, 4:12:42 AMJul 19
to ibexexperiments
Hi,

I wonder, when using MTurk, do people create 4 lists on Ibex for MTurk (suppose one has 4 conditions)? Ibex uses counterbalance, so will we have a highly unbalanced distribution of lists when using MTurk (suppose some people click on the link at the same time)? Or, since Ibex counterbalances hits, it will make adjustment when too many people have done one or two particular lists?

Best regards,
Jun

Alexandre Cremers

unread,
Jul 19, 2021, 4:40:51 AMJul 19
to ibexexp...@googlegroups.com
The easiest way is to create a variable for the URL in your MTurk task and fill it with different URLs differing only by the xxx value you mentioned in your previous email.

You MTurk task would include a line like this:

Please click <a href="$URL">this link</a>

And then you provide mturk with a single column csv file that looks like this:
URL
.../ExperimentName/server.py?withsquare=1
.../ExperimentName/server.py?withsquare=2
.../ExperimentName/server.py?withsquare=3
...

You need to break down your MTurk task into multiple HITs anyway to keep the commission at 20%, so this is a convenient way to address two problems at once. Make sure that participants don't complete different versions of the survey however.

A
 

--
You received this message because you are subscribed to the Google Groups "ibexexperiments" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ibexexperimen...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/664ca4d1-c54a-47a3-8eba-225b439e962cn%40googlegroups.com.

Jun Lyu

unread,
Jul 19, 2021, 5:45:04 AMJul 19
to ibexexperiments
Thank you Alex! This is very helpful! 

So, about the withquare=i part, can i increment from 1 to N (where N stands for the number of participants)? Or is i capped by the number of counterbalanced lists?

Jun

Alexandre Cremers

unread,
Jul 19, 2021, 5:48:00 AMJul 19
to ibexexp...@googlegroups.com
You can let N go above the number of lists, Ibex manages it internally.
A

Reply all
Reply to author
Forward
0 new messages