Does counterbalancing work well on MTurk

[Jun Lyu]

Hi,

I wonder, when using MTurk, do people create 4 lists on Ibex for MTurk (suppose one has 4 conditions)? Ibex uses counterbalance, so will we have a highly unbalanced distribution of lists when using MTurk (suppose some people click on the link at the same time)? Or, since Ibex counterbalances hits, it will make adjustment when too many people have done one or two particular lists?

Best regards,

Jun

[Alexandre Cremers]

The easiest way is to create a variable for the URL in your MTurk task and fill it with different URLs differing only by the xxx value you mentioned in your previous email.

You MTurk task would include a line like this:

Please click <a href="$URL">this link</a>

And then you provide mturk with a single column csv file that looks like this:

URL
.../ExperimentName/server.py?withsquare=1
.../ExperimentName/server.py?withsquare=2
.../ExperimentName/server.py?withsquare=3
...

You need to break down your MTurk task into multiple HITs anyway to keep the commission at 20%, so this is a convenient way to address two problems at once. Make sure that participants don't complete different versions of the survey however.

A

 

--
You received this message because you are subscribed to the Google Groups "ibexexperiments" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ibexexperimen...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/664ca4d1-c54a-47a3-8eba-225b439e962cn%40googlegroups.com.

[Jun Lyu]

Thank you Alex! This is very helpful! 

So, about the withquare=i part, can i increment from 1 to N (where N stands for the number of participants)? Or is i capped by the number of counterbalanced lists?

Jun

[Alexandre Cremers]

You can let N go above the number of lists, Ibex manages it internally.

A

To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/f8be5e4c-c811-4305-a6c2-cc90888db2ffn%40googlegroups.com.