What's wrong with my counterbalanced list?

[Jeremy]

Dear all,

I conducted a pilot study with 15 subjects. In my AJT, each item should receive data from 7 or 8 participants. But some of the items only received data from 2 subjects. Is there anything wrong with my counterbalanced list?

Best wishes,

Jeremy

I marked the items that only received data from 2 subjects with ?. The other items are normal (collecting data from 7 or 8 subjects). Sentences were deleted for better reading experience.

var shuffleSequence = seq("setcounter", "intro", sepWith("sep", rshuffle(startsWith("AJT"))), sepWith("sep", startsWith("q")) );

[["AJT-CLNM.A.1", 54 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.B.1",  54], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.A.2",  55], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.B.2", 55 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.A.3", 56 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.B.3", 56 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.A.4",  57], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.B.4", 57 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.A.5",  58], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.B.5", 58 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.A.6", 59 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNM.B.6", 59 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-CLNN.A.1", 60  ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLNN.B.1",  60 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.C.1", 60  ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.A.2",  61], "AcceptabilityJudgment", {s: "  }],

 [["AJT-CLNN.B.2",  61 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLNN.C.2", 61  ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLNN.A.3", 62 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.B.3", 62  ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.C.3", 62 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.A.4",  63 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLNN.B.4",  63 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.C.4", 63  ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.A.5", 64  ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.B.5", 64 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLNN.C.5",  64 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLNN.A.6",  65 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.B.6",  65 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLNN.C.6", 65 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-TOPI.1", 66 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-TOPI.2", 67 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-TOPI.3", 68 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-TOPI.4", 69 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-TOPI.5", 70 ], "AcceptabilityJudgment", {s: "  "}],  

[["AJT-TOPI.6", 71 ], "AcceptabilityJudgment", {s: "  "}],  

? [["AJT-CLBF.A.1",  72 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLBF.B.1", 72 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.C.1",  72 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.D.1",  72 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.A.2",  73 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLBF.B.2",  73 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLBF.C.2",  73 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.D.2",  73 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.A.3",  74 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.B.3",  74 ], "AcceptabilityJudgment", {s: "  "}],

? [["AJT-CLBF.C.3", 74  ], "AcceptabilityJudgment", {s: "  "}],

?[["AJT-CLBF.D.3",  74 ], "AcceptabilityJudgment", {s: "  "}],

?[["AJT-CLBF.A.4", 75  ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.B.4",  75 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.C.4", 75  ], "AcceptabilityJudgment", {s: "  "}],

?[["AJT-CLBF.D.4", 75  ], "AcceptabilityJudgment", {s: "  。"}],

?[["AJT-CLBF.A.5", 76  ], "AcceptabilityJudgment", {s: "  "}],

?[["AJT-CLBF.B.5",  76 ], "AcceptabilityJudgment", {s: "  。"}],

[["AJT-CLBF.C.5",  76 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.D.5",  76 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.A.6",  77 ], "AcceptabilityJudgment", {s: "  "}],

?[["AJT-CLBF.B.6",  77 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.C.6",  77 ], "AcceptabilityJudgment", {s: "  "}],

[["AJT-CLBF.D.6",  77 ], "AcceptabilityJudgment", {s: "  "}],

[Alexandre Cremers]

Your groups are not equal, and all the problematic items are in groups with more than 2 items. This is technically allowed, but it will most likely create problems like what you saw.

A

--
You received this message because you are subscribed to the Google Groups "ibexexperiments" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ibexexperimen...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/70b8dba6-5c7c-47ed-9b5f-080c586c44e7n%40googlegroups.com.

[Jeremy]

Thank you Alex! Then what can I do with the group? My experiment includes groups of different number of conditions. Some of them are 2, whereas others are 3 or 4. Do I need to create separete experiments?

Best wishes,

Jeremy

[Alexandre Cremers]

Surely you can't use a latin square then. You need to think more carefully about your experimental design, and we can't really help you with that.

When you know what you want to show to your participants, then we can help you program it...

To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/77fc2e69-aa54-44ab-b12c-c686f0106e21n%40googlegroups.com.

[Jeremy]

Hi Alex. I find that 'equalGroupSizes' has to be set false. I will try it.

Best wishes,

Fei

[Alexandre Cremers]

It is set to false by default. It won't solve your problem: if your groups have different sizes, you can't possibly get the same number of participants for each item.

You could ensure that each item within a group has an equal number of participants (in which case you would need a multiple of 12). In this case you need to ensure that your participants are properly distributed among your latin square cells (check past discussions on this topic).

A

To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/8346bb35-9e31-48e1-83e4-32d466f745ffn%40googlegroups.com.

[Jeremy]

Hi Alex,

I was wondering it really has something to do with the latein square design. I have another experiment with 6 lists and 8 lists. The experiment works fine. The data in my experiment is too extreme. Some of the items only get assigned to one participant (out of 15).

Best wishes,

Jeremy

[Alexandre Cremers]

On top of the unbalanced list, there's always a potential issue with too many participants getting assigned to the same list. This happens when all participants join the experiment at the same time because they're only counted when they send their results. There have been a few discussions about this in the past, but in short the easiest way to balance participants among your lists is to use URLs with the "withsquare" option.

Best,

Alexandre

To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/0ec32988-f004-44fc-95fb-46018ce487e3n%40googlegroups.com.

[Jeremy]

Thanks Alex. The 'server.py?withsquare=n' works the best for me.

Best wishes,

Jeremy

[Özge BAKAY]

Hi,

I had a similar problem: I have four lists, but one of them is completed by 17 participants whereas the others are by 11 each. I understand the cause of this problem. Above you said that sending the link to the participants with "withsquare" solves the problem. The link to my experiment is this: https://spellout.net/ibexfarm/manage/Offline2 Can you tell where/how to add this to an experiment link?

Best,

Özge

[Alexandre Cremers]

Hi,

This is the link to the managing page for your experiment. The link to your experiment would be something like "https://ibex.spellout.net/experiments/USERNAME/EXPERIMENTNAME/view". Just add "?withsquare=XXX" at the end of that link.

Best,

Alexandre

To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/6930e7f1-59fb-481b-a93f-d06790550387n%40googlegroups.com.

[Özge BAKAY]

Hi,

I tried this but it did not work: https://spellout.net/ibexexps/ozgebakay/Offline2/experiment.html/view?withsquare=2 I am using IbexFarm, not PC Ibex. Could this be why?

Özge

To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/CABk4Z5fW3QeT65Bwox9KuFdBKMKYJSK67YDs84e33V%2BcChckWg%40mail.gmail.com.

[Alexandre Cremers]

You're using the old Ibex, so the link should be: https://spellout.net/ibexexps/ozgebakay/Offline2/experiment.html?withsquare=2

That one seems to work, although I let you check if it shows the correct items.

A

To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/CADc6OM_y5%3DauPOgX39VQ7%3DfH7-fUFQPg_eNj7ThO2P-EEHxGGw%40mail.gmail.com.

[Alex Drummond]

With the old Ibex Farm you may need to replace 'experiment.html' with
'server.py' in Alexandre's link for the withsquare parameter to take
effect.

Moving over to the new Ibex Farm is strongly encouraged!

Alex

> To view this discussion on the web visit https://groups.google.com/d/msgid/ibexexperiments/CABk4Z5dpx%3D7s3yJkXAHVN8mC3DLBeSm0M9vs-1kCMYuJXrP0KQ%40mail.gmail.com.