Participants show too few test items (Latin Square problems)

[Toby Lowther]

Hello!

I'm currently working on a fairly complex experiment, which I'm running through IbexFarm. The experiment has two parts (different kinds of trial), each composed of a number of test sentences with filler sentences destributed among them.

However, in both halves of the trial, participants are being shown too few test sentences. The first half is supposed to include 14 test sentences (each chosen pseudorandomly from one of 4 lexical conditions), while the second half is supposed to include 8 test sentences (chosen from 4 lexical conditions). However, when I run the experiment, each participant is only given 10 test sentences in the first half, and only 4 in the second.

The shuffle sequence is defined as follows:

var shuffleSequence = seq("setcounter", 

            "intro", 

            "instructions", 

            "demographics", 

            "scale", 

            sepWith("sep", seq("practice", rshuffle(startsWith("test."), startsWith("F-")))),

            "instructions2",

            sepWith("sep", seq("xprac", rshuffle(startsWith("fc."), startsWith("X-")))));

where "test." items are test items in the first half, "F-" are fillers for the first half, "fc." are test for the second half, and "X-" are fillers for the second half. ("practice" and "xprac" are practice items.)

The test items in the first half are divided into 14 condition groups using the standard notation, for example,

[["test.eng.eng.eng.ov",1], "AcceptabilityJudgment", {s: "He thinks that the father the son meets"}],

    [["test.eng.eng.ger.vo",2], "AcceptabilityJudgment", {s: "He thinks that der Vater meets den Sohn"}],

    [["test.eng.eng.ger.ov",3], "AcceptabilityJudgment", {s: "He thinks that der Vater den Sohn meets"}],

    [["test.eng.ger.eng.vo",4], "AcceptabilityJudgment", {s: "He thinks that the father trifft the son"}],

    [["test.eng.ger.eng.ov",5], "AcceptabilityJudgment", {s: "He thinks that the father the son trifft"}],

    [["test.eng.ger.ger.vo",6], "AcceptabilityJudgment", {s: "He thinks that der Vater trifft den Sohn"}],

    [["test.eng.ger.ger.ov",7], "AcceptabilityJudgment", {s: "He thinks that der Vater den Sohn trifft"}],

    [["test.ger.ger.ger.vo",8], "AcceptabilityJudgment", {s: "Er denkt dass der Vater trifft den Sohn"}],

    [["test.ger.eng.ger.vo",9], "AcceptabilityJudgment", {s: "Er denkt dass der Vater meets den Sohn"}],

    [["test.ger.eng.ger.ov",10], "AcceptabilityJudgment", {s: "Er denkt dass der Vater den Sohn meets"}],

    [["test.ger.ger.eng.vo",11], "AcceptabilityJudgment", {s: "Er denkt dass the father trifft the son"}],

    [["test.ger.ger.eng.ov",12], "AcceptabilityJudgment", {s: "Er denkt dass the father the son trifft"}],

    [["test.ger.eng.eng.vo",13], "AcceptabilityJudgment", {s: "Er denkt dass the father meets the son"}],

    [["test.ger.eng.eng.ov",14], "AcceptabilityJudgment", {s: "Er denkt dass the father the son meets"}],

Likewise, the test items for the second half are in 8 groups.

[["fc.eng.eng.eng",1], "Question", {q: "He knows that the teacher…", as: ["helps the student.", "the student helps.", "Neither of the above."]}],

    [["fc.eng.eng.ger",2], "Question", {q: "He knows that der Lehrer…", as: ["dem Schüler helps.", "helps dem Schüler.", "Neither of the above."]}],

    [["fc.eng.ger.eng",3], "Question", {q: "He knows that the teacher…", as: ["hilft the student.", "the student hilft.", "Neither of the above."]}],

    [["fc.eng.ger.ger",4], "Question", {q: "He knows that der Lehrer…", as: ["dem Schüler hilft.", "hilft dem Schüler.", "Neither of the above."]}],

    [["fc.ger.ger.ger",5], "Question", {q: "Er weiß dass der Lehrer…", as: ["hilft dem Schüler.", "dem Schüler hilft.", "Neither of the above."]}],

    [["fc.ger.eng.ger",6], "Question", {q: "Er weiß dass der Lehrer…", as: ["dem Schüler helps.", "helps dem Schüler.", "Neither of the above."]}],

    [["fc.ger.ger.eng",7], "Question", {q: "Er weiß dass the teacher…", as: ["hilft the student.", "the student hilft.", "Neither of the above."]}],

    [["fc.ger.eng.eng",8], "Question", {q: "Er weiß dass the teacher…", as: ["the student helps.", "helps the student.", "Neither of the above."]}],

If it's relevant, there are 28 possible fillers for each part.

As far as I can tell, there's no reason why the code should only draw 10 test sentences (rather than 1 from all 14 groups), nor why it should only draw half the number of test sentences (4 out of 8) in the second half.

Any advice would be much appreciated!

Sincerely,
Toby S. Lowther

[Toby Lowther]

Solved! - the problem was drawing from the conditions 1-8 in both parts of the study - making the second part 15-22 instead of 1-8 fixed it!