I have a study in which I have participants try to guess other participants' IAT scores from an old study. There are 400 old participants from a previous study, with 5 IAT scores each, and each new participant makes guesses for 3 old participants. The issue that I'm encountering is that if each new subject just gets a random sample of 3 old subjects, the distribution of number of unique guesses for each old participant is super unequal and remains unequal even if quite large numbers of new participants were to be recruited (which I confirmed with some R simulations).
My question is whether there is a relatively easy way to make sure that the distributions are more equal, e.g., by creating a previous pre-randomized list and keeping track of the serial number of each new participant being recruited for the study. If this is not trivial, we might be able to find some type of workaround, but for our analyses it would be super helpful if we could get comparable numbers of guesses for each old participant.
Thanks so much for any pointers on this!