Hi all,
I have a dataset with Likert-type judgments (multiple judges) of productions of multiple participants at multiple tasks (but not all participants took the same tasks, it's a bit of planned missingness design in terms of tasks).
I am trying to estimate each individual's ability reflected in the different productions judged, taking into account both judge characteristics (different uses of the judgment scale, severities) with a GRM or GPCM and (random) task difficulty. That's the last part I'm a bit confused with.
I know how to fit a GRM and GPCM model to account for judge variability (like a "regular" IRT), but I am unsure how to also control for task difficulty when the participants have taken different tasks.
So far what I came up with from the vignette is to use a "regular" GRM or a GPCM with the judgments (treated as items), with the function mixedmirt(), where Theta would be modeled as a function of an intercept (the individual's ability) + a random intercept per task :
model = 1,
itemtype = "graded",
lr.fixed = ~ 1 + id, #the latent trait estimate depends on the participant
lr.random = ~ 1|task #the latent trait depends on the task (randomly picked from a population of tasks)
And my task-controlled ability estimates would be the "id" mean estimates, I guess (which I hope to export in some way). Also I hope to study task difficulty with the fixed task effect.
Does that sound about right?
---
Also, from my understanding, the previous model assumes that for a task, there is a unique difficulty level for all participants. So a follow-up question would be, what if I can't make that assumption (it other terms, that the difficulty of the task can vary randomly per participant?) or want to test it? Would I need a random slopes and intercept model, such as...
lr.fixed = ~ 1 + id
lr.random = ~ 1|task + id|task
?
Sorry if all of this is confusing, and thank you in advance for your help...and also: Congrats for such a great R package.
Nils