Re: Estimating Person Fit on-the-fly

[Phil Chalmers]

It would probably require a bit of code tweaking to make this work, but in any case mirt's person-fit statistics are very limited in scope anyway. You might have more luck using the PerFit package, which generally works well with mirt objects (at least, last time I checked), and may be easier to perform one at a time evaluations with less ad-hoc code than hacking at mirt directly. HTH. 

Phil

On Fri, Jan 14, 2022 at 5:47 AM aide...@gmail.com <aide...@gmail.com> wrote:

Hi there, 

Is it possible to estimate person infit/outfit or Zh value using just one participant's response pattern given the items' psychometric properties? If not, is there an established method to evaluate person fit on-the-fly in an adaptive environment? 

The goal is to be able to calculate interim person index as one would calculate interim theta in a CAT environment. This will allow us to cut off the CAT even further if we suspect someone is not doing the test properly. 

For example, 

 

require(mirt)
data(Science, package = 'mirt')
mod <- mirt(Science, 1)
all_Theta=fscores(mod, method='MAP')
mirt::personfit(mod, Theta=all_Theta) # This works

# Try to calculate the person fit for a specific person. 
person_Theta=fscores(mod, method='MAP', response.pattern = c(1,2,3,4))
mirt::personfit(mod, Theta=person_Theta[1]) # This doesn't work

Thank you!

Aiden 

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/42e445d0-b1d2-4fa7-a590-bcf6879f387an%40googlegroups.com.

[aide...@gmail.com]

Thanks for the pointer Phil. That's very helpful! I'll look into perFit package instead. 

[cff...@gmail.com]

Hi Phil,

I wonder if I can confirm that the status of this is the same as it was when this issue was first asked about? I have some use for person fit statistics from multidimensional models, in the same way as asked in the original question (fit model with one sample, apply to a separate sample). Unless I'm missing something, I don't see PerFit being able to do this. Of course, no problem if there's no update yet! I wish I had more time to contribute...

Thank you!

Carl

[Phil Chalmers]

Hi Carl,

There hasn't really been any movement in this area, unfortunately. Though it is of course still possible to do with a bit of setup. If you have the model fitted to a set of sample data then you can of course extract these estimates and create a dummy model that contains the new observations. Something like the following would work:

mod <- mirt(sample_data) # your fitted model

sv <- mod2values(mod) # translate into parameter estimation structure

mod_new <- mirt(new_data, pars=sv, TOL=NaN) ## create new dummy mod that contains the new responses

personfit(mod_new)  # obtain person fit values

If the new_data above doesn't have the same response variance for each item then you'll also have to supply a technical$customK input to trick the program into thinking there are more categories than what were present, though it's admittedly a little clunky. Happy to help with this if needed.

Phil

To view this discussion visit https://groups.google.com/d/msgid/mirt-package/3d051ce0-50fc-44f8-a547-37e6c03eb29fn%40googlegroups.com.

[Carl Falk]

Thank you, Phil! This may do the trick.