Error: data contains completely empty response patterns. Please remove

[Virginia Ros‏]

Hi Phil, 

With the new version of the 'mirt' package I get an error that didn't happen with the previous versions.

When I try to run any multirgroup model (one or two factors) using this script

s1 <- 'F = 1-9' 

model1Factor <- multipleGroup(data, s1, group = group, survey.weights= peso ) 

I get this error

Error: data contains completely empty response patterns. Please remove

And I cannot make my mind around what's going on because it worked until the update and I am using the same data and the same syntax. 

Thank you very much, 

Virginia

[Phil Chalmers‏]

Hi Virginia,

This is new behaviour for mirt, and was brought to my attention by another user as being non-kosher behaviour for IRT software. Essentially, what has happened is one or more rows in your data contain only NA's and therefore have no observed responses. Vectors like these contribute nothing to the model, have no person fit capability, and do not need to be scored (e.g., you might as well set their ability and precision equal to the mean and SD of the prior when scoring via EAP). So, from here on out these response vectors should be removed entirely.

Here's a quick way to do this:

dat <- subset(dat, rowSums(is.na(dat)) != ncol(dat))   

Cheers.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[virginia‏]

thanks! that was quick and easy...
I thought it could be that after sending the message but was confused because it was the same dataset I was using last week.
Anyway, it makes sense, thanks!

[Brenton Wiernik‏]

Rather than returning the error, could the mirt function instead automatically remove these rows and then throw a warning message? I have data with such rows with some frequency (e.g., some participants didn't take all measures), and it would be a pain to have to run the subset command with every mirt.

[Phil Chalmers‏]

I'm very hesitant to modify input data, especially because users could get tripped up when trying to do things like matching latent trait estimates with external data for further analyses. I don't think it's too much just to make a new object that is a subset of the original input (in most cases this would just be one extra line), and this will make the data operations much more explicit. 

The only other option is to add a new flag to the technical list, like technical = list(removeEmptyRows = TRUE); but by default I would insist that this is turned off.

Phil

On Thu, Sep 17, 2015 at 5:51 PM, Brenton Wiernik <quill...@gmail.com> wrote:

Rather than returning the error, could the mirt function instead automatically remove these rows and then throw a warning message? I have data with such rows with some frequency (e.g., some participants didn't take all measures), and it would be a pain to have to run the subset command with every mirt.

[Brenton Wiernik‏]

I guess my expectation would be that the function would remove these rows from the computation, but insert NAs into the person parameter/factor score rows to facilitate use with external data.

[Phil Chalmers‏]

This is part of the problem though, because factor scores are often defined when a vector contains only NA's. For example, when using EAP or MAP estimates a vector of NA's should technically return an estimate with all 0's, where the SEs are equal to the sqrt of the prior variance. So returning NA's for these cases is in fact wrong, but for ML/WLE estimation NA's returned would be more realistic. Same reasoning goes for plausible value imputations where these response patterns are perfectly valid but just add noise to the imputations. So it's not really clear which is better/more correct, therefore I think avoiding this issue altogether is the best route.

Phil

On Sat, Sep 19, 2015 at 12:11 AM, Brenton Wiernik <quill...@gmail.com> wrote:

I guess my expectation would be that the function would remove these rows from the computation, but insert NAs into the person parameter/factor score rows to facilitate use with external data.

[Aaron Kaat‏]

Hello Phil,

I'm a fan of the technical(removeEmptyRows=T) option, but I wonder if it causes other problems -- specifically with itemfit().  See my copy/paste code below and the comments regarding 

library(mirt)

jointitems <- c(PF,PWB) #combines list of column names

sub_dat <- subset(myhealth_dat, rowSums(is.na(myhealth_dat[,jointitems])) != ncol(myhealth_dat[,jointitems]))   #manually remove empty rows

oneF <- mirt(sub_dat[,jointitems],1,itemtype="graded",method="EM",quadpts=81,verbose=T,technical=list(NCYCLES=1000)) #converges on 456

oneF_rev <- mirt(myhealth_dat[,jointitems],1,itemtype="graded",method="EM",quadpts=81,verbose=T,technical=list(NCYCLES=1000,removeEmptyRows=T)) #converges on 416

PF_PWB_IF <- itemfit(oneF,method="EAP",impute=10,digits=3,Zh=F,S_X2=T) #works great

PF_PWB_IF_rev <- itemfit(oneF_rev,method="EAP",impute=10,digits=3,Zh=F,S_X2=T) # "Error in scores[match(sfulldata, stabdata2), , drop=FALSE] : subscript out of bounds"

Aaron

[Phil Chalmers‏]

Could you include data for the PF, PWB, my_healthdata, etc, objects? I can't reproduce the issue in the current form.

--

[Aaron Kaat‏]

See attached - I used the extract.mirt( ,"parvec") and simdata() to recreate the error as best as possible. Data is same size with same pattern of missingness, and same error.

Aaron

[Phil Chalmers‏]

Perfect, thanks. Should be patched now on the dev now, but let me know if you still have issues with your real data.

Phil

[pda...@ucdavis.edu‏]

Hi Phil, 

I am receiving this error even though I don't have any rows with completely missing data. However, I do have some rows with missing data on 6 out of the 8 questions I am trying to assess. Would this be causing the error?

Thanks!

Pega

[Phil Chalmers‏]

Hi Pega,

No, this shouldn't be the reason, missing data is not really a problem for the functions. Are you 100% certain you don't have at least one row with all NAs? If so, please open and issue on Github and supply a short but reproducible example, and I'll try and track down the problem. Cheers.

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[pda...@ucdavis.edu‏]

I'm pretty certain. I used the code you provided in previous comments and the subsetted data set contains the same number of rows as the original. I don't know how to open an issue on github so I will post the dataset and code in this email, if thats ok. 

itemnames <- colnames(mAAvW[ ,1:8])

dat <- subset(mAAvW, rowSums(is.na(mAAvW)) != ncol(mAAvW)) 

refmodel <- multipleGroup(data = mAAvW[ ,1:8], model = 1, group = mAAvW$group, SE = TRUE, 

       invariance = c('free_means', 'free_var', itemnames))

[Phil Chalmers‏]

table(rowSums(is.na(mAAvW[ ,1:8])))

gives

   0    1    2    3    4    5    6    8 
1167  172   47   21    5    5    2    1 

So there is 1 row with all NAs; row 1355 to be exact.

> which(rowSums(is.na(mAAvW[ ,1:8])) == 8)
4087 
1355 

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[Pega Davoudzadeh‏]

Wow, that one sneaky little row. Snuck right past me. Thank you and sorry for any inconveniences!

--

Pega Davoudzadeh, M.A., Ph.D. 

Department of Psychology

University of California, Davis