Using DIF in MIRT

[Jaime Leth-Steensen élève]

> library("hemp")

Loading required package: psych

Loading required package: lattice

Loading required package: lavaan

    etc……

Warning messages:

1: package ‘lavaan’ was built under R version 3.5.3 

2: package ‘lme4’ was built under R version 3.5.3 

> library("mirt")

> datb = read.delim("full3b.dat", header=T, sep = "\t")

> twopl_mod <- "F = 1 - 20"

> twopl_fit <- multipleGroup(data = datb[,2:21], model = twopl_mod, SE = TRUE, group = datb$Group)

Iteration: 241, Log-Lik: -3206.909, Max-Change: 0.00010

Calculating information matrix...

> results_irtlr <- DIF(MGmodel = twopl_fit, which.par = c("a1", "d"), scheme = "add")

No hyper-parameters were estimated in the DIF model. For effective

DIF testing, freeing the focal group hyper-parameters is recommended.

EM cycles terminated after 500 iterations.

> irtlr_summary(results_irtlr, alpha = 0.05)

       Item chi_square df p_value

1   F._1  21.803270  2   0.000

2   F._2  29.918405  2   0.000

3   F._4  33.851384  2   0.000

4   F._5  38.064229  2   0.000

5   F._6  44.115362  2   0.000

6   F._7  41.360044  2   0.000

7   F._8  21.133380  2   0.000

8   F._9  39.686292  2   0.000

9  F._10  55.298514  2   0.000

10 F._12  49.459373  2   0.000

11 F._14   9.188372  2   0.010

12 F._15  12.512723  2   0.002

13 F._16  27.718272  2   0.000

14 F._17  46.547650  2   0.000

15 F._18  13.788179  2   0.001

16 F._19  62.635199  2   0.000

17 F._20  35.745867  2   0.000

 

Comment: This is what I got with code taken from Handbook of Educational Measurement

Using R - Chap. 11. After checking the https://rdrr.io/cran/mirt/man/DIF.html link and realizing

that the latent mean and sd had to be fixed in at least one of the groups using a set 

of anchor items, I then went back to the SPSS file and moved F.3, F.11, F.13, and F.14 (it

had the highest p-value and the others had no initial `DIF`) to the end of the data file. 

Then I resaved it as a new tab-delimited .dat file (and got rid of the “hieroglyphics” at 

the start of that dat file using the MPlus Editor) and ran the following R code (from the 

examples on the link).

 

> datc = read.delim("full4.dat", header=T, sep = "\t")

> itemnames <- colnames(datc)

> twopl_fit2 <- multipleGroup(data = datc[,2:21], model = twopl_mod, SE = TRUE, group = datc$Group, invariance = c(itemnames[18:21], ‘free_means’ ‘free_var’))                                              

Iteration: 254, Log-Lik: -3209.552, Max-Change: 0.00009

Calculating information matrix...

> results_irtlr2 <- DIF(MGmodel = twopl_fit2, which.par = c("a1", "d"), items2test=2:17)

> irtlr_summary(results_irtlr2, alpha = 0.05)

      Item chi_square df p_value

1  F._2  12.557788  2   0.002

2 F._10   8.668266  2   0.013

3 F._16   6.752910  2   0.034

4 F._19   9.474107  2   0.009

Question: Is this analysis fine (i.e., or am I still missing something - because 

right now I am very confused as to whether I am now getting the DIF result that I 

need here)?

[Phil Chalmers]

I can't answer if it's fine or not, but yes the last line looks more kosher to what a MG DIF analysis looks like (selection of anchors, freeing the scale of the focal groups). Only under very rare circumstances would it be reasonable to assume the latent means and variances/covariances are equal across groups.

Phil

---

Le présent courriel et toutes les pièces jointes peuvent contenir de l’information privée, exclusive, privilégiée ou confidentielle, sujette au droit d’auteur s’adressant uniquement à l’individu ou à l’organisme ou à l’agent responsable de le lui livrer. Toute utilisation, copie ou distribution non autorisée du contenu de ce courriel est interdite. Si vous croyez que ce message est un pourriel au sens de la Loi canadienne anti-pourriel, veuillez le faire suivre à l’adresse suivante : pour...@ecolecatholique.ca . Si vous avez reçu ce courriel par erreur, veuillez en informer l’expéditeur par retour de courriel et supprimer de votre système ce message et tout document joint. Merci de votre collaboration.

---

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/eed55735-3e8a-48af-83c5-cc0fe1e55ea8%40googlegroups.com.

[Jaime Leth-Steensen élève]

Wow thanks for the reply. I have a followup if you don't mind (surprise , surprise)

1. I then tried to determine whether the DIF for each of these items was non-uniform or uniform by 

running the following two lines. As you can see Item 2 had non-uniform DIF and Item 19 had 

uniform DIF (these are binary items) but there were no results here for the other two items (so the 

folks that I'm running the analysis for are very disturbed by this and are not sure what to report).

Any thoughts for us about this?

> results_irtlr2 <- DIF(MGmodel = twopl_fit2, which.par = c("a1"), items2test=2:17)

> irtlr_summary(results_irtlr2, alpha = 0.05)

     Item chi_square df p_value

1 F._2    10.1713  1   0.001

> results_irtlr2 <- DIF(MGmodel = twopl_fit2, which.par = c("d"), items2test=2:17)

> irtlr_summary(results_irtlr2, alpha = 0.05)

      Item chi_square df p_value

1 F._19   3.908045  1   0.048

2. Next, when we ran these data through IRTPro (with the same set of four anchor items), it did 

indicate DIF for these same four items (now accompanied by significant slope and/or threshold

for each) but now also for Item 7 and Item 17 (with quite small ps for each).

Again, the people I am doing this for are now really confused as to which results to report.

Any other thoughts on this situation for us?

[Phil Chalmers]

For a 2PL item testing non-uniform DIF should be focusing on both the slope and intercept at once as joint test; otherwise, the results make little sense (it would analogous to testing an interaction effect in a regression  model by constraining the initial starting locations [intercepts] to be the same, which doesn't usually make much sense). Hence, your which.par should be which.par = c("a1", "d"). IRTPRO  and mirt should give effectively identical results, so I think this can be attributed to user specification error.

Phil

---

Le présent courriel et toutes les pièces jointes peuvent contenir de l’information privée, exclusive, privilégiée ou confidentielle, sujette au droit d’auteur s’adressant uniquement à l’individu ou à l’organisme ou à l’agent responsable de le lui livrer. Toute utilisation, copie ou distribution non autorisée du contenu de ce courriel est interdite. Si vous croyez que ce message est un pourriel au sens de la Loi canadienne anti-pourriel, veuillez le faire suivre à l’adresse suivante : pour...@ecolecatholique.ca . Si vous avez reçu ce courriel par erreur, veuillez en informer l’expéditeur par retour de courriel et supprimer de votre système ce message et tout document joint. Merci de votre collaboration.

---

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mirt-package/9f077a5a-c631-4ff4-8ad3-906cece053b4%40googlegroups.com.