DTF

[Jae Yung Kwon]

Hi Phil and others in this group,  

When I tried to run DTF with my 2 factor MIRT model, I received an error message: Error: mod@Model$nfact == 1L is not TRUE

Does DTF not work for this model? Any tips to get the below DTF code to work would be greatly appreciated. 

model1=mirt.model('

Physical= 1,2,3,4,5,6

Mental= 7,8,9,10,11,12 

COV=Physical*Mental')

mod=multipleGroup(dat, model1, group, SE=TRUE,invariance=c('slopes','intercepts','free_means'))

DTF(mod)

[Phil Chalmers]

The error is correct, DTF() does not currently support multidimensional models. I'm not so sure you'd want it anyway, particularly for this type of model structure, so I don't think you're at a loss. Cheers.

Phil

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jae Yung Kwon]

Hi Phil, 

  Thank you for pointing this out. After much thought about how to show the effect size of DIF items (itemplot for MIRT shows 3 dimensional model, which is hard to interpret), would it also work to use a1 and the highest difficulty parameter, d1, and plot the ICC? So for V4 item in the output below, I can plot ICCs for gender based on a1 and d1?

#Multidimensional IRT

model1=mirt.model('

Physical= 1,2,3,4,5,6

Mental= 7,8,9,10,11,12 

COV=Physical*Mental')

#Completely independent group

mod2=multipleGroup(dat, model1, group=group, invariance=c('free_means','free_varcov'))

coef(mod2, simplify=TRUE)

> coef(mod2, simplify=TRUE)

$`1`$items

         a1      a2      d1      d2       d3      d4       d5

V4  8.253 0.000 14.185  8.032  1.961 -3.629     NA

  $`2`items

          a1     a2     d1     d2       d3      d4       d5

V4  5.465 0.000 9.147  6.115  1.650 -2.029     NA

Thank you in advance for your feedback,

Jae Yung

[Phil Chalmers]

Sure that could work. And I think the drop.zeros input might be helpful here as well to force itemplot to only create graphics in the lowest dimension. I haven't tested it for multiple-group models, but I see no reason why it shouldn't be possible (if it doesn't then let me know and I'll patch the function). Cheers. 

Phil

[Jae Yung Kwon]

Hi Phil,

   There doesn't seem to be any change. The drop.zero point drops the discricrimination/slope parameter, not the difficulty/d parameter. Would it help and perhaps others if I shared what I have so far and the data I used? 

[Phil Chalmers]

Hi Jae Yung,

I'm not quite sure I follow what you are looking for, sorry. Why do you need to separate the intercepts? Are the graphics just too busy?

Phil

[Jae Yung Kwon]

Hi Phil,

    Yes, it is difficult to see the size of DIF between gender when it is 3 dimensional so being able to graph with one intercept or the highest d in the MIRT model would make it easier to see. Is there a way this can be done in the MIRT package? 

[Jae Yung Kwon]

Hi Phil,

  I'm on my last leg of trying to find DTF by comparing DIF-adjusted and non-adjusted MIRT model and wondering whether I can set up the final model as below? 

model <- mirt.model('Theta = 1-12 

                    CONSTRAINB = (1-5,7-12,a1,a2), (1,6-12,d1,d2,d3,d4,d5)') 

final.mod=multipleGroup(dataset[,-1],model, as.character(dataset$Gender), SE=TRUE, 

                        invariance=c('free_means','free_varcov'))

Unfortunately, this gives me a warning sign: Error: CONSTRAINB = ... indexed a parameter that was not relavent accross groups

In addition: Warning message:

In UpdateConstrain(pars = pars, constrain = constrain, invariance = invariance,  :

  NAs introduced by coercion

Any specific feedback would be greatly appreciated, and thank you again for the great MIR package.

[Seongho Bae]

Hi, Jaeyung

Why did you set dataset$Gender as character? That's may cause that messages when you calibrate it.

It may works fine;

mod_final <- multiplegroup(data = dataset[,-c(1)], model = model, group = as.factor(dataset$Gender), SE = T, invariance = c('free_means', 'free_varcov'))

Seongho

2016년 9월 16일 금요일 오전 5시 54분 56초 UTC+9, Jae Yung Kwon 님의 말:

[Jae Yung Kwon]

Hi Phil and Seonho,

  Thank you both for your feedback, still getting errors in the CONSTRAINB section. Does the MIRT package work for the model below? How could I set up a1,a2, and constraints for multiple d1-d5? 

#DTF comparing Dif-adjusted and non-adjusted model

model <- mirt.model('

                    Physical= 1,2,3,4,5,6

                    Mental= 7,8,9,10,11,12 

                    COV=Physical*Mental 

                    CONSTRAINB = (1-5,7-12,a1,a2), (1,6-12,d1,d2,d3,d4,d5)') 

final.mod=multipleGroup(dataset[,-c(1)], model = model, group = as.factor(dataset$Gender), SE=T, 

                        invariance=c('free_means','free_varcov'))

Error in UpdateConstrain(pars = pars, constrain = constrain, invariance = invariance,  : 

  trying to get slot "parnum" from an object of a basic class ("NULL") with no slots

In addition: Warning message:

In UpdateConstrain(pars = pars, constrain = constrain, invariance = invariance,  :

  NAs introduced by coercion

[Phil Chalmers]

It looks fine syntactically. How many unique categories are there for the items with intercept constraints? It's possible they don't have the same number.

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[Phil Chalmers]

Oh wait, the slope constraints still look off. I think you need to separate them into (1-5, a1), (7-12, a2).

[Jae Yung Kwon]

Hi Phil,

  Thank you for your suggestion, I was able to build a final model that accounts for the DIF effects; however, I am struggling with interpretation of which group at the test level (e.g., males or females) is scoring higher or lower because the DTF function does not work in this case. You mentioned that DTF was part of your dissertation and wondering if you had any suggestions.

#DTF comparing Dif-adjusted and non-adjusted model

model <- mirt.model('

                    Physical= 1,2,3,4,5,6

                    Mental= 7,8,9,10,11,12 

                    COV=Physical*Mental 

                    CONSTRAINB = (1-5,a1),(7-12,a2),(1,6-12,d1),(1,6-12,d2),

                    (1,6-12,d3),(1,6-12,d4),(9-11,d5)')

                    

final.mod=multipleGroup(dataset[,-1],model, as.factor(dataset$Gender), SE=TRUE, 

                        invariance=c('free_means','free_varcov'))

> anova(final.mod,mod.ind)

Model 1: multipleGroup(data = dataset[, -1], model = model, group = as.factor(dataset$Gender), 

    invariance = c("free_means", "free_varcov"), SE = TRUE)

Model 2: multipleGroup(data = dataset[, -1], model = model1, group = as.factor(dataset$Gender))

       AIC     AICc    SABIC      BIC    logLik     X2  df     p

1 23024.58 23035.51 23140.82 23359.97 -11443.29    NaN NaN   NaN

2 23034.68 23069.54 23236.84 23617.96 -11397.34 91.904  51 4e-04

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.

[Phil Chalmers]

On Thu, Sep 22, 2016 at 12:20 PM, Jae Yung Kwon <pres...@jykngroup.com> wrote:

Hi Phil,

  Thank you for your suggestion, I was able to build a final model that accounts for the DIF effects; however, I am struggling with interpretation of which group at the test level (e.g., males or females) is scoring higher or lower because the DTF function does not work in this case.

What do you mean it does not work? It should work just fine if the model has been set up correctly.

 

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[_adb_]

I have a very similar set-up and I'm wondering whether you could comment on "best practices" for analyses of DIF and DTF for a 2-dimensional graded response model.

I'm overwhelmed with different approaches I've seen (Raju's "volume test", Lord's Wald test, the lordif package, LRTs). I'm absolutely new to mirt and any pointers to generally accepted good practices (and potentially even sample code) would be very helpful. I guess every method has its pros / cons but I'm just looking for what is doable with the mirt package and what makes sense to spend time on / read more about.

Thank you very much for your help!

Andy

[Phil Chalmers]

See this thread: https://groups.google.com/forum/#!topic/mirt-package/MoAvuJFfyL4

Phil

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[Jae Yung Kwon]

Well, you mentioned that DTF in multidimensional models (which my model was set up in) does not work so there were 2 things that I tried to do: 

1) I tried to use itemplot with drop.zeros to make it easier to see where DIF was occurring; however, I was not able to interpret the graph below. 

2) The second method might be to look at the extent of bias one may anticipate based on predetermined cutoffs such as determining percentage of people with DIF impact that exceeds minimal important difference. 

I think _adb_ was having similar issues as mine.

Any other thoughts on how to determine best the effect size of DIF would be greatly appreciated.

Warm regards,

Jae Yung

[_adb_]

Hi - (sorry, it looks like my follow-up message wasn't sent yesterday),

I looked at the link you (Phil) sent but it does seem like it only refers to the uni-dimensional case. My question was rather about what type of analysis would be recommendable for assessments of DTF (and DIF) in a two-dimensional set-up (using the mirt package).

I had come across multiple options (for the first time), such as Raju's "volume test", Lord's Wald test, the lordif package, and LRTs. From your link, an inspection of two-dimensional graphs in the same flavor of Chalmers, Counsell, & Flora, 2015, also looks like an option (but the DTF command is not available, as mentioned).

Can you recommend any two-dimensional approach in particular?

Thanks again for your help,

Andy

[Phil Chalmers]

On Fri, Sep 23, 2016 at 2:11 AM, Jae Yung Kwon <pres...@jykngroup.com> wrote:

Well, you mentioned that DTF in multidimensional models (which my model was set up in) does not work so there were 2 things that I tried to do: 

1) I tried to use itemplot with drop.zeros to make it easier to see where DIF was occurring; however, I was not able to interpret the graph below. 

Yeah, multidimensional DIF is difficult to visualize. I'm not even sure I can make many recommendations at this point. Perhaps manually compare the difference between expected scoring surfaces (I just noticed that the current version doesn't support itemtype(mod, 1, type = 'score') for MG objects....so I'll add that for the next release).

2) The second method might be to look at the extent of bias one may anticipate based on predetermined cutoffs such as determining percentage of people with DIF impact that exceeds minimal important difference. 

That's difficult to do because you're trying to combine two pieces of information together (expected bias, implied at the population level, and those impacted, which is a function of measurement precision for each individual). It's likely possible, though not really very easy. 

 

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.

[Phil Chalmers]

On Fri, Sep 23, 2016 at 1:17 PM, _adb_ <andreas....@gmail.com> wrote:

Hi - (sorry, it looks like my follow-up message wasn't sent yesterday),

I looked at the link you (Phil) sent but it does seem like it only refers to the uni-dimensional case. My question was rather about what type of analysis would be recommendable for assessments of DTF (and DIF) in a two-dimensional set-up (using the mirt package).

DTF is difficulty in more than 1 dimension, and may not be what you want anyway.

 

I had come across multiple options (for the first time), such as Raju's "volume test", Lord's Wald test, the lordif package, and LRTs. From your link, an inspection of two-dimensional graphs in the same flavor of Chalmers, Counsell, & Flora, 2015, also looks like an option (but the DTF command is not available, as mentioned).

For DIF, mirt will support Wald tests (wald()) and LRT (anova()). There is a massive number of DIF procedures out there, but those two will support multidimensional models (while many others will not). HTH.

Phil

 

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.