Functioning of drop_sequential in DIF analyses

[Dirk Pelt]

Hi Phil,

I am have trouble with understanding the DIF function, particularly the drop_sequential option. If I 'manually' recreate what is I think is going on in the loop, I get different results.

Let's say I first run a constrained model on 10 items between 2 groups (using invariance = c(colnames(data), 'free_means', 'free_var') ), and then find that item 1 through 4 are flagged as displaying DIF given seq_stat. I would guess that the next step in drop_sequential would be to run a new model, with item 5 through 10 as anchors, and then testing DIF again on items 1 through 4. I recreate this as follows:

itemnames <- colnames(data)

model_anchor<-multipleGroup(data, model = 1, group = group, invariance = c(itemnames[1:4], 'free_means', 'free_var'))

dif.anchor <- DIF(model_anchor, c('a1','d1'), items2test = 5:10)

etc. etc.

Following this procedure I arrive at other DIF items then the drop_sequential option. Also, the other way around; after running drop_sequential, a number of items are flagged as DIF (let's say item 8.through 10). If I then run a model with item 1 through 7 as anchors, subsequently test item 8 through 10 for DIF, then suddenly item 8 is not flagged for DIF anymore. Shouldn't this be impossible? 

I have been looking at the source code but I am still confused. Could you tell me what happens next in the above example, perhaps using the multipleGroup and DIF? 

Many thanks, 

Dirk

[Phil Chalmers]

On Fri, Mar 2, 2018 at 3:41 AM, Dirk Pelt <dirkp...@gmail.com> wrote:

Hi Phil,

I am have trouble with understanding the DIF function, particularly the drop_sequential option. If I 'manually' recreate what is I think is going on in the loop, I get different results.

Let's say I first run a constrained model on 10 items between 2 groups (using invariance = c(colnames(data), 'free_means', 'free_var') ), and then find that item 1 through 4 are flagged as displaying DIF given seq_stat. I would guess that the next step in drop_sequential would be to run a new model, with item 5 through 10 as anchors, and then testing DIF again on items 1 through 4.

That's actually not what's going on this in this scheme. The first part you described is correct, where if items 1:4 were flagged as having DIF then a new model would be created reflecting this using 6 rather than 10 anchors. However, once they are flagged they are not re-tested in the next stage. Only the remaining items that are constrained (i.e., the 6 anchors) are tested for DIF. 

It sounds like what you are looking for is a different algorithm, of which for this type of search there are clearly many. I'd be happy to add something like this, though I think the logic is a little odd (if 4/10 items are flagged with DIF, the next stage should be tested with a new model with more parameters, but all 10 items are again tested). I've organised the DIF function this way because once an items is flagged for DIF, there's a pretty good chance it would be flagged again when retested in this 2nd, 3rd, ..., nth step. This likely reflects your issue with testing DIF in the other direction as well, starting with a more parameterized model.

Phil 

 

I recreate this as follows:

itemnames <- colnames(data)

model_anchor<-multipleGroup(data, model = 1, group = group, invariance = c(itemnames[1:4], 'free_means', 'free_var'))

dif.anchor <- DIF(model_anchor, c('a1','d1'), items2test = 5:10)

etc. etc.

Following this procedure I arrive at other DIF items then the drop_sequential option. Also, the other way around; after running drop_sequential, a number of items are flagged as DIF (let's say item 8.through 10). If I then run a model with item 1 through 7 as anchors, subsequently test item 8 through 10 for DIF, then suddenly item 8 is not flagged for DIF anymore. Shouldn't this be impossible? 

I have been looking at the source code but I am still confused. Could you tell me what happens next in the above example, perhaps using the multipleGroup and DIF? 

Many thanks, 

Dirk

--
You received this message because you are subscribed to the Google Groups "mirt-package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dirk Pelt]

Thanks for the information. I saw that I made a mistake in my code by the way, for clarity I have changed it below.

itemnames <- colnames(data)

model_anchor<-multipleGroup(data, model = 1, group = group, invariance = c(itemnames[5:10], 'free_means', 'free_var'))

dif.anchor <- DIF(model_anchor, c('a1','d1'), items2test = 1:4)

Best, 

Dirk

To unsubscribe from this group and stop receiving emails from it, send an email to mirt-package...@googlegroups.com.