Error from multi-group analysis

[Calvin Kincade]

I am running an SEM model. The unconstrained model runs as expected and no errors. When I add the grouping variable, I get this error:

Error: Can't combine `..1` <character> and `..2` <double>.

Run `rlang::last_error()` to see where the error occurred.

The only difference between the multiple group model and the baseline model is adding the group argument (in blue).

output <- sem(SEM.model, 

           data=mydata,

           group="political",

           ordered=c("x1" , "x2" ,"x3" ,"x4" , "x5" , "x6","x7" ,"x8", 

                     "x9" ,  "x10" , "x11", "x12"))

I have confirmed that the political grouping variable has three levels (1,2,3):

political

<labelled<double>[600]>

[1] 1 1 1 1 1... [559] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3

Labels:

 value        label

     1      liberal

     2     moderate

     3 conservative

Any thoughts on what is causing the error?

[Patrick (Malone Quantitative)]

Try it without the "" around political. You want it to pull the values of the variable, not the variable name.

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/9f51d45f-bc65-4166-81df-e417b6d62cd0n%40googlegroups.com.

--

Patrick S. Malone, Ph.D., Malone Quantitative
NEW Service Models: http://malonequantitative.com

He/Him/His

[Calvin Kincade]

Thank you for responding, Patrick.  I took out the quotes and got this error:

Error in lav_data_full(data = data, group = group, cluster = cluster,  : 

  lavaan ERROR: grouping variable ‘2’‘2’‘2’‘1’‘1’‘2’‘1’‘1’‘1’‘2’‘1’‘1’‘1’‘2’‘1’‘2’‘1’‘2’‘1’‘2’‘1’‘1’‘1’‘1’‘1’‘2’‘2’‘2’‘1’‘1’‘2’‘2’‘2’‘1’‘1’‘1’‘1’‘1’‘2’‘2’‘1’‘2’‘1’‘2’‘2’‘1’‘2’‘2’‘1’‘1’‘2’‘1’‘2’‘2’‘2’‘1’‘2’‘2’‘2’‘1’‘1’‘2’‘2’‘2’‘1’‘1’‘2’‘2’‘2’‘1’‘2’‘2’‘1’‘2’‘2’‘2’‘1’‘2’‘2’‘2’‘1’‘2’‘2’‘2’‘2’‘2’‘2’‘2’‘1’‘1’‘2’‘1’‘2’‘2’‘1’‘2’‘1’‘2’‘2’‘1’‘2’‘2’‘1’‘1’‘2’‘1’‘1’‘2’‘1’‘2’‘1’‘2’‘1’‘1’‘1’‘1’‘1’‘2’‘1’‘2’‘2’‘2’‘2’‘1’‘2’‘2’‘1’‘1’‘2’‘2’‘2’‘2’‘2’‘1’‘2’‘1’‘2

In addition: Warning message:

In if (!(group %in% names(data))) { :

  the condition has length > 1 and only the first element will be used

In all the tutorials, I've seen quotes used in the group argument. Such as in the Lavaan tutorial:

fit <- cfa(HS.model, data = HolzingerSwineford1939, group = "school")

In the first run with the 'can't combine error', I attached the data. I tried it again without attaching the data, using mydata$political -- with and without quotes.  Same results.

[Patrick (Malone Quantitative)]

Ah, my mistake. Sorry to lead you the wrong way. I have no idea, then.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/1c02eb85-9300-4714-81a1-c6b6dad34226n%40googlegroups.com.

[Patrick (Malone Quantitative)]

What does str(mydata$political) return?

[Calvin Kincade]

Update--I reconfigured the politics variable from three groups to two and that worked (group = "political").  It must have been something in the way I constructed the variable in the first place.

[Calvin Kincade]

No problem; It did the same when I tried that.

[Calvin Kincade]

Now I have a new problem with the same dataset/analysis. Getting the groups function to work, I run the unconstrained model (group="political") and no constraints imposed--this is model.1. Then I run a model with the intercepts and regression coefficients constrained: group="politics",

group.equal=c("intercepts","regressions")

This is model.2.

When I try to run the chi-square test, I get this error:

> anova(model.1, model.2)

Error in UseMethod("anova") : 

  no applicable method for 'anova' applied to an object of class "character"

Can't think why this is coming up.  I have categorical variables so using the DWLS estimator. 

[Terrence Jorgensen]

  no applicable method for 'anova' applied to an object of class "character"

Can't think why this is coming up. 

Are you passing the character string with model syntax, rather than the object that contains the results?  Using the example from your original post, you should pass 

lavTestLRT(output) # anova() simply calls this function

not 

lavTestLRT(SEM.model)

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Calvin Kincade]

Yes! I see my error now--use the output results, not the model syntax. Thank you!

Then one more question--I am trying to determine if one variable in a fairly complex model is different across groups (seeing if there is moderation).  I run the multigroup model with no constraints (model.1); then I run the model with loadings, intercepts, residuals, lv.variances, and lv.convariances constrained to be equal (model.2). Because I have categorical variables and thus am using the WLSMV estimator, the chi-square difference test is the scaled Chi-Squared Difference Test (method = “satorra.2000”).  The p-value is not significant (p=0.333); therefore, I conclude the parameters are invariant across models (though my variable of interest, X1, is significant in one group, not in the other).

Nevertheless, I want to check the invariance of one variable's slope across the two groups (model.3): I constrain only that parameter in the model: c(a,b)*x1 I removed the function: group.equal=c("loadings", "intercepts", "residuals", "lv.variances", "lv.covariances", "regressions"), from the syntax.

That all seems to work--the models are both a good fit and there are no errors--I see X1's regression parameters labeled in both group outputs. When I check model.1 vs. model.3 in the chi-square test, it tells me that the chi-square test between model.1 and model.3 has the same degrees of freedom and I get this warning:

Warning message:

In lavTestLRT(output.1, output.3) :

  lavaan WARNING: some models have the same degrees of freedom

My question is this: shouldn't the models be nested given I included the constraint for x1's slope in model.3?

[Terrence Jorgensen]

The p-value is not significant (p=0.333); therefore, I conclude the parameters are invariant across models

Your test is invalid because you did not additionally constrain thresholds, so your latent item-response scales are not linked.  Including "intercepts" is meaningless with categorical indicators, unless you explicitly free some intercepts in the syntax, which is only identified if you constrain thresholds.  Read the Wu & Estabrook (2016) article.

I constrain only that parameter in the model: c(a,b)*x1

You haven't provided your actual model syntax, so it is hard to tell what you are actually doing.  This is only the right-hand side of a parameter specification in lavaan model syntax, but I can tell it does not constrain the parameter to equality.  You labeled the parameter "a" in the first group and "b" in the second group, and you can verify in the summary() output whether the estimates are in fact unequal.  Giving the parameter the same label would constrain it to equality.

https://lavaan.ugent.be/tutorial/groups.html

My question is this: shouldn't the models be nested given I included the constraint for x1's slope in model.3?

They might be nested, but the message says their df are the same, so if they are nested, they are equivalent.  You can verify this by printing each model and comparing their fit statistics.

[Calvin Kincade]

Thank you for the references, Dr. Jorgensen. Just to be clear, here is what I am doing: I want to test whether one variable’s regression coefficient, Z1, is invariant across two groups (Z4) in a structural model. In essence, I am attempting a moderation using multi-group analysis with SEM and categorical variables.

Here is the first model:
model.1 <- ‘
#measurement model 
Y1 =~ X1 + X2 + X3 +X4 + X5
Y2 =~ X6 + X7 +X8
#structural model 
Y1 ~ Y2 + Z1 + Z2 + Z3
Y2 ~ Z1 + Z2 + Z3’

fit_model.1 <-sem(model.1,
                                 data=mydata, 

                                 group = “Z4”,
                                 ordered=c(“X1” , “X2” , “X3” , “X4” , “X5”,
                                  “X6” , “X7” , “X8” , “Z1” ,“Z2” , ”Z3”))

Per your previous response, I then constrain the regression coefficients for Z1 on Y1 to be equal across both groups in model 2, or:

Y1 ~ Y2 + c(b1, b1)* Z1 + Z2 + Z3

Here is the fit code for structural invariance model, constraining equality for thresholds, lv.covariances plus the one regression path for Z1 noted above:

fit_model.2 <-sem(model.2, data=mydata,                                   

                                  group = “Z4”,
                                  group.equal=c(“thresholds”, "lv.covariances"),
                                  ordered=c(“X1” , “X2” , “X3” , “X4” ,+ “X5”,
                                                      “X6” , “X7” , “X8” , “Z1” ,“Z2” , ”Z3”))

 Now when I fit the model, the degrees of freedom are different; the chi-square test is not significant (p = 0.6745). No errors.

I conclude that the model is invariant with regard to the regression path for Z1 and there is no moderation concerning Z1. Does that look correct? (The Wu and Estabrook article was useful to understand the logic, but not the execution in Lavaan).

[Terrence Jorgensen]

Does that look correct? 

No, you cannot compare latent covariance-structure parameters without assuming metric invariance.  You need to constrain thresholds and loadings (but not necessarily intercepts) and test that against the configural model to see whether metric invariance is a tenable null hypothesis.  Then you can additionally constrain the latent regression slope to test that null hypothesis.  I have no idea why you would be constraining latent-variable covariances to equality.

[Calvin Kincade]

I see, thank you. I was cobbling together several tutorials so not sure myself (the categorical variables with thresholds is throwing me off--most examples for invariance testing use continuous data).

So following the standard test for invariance: 

(1) configural 
group = “Z4”,

(2) weak/metric

group = “Z4”,

group.equal=c("loadings")

(3) strong

group = “Z4”,

group.equal=c("loadings","thresholds")

(4) regression coefficients (if pervious invariance established)

group = “Z4”,

group.equal=c("loadings","thresholds", "regressions")

Or, use instead of "regressions" in group.equal, specify the one slope (Y1 ~ Y2 + c(b1, b1)* Z1 + Z2 + Z3)

[Patrick (Malone Quantitative)]

With categorical data, most sources say to skip Step 2.. Constraining loadings without constraining thresholds is not a useful test.

--

You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/544154c6-61c5-4679-9b87-715d62945475n%40googlegroups.com.

[Terrence Jorgensen]

Constraining loadings without constraining thresholds is not a useful test.

Indeed, without equal thresholds, the latent item responses are not on linked scales, so loadings are not comparable.  If you have at least 4 categories, you could constrain thresholds first (freeing intercepts and residual variances) followed by loadings for valid separate tests, if you are interested in that (see the ?measEq.syntax help page for assistance).

[Calvin Kincade]

Thank you for the tip! Could you point me to a source? (it would be nice to have something definitive to cite.) This is all beginning to make sense. I truly appreciate the comments and information.

[Terrence Jorgensen]

Could you point me to a source?

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5458787/

[Calvin Kincade]

Thank you! I came across this article that was very helpful, especially in explaining how thresholds figure into invariance testing: https://www.journals.uchicago.edu/doi/full/10.1086/681607

[Terrence Jorgensen]

I came across this article that was very helpful, especially in explaining how thresholds figure into invariance testing: https://www.journals.uchicago.edu/doi/full/10.1086/681607

I would be skeptical of the value of that tutorial.  It propagates the misunderstanding(s) that Wu & Estabrook (2016) have tried to clear up.  Namely, as I stated above: without equal thresholds, the latent item responses are not on linked scales, so loadings are not comparable.  The article you link to talks about simultaneously constraining thresholds and loadings (and intercepts, by virtue of not freeing them: another misunderstanding they propagate), which is fine, it just provides a single omnibus test of scalar invariance relative to configural invariance.  In comparison, they free loadings first, then thresholds, which is problematic for the reason in bold above.

[Juan Diego Hernández Lalinde]

Dear Professor Terrence,

I hope you are very well.

I have some questions regarding comparison of non-nested models obtained by CFA:

Q1: Is it appropriate to compare (in a descriptive way) fit indexes like CFI, TLI, GFI, AGFI, RMSEA, SRMR?
Q2: Is it appropriate to compare (in a descriptive way) indexes like AIC and BIC?
Q3: What about the Vuong’s test? Do you recommend it?
Q4: Do you recommend any tutorial or papers to read about non-nested models comparison?
Q5: Do you recommend any package in R to make comparison of non-nested models?

Q6: If Vuong's test is appropriate, do you recommend any package in R to perform it?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/af804524-8229-4bff-b6f1-74bfa88f6c9do%40googlegroups.com.

[Terrence Jorgensen]

I have some questions regarding comparison of non-nested models obtained by CFA:

We were discussing measurement-invariance models above, which should be nested (otherwise, something is misspecified).  But perhaps you are not talking about testing invariance.

 

Q1: Is it appropriate to compare (in a descriptive way) fit indexes like CFI, TLI, GFI, AGFI, RMSEA, SRMR?

Not the way it is described in 99.9% of the SEM literature.  Most fit indices were not designed for model comparison, and the "change in" a fit index has never stood up to even minor scrutiny.

RMSEA can be calculated using the chi-squared (and df) difference, which was called the root deterioration per restriction (RDR) in the long-long-ago literature.  It was abandoned early on because its sampling distribution tends to be quite erratic, just like RMSEA's for any model with few df or small N. 

Incremental fit indices (CFI, TLI) are a notable exception.  If you read the source proposing them (link), they can be used to compare the fit of any set of nested or nonnested models, on the condition that the null model (baseline.model= argument in the fitIndices() function) used to calculate them is nested within each of the hypothesized models being compared (all of which, by definition, are nested within the same saturated model).

 

Q2: Is it appropriate to compare (in a descriptive way) indexes like AIC and BIC?

Information criteria were developed specifically to aid model comparison, but they are only available using a likelihood-based estimator, not least-squares estimators.  So in the latter case, only incremental fit indices might be an option.

Q3: What about the Vuong’s test? Do you recommend it?

It shows promise

Q4: Do you recommend any tutorial or papers to read about non-nested models comparison?

http://www.quantpsy.org/pubs/merkle_you_preacher_2016.pdf 

 

Q5: Do you recommend any package in R to make comparison of non-nested models? 

Q6: If Vuong's test is appropriate, do you recommend any package in R to perform it?

nonnest2 (see above article)

[Juan Diego Hernández Lalinde]

Thanks a lot, Professor.

I had already read about the nonnest2 package, but it only provides the Vuong's test under maximum likelihood estimator. As I understand, there is no alternative to perform this test when a robust maximum likelihood estimator has been used (e.g. MLM or MLR). Am I right?. Is there any way to perform Vuong's test to compare non-nested models when MLM or MLR has been used to fit them?

Thanks again. Best regards.

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/c3050c44-0a73-4d11-962c-634db878cc53n%40googlegroups.com.

[Terrence Jorgensen]

Is there any way to perform Vuong's test to compare non-nested models when MLM or MLR has been used to fit them?

Not that I know of. 

[Juan Diego Hernández Lalinde]

Thanks, Professor.

Have a nice day. 

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/c36aebda-927e-4673-a552-871ce648e108n%40googlegroups.com.