Saving ID Variable along predict()ed values

1,514 views
Skip to first unread message

Marcus

unread,
Aug 13, 2014, 5:42:19 AM8/13/14
to lav...@googlegroups.com
Hi there,

I am currently struggling a bit with predict()ing latent factor scores from a fitted lavaan-model. I know that these are not "real" factor scores and that predicting them is not generally recommended, but let's just say I need to do it.

The model fitting and predict()ing actually works fine, except that lavaan removes 3,000 participants with no data at all out of 12,000 available. The problem is that I can't figure out which exact participants are excluded (na.omit() on the original data doesn't reproduce the N used by lavaan). As a consequence, I cannot attach the predicted values to the original data frame. Thus, I have two related questions:

1. Is it possible to figure out which individuals were excluded? lavaan warns me that some had no data but the output is truncated because of the sheer mass.

2. Is it possible to provide predict() with an id-variable so that this id is saved along the predicted values?

All the best,

Marcus

Sunthud Pornprasertmanit

unread,
Aug 13, 2014, 9:34:03 AM8/13/14
to lav...@googlegroups.com
Here is a dirty trick that you may use for matching the id variable. I just simply make a new model that includes the id variable and all observed variables used in the original model. 

newdata <- HolzingerSwineford1939

newdata[seq(1, 301, 3), -1] <- NA


mod1 <- 'f1 =~ x1 + x2 + x3'

fit1 <- cfa(mod1, newdata)

pred1 <- predict(fit1)


mod2 <- 'id ~~ x1 + x2 + x3

'

fit2 <- cfa(mod2, newdata)

dat2 <- fit2@Data@X[[1]]

pred1 <- data.frame(pred1, id = dat2[,1])



--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at http://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

Dr. Hans Hansen

unread,
Aug 18, 2014, 8:42:41 AM8/18/14
to lav...@googlegroups.com

> I know that these are not "real" factor scores and that predicting them is not generally recommended

Why is that?

Dr. Hans Hansen

unread,
Aug 18, 2014, 10:08:44 AM8/18/14
to lav...@googlegroups.com

Ok, i found the posts discussing factor indetermancy... but i don't understand the consequences. are predicted scores useless? or not exact? or biased?

thanks a lot!

yrosseel

unread,
Aug 18, 2014, 12:06:43 PM8/18/14
to lav...@googlegroups.com
On 08/13/2014 11:42 AM, Marcus wrote:
> 1. Is it possible to figure out which individuals were excluded? lavaan
> warns me that some had no data but the output is truncated because of
> the sheer mass.

Officially, this is not possible. Unofficially, you can (currently,
lavaan 0.5-16) get the 'used' case indices from the slot:

fit@Data@case.idx[[1]]

if 'fit' is the name of your fitted lavaan object. The [[1]] at the end
if for the first group.

Yves.

Edward Rigdon

unread,
Aug 18, 2014, 2:29:04 PM8/18/14
to lav...@googlegroups.com

Hans—

     Factor indeterminacy means that the common factor includes two parts: (1) a weighted function of all observed variables in the model, and (2) an arbitrary vector—call it S.  The pseudo factor scores that programs, including lavaan, give you are just the first part, omitting S.  So the fundamental problem with using these scores to represent a common factor is that they are not equal to the common factor.  So having gone to all this trouble, you have failed to achieve your objective.

     Note that S is “arbitrary,” not “random.”  If S were “random,” as in “random measurement error,” then you could easily adjust results from using pseudo factor scores to the results that you would have obtained from using actual factor scores.  And you would know that the correlation between pseudo factor scores and some outside variable would overstate the correlation between the factor and the outside variable, by a degree based on the extent of indeterminacy.  In other words, you could fix the problem.

     But S is “arbitrary.”  The only rule for S is that the S for each common factor must be orthogonal to every other S and orthogonal to every observed variable in the factor model.  Now, take your external variable, that you want to predict using these pseudo factor scores.  That variable is NOT in the factor model.  So we could construct an S using that criterion variable itself as our raw material.  First, we predict your criterion using the observed variables.  Then we take the residual, adjust it to get the proper variance, and we call that S.  Using that S would mean that the factor would have a HIGHER correlation with the criterion variable.  Or we multiply that by -1, and use that as S.  In this case, the factor could have a substantially LOWER correlation with the criterion.

     Let me emphasize that, if we construct S correctly, then both of these sets of actual factor scores would be perfectly consistent with the factor model.  They would BOTH be correct, in every way.  This uncertainty is the price of specifying factor models with a substantial degree of indeterminacy.  Researchers could keep indeterminacy low by specifying models with many strong indicators per common factor.  But that would magnify statistical power, leading to poor fit statistics, and it would increase the cost of data collection.

--Ed Rigdon

--

Dr. Hans Hansen

unread,
Aug 20, 2014, 1:19:39 PM8/20/14
to lav...@googlegroups.com
thanks for the explaination! some questions:

  1. it seems that hence it would be a sensible thing to do to include the criterion-variable in the SEM? i would think that then the common factor is really used in the estimation!
  2. is there any rule of thumb what "many strong indicators" could be?

thanks again!

 

Edward Rigdon

unread,
Aug 20, 2014, 2:05:03 PM8/20/14
to lav...@googlegroups.com

Hans—

     Yes, it makes a great deal of sense to include the criterion variable in the original factor model, rather than define the factor in one model and predict in another.

     You can easily calculate factor indeterminacy using model parameter values.  By varying those terms, you can see how changes affect the result.  I favor Louis Guttman’s (1955) index.  Guttman’s rho is equal to the minimal correlation between two different sets of factor scores for the same common factor.  This index ranges from -1 to +1, and is calculated as:

 

Guttman’s rho = [2 x R2(F,Y)] - 1

 

where R2(F,Y) is R square for the common factor when predicted by the observed variables.  R2(F,Y) is the “determinacy” number given by most SEM packages.

 

You can also calculate R2(F,Y) yourself as:

 

Diag(P x t(L) x inv(Sigma) x L x P)

 

Where:

P is factor covariance matrix

L is the loading matrix

Sigma is the observed variable covariance matrix

Diag means diagonal elements

t() means transpose

inv() means inverse

 

You will also find that Guttman’s rho tracks closely with composite reliability, especially for models with just one factor.  Indicators per factor, number of factors, strength of loadings and strength of factor correlations all affect rho, to some extent.

 

     How much indeterminacy can you stand?  The field is far from acknowledging the need to worry about this question.  If you took 0.80 (a common default minimum for statistical power), then in a model with, say several correlated factors, you might achieve rho = +0.80 with 3 indicators per factor with standardized loadings all at 0.90.  If standardized loadings are only 0.80, you will need 6 indicators per common factor.  Eitehr way, that’s more than you are going to see in the typical published application.

--Ed Rigdon

 

Guttman, L. (1955). The determinacy of factor score matrices with implications for five other basic problems of common-factor theory. The British Journal of Statistical Psychology, 8, 65-81.

 

 

From: lav...@googlegroups.com [mailto:lav...@googlegroups.com] On Behalf Of Dr. Hans Hansen
Sent: Wednesday, August 20, 2014 1:20 PM
To: lav...@googlegroups.com
Subject: Re: Saving ID Variable along predict()ed values

 

thanks for the explaination! some questions:

--

Edward Rigdon

unread,
Aug 20, 2014, 5:09:19 PM8/20/14
to <lavaan@googlegroups.com>
I'm sorry--I made a mistake.  I wrote that Guttman's rho tracks along with composite reliability.  I should have written that R2(F,Y), the R2 for the common factor as predicted by the observed variables, tracks along with composite reliability.  So Guttman's rho will tend to me smaller / more negative.

--Ed Rigdon

Sent from my iPad

Dr. Hans Hansen

unread,
Aug 24, 2014, 4:38:07 PM8/24/14
to lav...@googlegroups.com
great, thanks!

Marcus

unread,
Aug 25, 2014, 9:07:26 AM8/25/14
to lav...@googlegroups.com
Thank you all for your help, suggestions, and additional inputs. Yves' solution works fine -- there's nothing more helpful than a responsive developer (except an exceptionally kind community at his side :) ).

Just to make it complete: Sunthud's approach, which I tried myself in a similar way, does not work because the ID variable has a completely different "scale" than all other variables. lavaan also mentions this and does not fit the model.

Yves, are there plan to make your approach "official", or what are the reasons to not have the ids accessible, respectively?

Michael Paul Grosz

unread,
Aug 25, 2014, 3:51:22 PM8/25/14
to lav...@googlegroups.com
Dear Edward Ridgon,

Thank you for your elaborated comments.
I noticed myself that the correlations between predicted variables ("biased factor scores" for several variables) are not of equal in size to the correlations between the same variables when they are part of the same SEM model. And I think you delivered the explanation for this phenomenon. I am wondering though why SEM software like lavaan does not base their predict function on both the weighted function of observed variables and S. It would be really handy to get predicted factor scores that estimate the standing of people on the latent trait of interest especially because including many latent traits and external criteria into SEM models often causes computational difficulties.

best,
Michael

Edward Rigdon

unread,
Aug 25, 2014, 6:24:32 PM8/25/14
to lav...@googlegroups.com
Michael--
     Remember that each S is arbitrary.  Where factors are indeterminate, there is no one answer.  Any one set of actual factor scores is just one of the infinity of possible sets, none of them any more correct than any of the others.  It is not hard to create your own S for each common factor, if that males things better.
    This limitation of the factor model would not seem so strange if the literature had not spent decades suppressing this topic--telling users that "fit" is all they needed, which in turn leads users to specify factor models with higher indeterminacy.  You can minimize the indeterminacy problem by specifying factor models with more and better indicators, though that makes it even more challenging to obtain good fit. Factor models with few and only moderately strong indicators per factor, which are more likely to pass fit tests, are going to have substantial indeterminacy.  There is no free lunch.
--Ed Rigdon

Sent from my iPad

Michael Paul Grosz

unread,
Aug 26, 2014, 7:13:39 AM8/26/14
to lav...@googlegroups.com
Dear Ed Ridgon,

Thanks for the response.

Ok, so, because the S for a latent trait is arbitrary and there is no one answer, I am asking myself: how does SEM software like lavaan determine the S for a latent trait in the SEM model?
Does lavaan pick a S that increases the correlation between the variables (latent traits) in the SEM model? (in this case, other variables in the model would serve as additional indicators)
But you said that the rule for S is that it is orthogonal to any other S in the model and orthogonal to any observed variable in the model. So, I guess this rule is in place to make sure that the correlations between variables in the SEM model do not get under- or overestimated. Is that the reason why we should have both variables in the model?

Do you think it is ok to use the predicted factor scores when there are many decent indicators in the model on which the predictions are based (because then the degree of indeterminacy would be low). So, do you think it would be ok to use the "pseudo factor scores" when I have say 10 indicators with standardized loadings of .5 on average?
On the other hand, if we have only few or weak indicators of a construct, then it would be better to compute correlations/regressions etc. within the SEM model instead relying on the "pseudo factor scores", right?

A potentially related question: Could it be that the S of the latent trait A also influences the factor loadings of the indicators (items) of A. In my experience, an items factor loading on the latent trait A is slightly different in a SEM model with only the latent trait A in the model compared to a SEM model in which there is not only A with the indicators of A but also a second latent trait B and the indicators of B in the SEM model (at least that's the case when A and B are correlated). Why are the factor loadings on one factor not independent of the presence or absence of another factor and its indicators in the SEM model? could the orthogonality of the S of A and the S of B be the reason?

Michael

Edward Rigdon

unread,
Aug 26, 2014, 8:59:16 AM8/26/14
to lav...@googlegroups.com
     Factor models are great for estimating factor model parameters.  Think of a factor model as a pocket universe with its own set of rules.  Indeterminacy becomes an issue when you try to go outside the model.  That is a contrast with composite-based models, where a weighted sum is a weighted sum is a weighted sum.  If you want to use a common factor as a predictor, you should include the dependent variable within the factor model.
     Loadings of 0.5 are small.  Ten indicators would not be enough.  Everything is a matter of degree, but there is certainly some point where indeterminacy is small enough to be ignored.
     Introducing another common factor, or even just more indicators, can certainly change factor model outcomes.  A model is only fit within the given set of conditions.  Whenever new variables are included in a model, the "best" solution can certainly change.   Evidence from a factor model is conditional on the model.
--Ed Rigdon

Sent from my iPad

Michael Paul Grosz

unread,
Aug 27, 2014, 6:21:32 AM8/27/14
to lav...@googlegroups.com
Many thanks for your response!
Michael

yrosseel

unread,
Sep 4, 2014, 1:45:30 PM9/4/14
to lav...@googlegroups.com
On 08/25/2014 03:07 PM, Marcus wrote:
> Yves, are there plan to make your approach "official", or what are the
> reasons to not have the ids accessible, respectively?

In dev lavaan 0.5-17.710 (or higher), you can now use

inspect(fit, "case.idx")

to extract the case id's.

Yves.

Reply all
Reply to author
Forward
0 new messages