Multilevel modelling and ICC

[Nic...@gmail.com]

I want to know if I should use multilevel modelling for my model.
Therefore, I want to calculate the intraclass correlation (ICC) in
SPSS. Using the instructions on http://davidakenny.net/dyad.htm I've
got most of the job done. But I would like to ask a couple of questions
concerning some of the details, especially since the instructions on
http://davidakenny.net/dyad.htm are for dyads and in my model there are
more than two persons in each group. Below I have typed what I have
done step-by-step. Hope you can help.

My model
*Dependent variables: dichotomous; first-level variable
*1 independent group-variable: categorical (4 groups); first-level
variable; is an antecedent of the dependent variable
*4 independent variables: continuous; second -level variables; all are
antecedents of the dependent variable

I want to test whether the continuous variables have different
parameters for different groups. Because then I will need to use
hierarchical linear modeling. To this end, I want to calculate the ICC
using SPSS.

STEPS:
1. MEMBER = ID of person

2. GROUP = category that person is in. There are more than two persons
in each group.

** Doesn't it for the analysis matter that the GROUP-ID is also an
antecedent of Yi (i.e., there is a direct relationship from GROUP-ID to
Yi)? **

3. XPARi = the i independent continuous variables

4. Yi = the dichotomous dependent variable

** Is this analysis still valid for a dichotomous dependent variable?
**

5. Sort data by GROUP

6. ANALYSIS > MIXED MODELS > LINEAR
Type in group id in SUBJECTS
Type in the MEMBER variable name in REPEATED MEASURES if members are
indistinguishable or the distinguishing identifier (e.g., GENDER) if
members are distinguishable. (in my case: indistinguishable)
Pick COMPOUND SYMMETRY is members are indistinguishable and COMPOUND
SYMMETRY HETEREOGENOUS if people are distinguishable on the repeated
measures variable
CONTINUE

7. In LINEAR MIXED MODELS:
Type in the name of the DEPENDENT VARIABLE
Type categorical variables in FACTOR(S)

** Should I now also include the Group id as a FACTOR, because GROUP is
also an direct antecedent of the dependent variables? **

Type continuous variables in COVARIATE(S); include actor or own X as a
predictor and partner's X as predictors.

** I enter the four continuous antecedents as COVARIATES. I have no
variable for "partner's X" because I don't work with dyads
while the instructions are developed for when you have dyads. Is this a
problem? **

8. FIXED button
Add in relevant terms. Include relevant actor and partner effects.

** I added none / changes nothing in this screen. Should I have added
variables given my model? **

Pay close attention to the term in the box in the middle. This states
"Factorial".
Ordinarily make sure "INCLUDE INTERCEPT" box is checked.

9. RANDOM button
On the bottom of "SUBJECT GROUPINGS," move Group id from SUBJECTS into
COMBINATIONS.
Do not click "INCLUDE INTERCEPT"

10. STATISTICS button
Click PARAMETER ESTIMATES
Click TESTS FOR COVARIANCE PARAMETERS
Can ask for DESCRIPTIVE STATISTICS and CASE PROCESSING SUMMARY

11. I click OK and get the message: "A model with only a fixed
Intercept will be used since no custom model has been specified. You
can specify a custom model by clicking on the Fixed and/or the Randoms
buttons." ** Did I do something wrong? **

12. The syntax that I get is as follows:
MIXED
Y1 WITH XPAR1 XPAR2 XPAR3 XPAR4
/CRITERIA = CIN(95) MXITER(100) MXSTEP(5) SCORING(1)
SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)
PCONVERGE(0.000001, ABSOLUTE)
/FIXED = | SSTYPE(3)
/METHOD = REML
/PRINT = CPS DESCRIPTIVES SOLUTION TESTCOV
/REPEATED = MEMBER | SUBJECT(GROUP) COVTYPE(CS) .

13. Subsequently, I calculate the ICC from the SPSS output table:
"Estimates of Covariance Parameters".

[Nic...@gmail.com]

Does this mean that besides the dependent variables, I should enter the
following variables:
Fixed factors: Group-ID since Group-ID is not only defines subject but
is also an antecedent of the dependent variable
Fixed covariates: The four continuous independent variables
Random covariates: None, because if no variables are allowed to change
over the groups, the ICC that we calculate gives an indication whether
it would be better or not to let the varibles differ over the groups.

Would you agree that this is the way to go?
I have my doubts because I get the warning "The final Hessian matrix is
not positive definite although all convergence criteria are satisfied.
The MIXED procedure continues despite this warning. Validity of
subsequent results cannot be ascertained."

tclark wrote:
> you need to go back and add those predictors since that is what you
> hypothesized would be related to your outcome variable. these "fixed"
> set of predictors are what you expect to contribute significantly to
> the variance in your outcome variable. adding these predictors will
> allow you to calculate the partial ICC which tells you the proportion
> of variance in your outcome variable while controlling/accounting for
> the predictors. you can choose not to include the predictors in order
> to calculate your ICC to estimate group members level of similarity in
> general, without considering any predictors (test of the null model).
> however, refer to the handouts to appropriately address how to handle
> group data.
>
> Jay Weedon wrote:
> > On 9 Oct 2006 14:06:44 -0700, "Nic...@gmail.com" <Nic...@gmail.com>

> > I won't address the software questions since I don't use SPSS much. I
> > would suggest that trying to fit a linear mixed model when you have a
> > dichotomous dependent variable is a dodgy maneuver at best - you
> > should be thinking about generalized mixed linear models (I don't know
> > that SPSS supports these). And I don't really grasp why you need to
> > calculate the ICC (I'm not entirely sure that the concept of ICC even
> > makes sense with a dichotomous dependent variable) - if you want to
> > know whether there exists significant clustering, estimate the
> > variance term for the clustering effect and test whether it's
> > significantly different from zero.
> > JW

[Nic...@gmail.com]

[tclark]

enter id in random

[Nic...@gmail.com]

Thanks, it also seems that the four continuous independent variables
need to be entered as Random covariates as well as Fixed covariates.
Otherwise, the table "Estimates of Covariance Parameters" has only one
row with the Residual.
Is this correct?

[tclark]

to calculate the icc you need both values. are you sure you have
followed all of the steps in the handouts? try contacting tabachnick or
ullman to be absolutely sure. tabachnick's book "Using Multivariate
Statistics" has an entire chapter with output on multilevel models.

[Nic...@gmail.com]

I have finally been able to obtain a copy of the chapter on Multilevel
Linear Modeling in the book by Tabachnick. Initially I though, I could
use the ICC to determine whether or not to use multilevel linear
modelling but it turns out that you can only draw conclusions about the
intercept on basis of the ICC and not about whether or not there are
differences in slopes between groups.
My conclusion what I need to do:
- calculate the intraclass correlation of the intercept-only model to
determine whether the intercept differs between groups.
- calculate the entire multilevel model to determine whether the slopes
differ between groups.

Problem is however, that I need a multivariate multilevel model. That
is I have multiple dependent variables. In addition, all dependent
variable are dichotomous. Hence, I need a Bernoulli multivariate
multilevel model. Such a model cannot be estimated with SPSS. The
program HLM can estimate this model but I was hoping I could first test
whether I need multilevel modeling before I would begin to learn HLM.
But now it seems you can only test whether you need multilevel modeling
by building such a model.