Examining demand effects

14 views
Skip to first unread message

Chris Castille

unread,
Jun 9, 2020, 8:39:32 PM6/9/20
to lavaan
Hi there, 

Suppose that I have a set of two 4-item measures for x and y construct, respectively. I'm interested in examining (i) how the correlation between x and y vary across certain measurement conditions and (ii) whether the factor loadings vary across conditions. In this case, there are four measurement conditions: (1) control (i.e., all items are administered in the same order), (2) item order is randomized across the sample, (3) scale order is randomized across the sample (i.e., sometimes x is first; sometimes y is first), and (4) item order and scale order are both randomized (so a mix of conditions 2 and 3). 

Unfortunately, my sample sizes are low and so I'm looking into whether I can combine multiple datasets to boost my power to detect any effects. I think that dummy coding could help me to examine these effects (I may be mistaken). I could dummy code my participants as:
1. Control vs. Any treatment
2. Control vs. Item Randomization
3. Control vs. Scale Randomization
4. Control vs. Item and Scale Randomization (Bundle)

Assuming that I'm right, what would the code look like for examining the moderating effects of these conditions on (i) the x–y correlation and (ii) the factor loadings?

Chris

Terrence Jorgensen

unread,
Jun 12, 2020, 5:24:21 AM6/12/20
to lavaan
You could start with a constrained model (scalar invariance if you plan to test hypotheses about constructs using CFA, or strict invariance if you plan to do so using scale composites) to see if the model has no significant misfit.  But lack of power could also be the reason for a nonsignificant chi-squared.  A more powerful method is to use a single-group approach like RFA / MIMIC models.  See a description and example syntax in the top 2 articles:


The BRM article is open access, the IMPS chapter is accessible on ResearchGate.

Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Reply all
Reply to author
Forward
0 new messages