PLUM: Ordinal regression
Bruce Weaver
I've been trying to figure out a little bit about PLUM, the SPSS
procedure for ordinal regression. Here's what (I think) I've figured
out so far.
It seems that ordinal logistic regression (OLR), or PLUM, simultaneously
produces a series of models. THe number of models is k-1, where k is the
number of levels of the ordinal outcome variable. So if I have k=3
levels, it produces 2 models. The models make the following comparisons
(in terms of the outcome variable):
Model 1: 1 v [2,3]
Model 2: [1,2] v 3
In other words, the first model in effect recodes the outcome with 3
levels to a binary outcome with level 1 versus 2 and 3 pooled; and the
2nd model recodes the 3-level variable to a binary variable with 1 and 2
pooled versus 3.
The output from PLUM gives only one set of coefficients for each
predictor variable though. So, it seems to me that the coefficients it
produces must be some kind of (weighted?) average of the coefficients
from each of these k-1 models. And the coefficients are to be interpred
as the log-odds of being in a higher category.
There is an assumption of homogeneity of these coefficients across the
k-1 models, not unlike the assumption of homogenity of covariance in
ANCOVA (I guess).
Does this sound remotely accurate to those of you who understand ordinal
logistic regression better than I do?
I've appended a little syntax file in which I try to demonstrate this to
myself using the 1991 General Social Survey data file that ships with
SPSS. It contains links to a couple of sites I found with info on OLR
or PLUM.
Cheers,
Bruce
* ===================================================================
File name: plum 001.SPS
Written by: Bruce Weaver, bwe...@lakeheadu.ca
Date: 08-Mar-2005
Notes: Try to understand output from PLUM procedure.
* =================================================================== .
* Path macros for data files & syntax files.
define !dpath ()'C:\Program Files\SPSS\'!enddefine.
GET FILE= !dpath + '1991 U.S. General Social Survey.sav'.
freq happy race.
* Perform Ordinal Logistic Regression with HAPPY as outcome variable.
PLUM
happy BY race
/CRITERIA = CIN(95) DELTA(0) LCONVERGE(0) MXITER(100) MXSTEP(5)
PCONVERGE(1.0E-6) SINGULAR(1.0E-8)
/LINK = LOGIT
/PRINT = FIT PARAMETER SUMMARY TPARALLEL .
* A document I found says OLR simultaneously estimates multiple equations.
* The number of equations is the number of levels of the DV minus 1.
* For this example, the DV (HAPPY) has 3 levels, so 2 equations are
estimated.
* They make the following contrasts (on the DV):
* Equation 1: 1 v [2,3]
* Equation 2: [1,2] v 3.
* URL for document: http://staff.washington.edu/~glynn/olr.pdf .
* So, can I run two separate binarly logistic regressions that make these
* contrasts, and average the coefficients to approximate the OLR
coefficients?
recode happy
(1=1) (2 3 = 2) (else = 9) into happy2.
recode happy
(1 2 = 1) (3=2) (else = 9) into happy3.
exe.
format happy2 happy3 (f1.0).
val lab
happy2 1 '1: very happy'
2 '2-3: pretty happy/not too happy' /
happy3 1 '1-2: very or pretty happy'
2 '3: not too happy'.
miss val happy2 happy3 (9).
crosstabs
/table = happy by happy2
/table = happy by happy3 .
LOGISTIC REGRESSION happy2
/METHOD = ENTER race
/CONTRAST (race)=Indicator
/PRINT = CI(95)
/CRITERIA = PIN(.05) POUT(.10) ITERATE(20) CUT(.5) .
LOGISTIC REGRESSION happy3
/METHOD = ENTER race
/CONTRAST (race)=Indicator
/PRINT = CI(95)
/CRITERIA = PIN(.05) POUT(.10) ITERATE(20) CUT(.5) .
* If I understand what I read in the notes mentioned above,
* the parameters for RACE=1 and RACE=2 in the PLUM output
* should be roughly equal to the means of those same coefficients
* from the two binary logistic regression analyses.
compute MEAN1 = mean(-.342, -.835).
compute mean2 = MEAN( .144, .016).
exe.
descr mean1 mean2.
* These values are not too far from the PLUM estimates,
* which were -.534 and 0.100.
* The document cited above says that OLR (or PLUM) coefficients may
* be interpreted in the same way as logistic regression coefficients.
* However, rather than having only 1 transition estimated, there are
* k-1 transitions (where k = number of levels of the outcome). So
* EXP(B) gives the odds ratio associated with being in a higher category.
* Here's another website with potentially useful info.
* http://www.xs4all.nl/~jhckx/spss/mlogist/plum.html .
* One more approach: Try pair of binary logistic regressions
* that compare 1 v 2, and 2 v 3.
recode happy
(1=1) (2=2) (else=9) into happy12.
recode happy
(3=3) (2=2) (else=9) into happy23.
exe.
format happy12 happy23 (f1.0).
miss val happy12 happy23 (9).
val lab happy12 happy23
1 '1- Very happy'
2 '2- Pretty happy'
3 '3- Not too happy'.
LOGISTIC REGRESSION happy12
/METHOD = ENTER race
/CONTRAST (race)=Indicator
/PRINT = CI(95)
/CRITERIA = PIN(.05) POUT(.10) ITERATE(20) CUT(.5) .
LOGISTIC REGRESSION happy23
/METHOD = ENTER race
/CONTRAST (race)=Indicator
/PRINT = CI(95)
/CRITERIA = PIN(.05) POUT(.10) ITERATE(20) CUT(.5) .
compute MEAN1 = mean(-.194, -.770).
compute mean2 = MEAN( .152, -.029).
exe.
descr mean1 mean2.
* These values are not too far from the PLUM estimates,
* which were -.534 and 0.100.
* =================================================================== .
--
Bruce Weaver
bwe...@lakeheadu.ca
www.angelfire.com/wv/bwhomedir
Michael....@colostate.edu
> One of my colleagues has some data with an ordinal outcome variable, so
> I've been trying to figure out a little bit about PLUM, the SPSS
> procedure for ordinal regression. Here's what (I think) I've figured
> out so far.
> It seems that ordinal logistic regression (OLR), or PLUM, simultaneously
> produces a series of models. THe number of models is k-1, where k is the
Hi Bruce,
Not, not really a series of models, just one:
For an ordinal response variable Y, it estimates the parameters
T_j and B of:
[Prob(Y<=j)]/[1-Prob(Y<=j)] = T_j -xB
Where T_j is a "threshold" or constant associated with the
jth category, x is a vector of covariates, and B is a vector
of slopes. Note the somewhat tricky - sign on xB. This
makes things come out so that positive Bs mean positive
effects in the ordinary sense, i.e., increasing values
of a covariate are associated with increasing values of Y.
As I found out several years ago when I asked SPSS tech support
about PLUM, it is a pretty darn faithful implementation of the
ordinal regression model presented in:
McCullaugh, P. 1980. "Regression Models for Ordinal Data (with
discussion)." J. Royal Stat. Society 42:pp. 109-142.
This article is surprisingly readable; even mortals can extract
something of use from it. However, what I consider the most
useful source (relatively deep but comprehensible) is:
Long, Scott. 1997. Regression Models for Categorical and Limited
Dependent Variables. Sage Pub.
see esp. Ch. 5 on ordinal models. This book is a great buy;
I'm just a happy customer, though, with nothing to gain.
> number of levels of the ordinal outcome variable. So if I have k=3
> levels, it produces 2 models. The models make the following comparisons
> (in terms of the outcome variable):
> The output from PLUM gives only one set of coefficients for each
> predictor variable though. So, it seems to me that the coefficients it
> produces must be some kind of (weighted?) average of the coefficients
> from each of these k-1 models. And the coefficients are to be interpred
> as the log-odds of being in a higher category.
Yes, there is only slope for all the categories. This is why the
ordinal reg. model (the standard one, anyway) is sometimes called
a parallel lines model. It (testably) imposes the restriction that
the effect of any X is the same in "moving" from partitition 1,2
up to 3+, for partition 1,2,3 up to 4+, etc. SPSS gives a likelihood
ratio test of this feature of the model, which is essentially
a comparison of an ordinal regression model, with one slope per
X, vs. a multinomial regression model, with different slopes for
each X for each response category. (The multinomial model really
is like several different binary models.)
> There is an assumption of homogeneity of these coefficients across the
> k-1 models, not unlike the assumption of homogenity of covariance in
Yup, you got it.
By the way: Although the parallel lines test often shows that
the ordinal regression model can be rejected, don't take it too
seriously. As you might imagine, it is quite sensitive to sample
size, and relatively trivial differences in fit (as indexed by
any of the pseudo-R^2) will trigger a small p-value for the
parallel lines test.
SPSS's PLUM model, I must say, is actually one of the nicer
ordinal regression packages out there, as it offers several
different link functions (not just the logit), and because
it offers the possibility to model the dispersion of Y as
well as the expected value. McCullaugh slightly mentions
this last thing, but I don't know of any other programs that
offers this feature, not even any of the user-written stuff
for Stata. But that's just as well, since I don't really
claim to understand it ... <grin> .
Regards,
--
=-=-=-=-=-=-=-=-=-==-=-=-=
Mike Lacy, Ft Collins CO 80523
Clean out the 'junk' to email me.
Bruce Weaver
Cheers,
Bruce
Michael....@colostate.edu wrote:
--
Bruce Weaver
bwe...@lakeheadu.ca
www.angelfire.com/wv/bwhomedir