r-Squared Question
Predictor
of regression lines. If r, which is squared to obtain r-squared, is
the correlation between the predicted Y and the observed Y, then
doesn't that mean that any regression line whose predicted Y is a
perfect linear function of the observed Y has an r (and thus r-squared)
of 1?
Thanks,
Will
Keywords: r2 r^2 r**2 r-square r-squared rsquare rsquared
Jerry Dallal
> I am trying to undertand r-squared (the coefficient of determination)
> of regression lines. If r, which is squared to obtain r-squared, is
> the correlation between the predicted Y and the observed Y, then
> doesn't that mean that any regression line whose predicted Y is a
> perfect linear function of the observed Y has an r (and thus r-squared)
> of 1?
>
>
> Thanks,
> Will
A line doesn't have an 'r'. Variables do. For a linear functional
relationship, r(observed, predicted)=1; r(X,Y) = 1 or -1.
Radford Neal
Predictor <pred...@bellatlantic.net> wrote:
>I am trying to undertand r-squared (the coefficient of determination)
>of regression lines. If r, which is squared to obtain r-squared, is
>the correlation between the predicted Y and the observed Y, then
>doesn't that mean that any regression line whose predicted Y is a
>perfect linear function of the observed Y has an r (and thus r-squared)
>of 1?
That's true. You may be a bit confused, however. The only way that
the predicted and observed Y can be related by a linear function is if
the predicted and observed Y are identical (ie, the linear function is
observed=predicted). You may be confusing "predicted value" with
"predictor" (also known as a covariate or explanatory variable). The
predicted value is the linear function of the covariates in which the
coefficients are those found by fitting to the data.
----------------------------------------------------------------------------
Radford M. Neal rad...@cs.utoronto.ca
Dept. of Statistics and Dept. of Computer Science rad...@utstat.utoronto.ca
University of Toronto http://www.cs.utoronto.ca/~radford
----------------------------------------------------------------------------
Predictor
clearer:
X Y
1 101
2 102
3 103
4 104
5 105
6 106
7 107
8 108
9 109
10 110
The relationship here is obvious, but bare with me. Assume that some
regression procedure (obviously not least squares) produces a linear
model, YHat:
X Y YHat
1 101 97
2 102 99
3 103 101
4 104 103
5 105 105
6 106 107
7 107 109
8 108 111
9 109 113
10 110 115
YHat has a correlation ( r ) with Y of 1.0. r-squared is hence 1.0.
What I'm getting at is: the r-squared is at its best possible value,
yet the model is obviously suboptimal. Have I gone wrong somewhere, or
is this a fundamental weakness of r-squared?
Thanks very much,
Will
Radford Neal
Predictor <pred...@bellatlantic.net> wrote:
I think people using r-squared commonly assume that the model is
fitted by least squares, and that the model is at least as general as
a linear regression with intercept, which guarantees that there is no
linear component left in the residuals. There is a perennial question
asked in this newsgroup about the meaning of r-squared when the model
has no intercept term, the usual conclusion being that quoting
r-squared is bound to be misleading in such a situation.
More generally, you might just want to look at the standard deviation
of the residuals. Of course, for a linear model fitted by least
squares, r-square equals 1 - s^2/v^2, where s is the standard
deviation of the residuals, and v is the standard deviation of the
response. On the whole, s is more meaningful than r-squared, because
it's expressed in meaningful units, and unlike r-squared it doesn't
vary with the scheme for sampling the data, assuming the sampling
isn't related to the response (and assuming the model is correct, with
there being no heteroskedasticity). It's also meaningful regardless
of how you fitted the model.
Eric Bohlman
news:1121197200.6...@g47g2000cwa.googlegroups.com:
> The relationship here is obvious, but bare with me. Assume that some
> regression procedure (obviously not least squares) produces a linear
> model, YHat:
>
> X Y YHat
> 1 101 97
> 2 102 99
> 3 103 101
> 4 104 103
> 5 105 105
> 6 106 107
> 7 107 109
> 8 108 111
> 9 109 113
> 10 110 115
>
> YHat has a correlation ( r ) with Y of 1.0. r-squared is hence 1.0.
> What I'm getting at is: the r-squared is at its best possible value,
> yet the model is obviously suboptimal. Have I gone wrong somewhere,
> or is this a fundamental weakness of r-squared?
In your (very unlikely in the real world) example, observed Y is an
*exact* linear function of X (Y=X+100). So is estimated Y (YHat=2X+95),
by definition. Correlation is invariant under linear transformations
(changes of location and/or scale), so the correlation between two linear
functions of X is exactly the same as the correlation of X with itself,
namely 1.0. Thus merely maximizing r-squared without any constraints
doesn't lead to an optimal model or even a unique solution; any linear
transformation of the estimates will have the same r-squared. Actual
regression methods involve maximizing r-squared subject to certain
constraints; the methods differ in the choice of constraints.
kennet...@sbcglobal.net
"Predictor" <pred...@bellatlantic.net> wrote in message
news:1121175957.7...@g49g2000cwa.googlegroups.com...
Jerry Dallal
It depends how you defined R^2. If you define it as the square of the
correlation between observed and predicted, then it's a weakness.
However if you define it as 1 - ResSS/TSS, then, for an arbitrary model
fitting procedure, R^2 isn't even constrained to the interval [0,1],
since ResSS might exceed TSS.
Here
> X Y YHat Y-Yhat
> 1 101 97 4
> 2 102 99 3
> 3 103 101 2
> 4 104 103 1
> 5 105 105 0
> 6 106 107 -1
> 7 107 109 -2
> 8 108 111 -3
> 9 109 113 -4
> 10 110 115 -5
Since the spacings between the Ys are the same as the spacings between
the residuals, it follows that ResSS=TSS and R^2=0. You do no better
predicting Y from the fitted line than you do from using the mean.
Jerry Dallal
It depends how you defined R2. If you define it as the square of the
correlation between observed and predicted, then it's a weakness.
However if you define it as 1 - ResSS/TSS, then, for an arbitrary model
fitting procedure, R2 isn't even constrained to the interval [0,1],
since ResSS might exceed TSS.
Here
> X Y YHat Y-Yhat
> 1 101 97 4
> 2 102 99 3
> 3 103 101 2
> 4 104 103 1
> 5 105 105 0
> 6 106 107 -1
> 7 107 109 -2
> 8 108 111 -3
> 9 109 113 -4
> 10 110 115 -5
Here, TSS=82.5 and ResSS=85, so R^2 = 1-85/82.5 = -0.03, and the fitted
line predicts worse than always using the sample mean.
Jerry Dallal
> It depends how you defined R2. If you define it as the square of the
> correlation between observed and predicted, then it's a weakness.
> However if you define it as 1 - ResSS/TSS, then, for an arbitrary model
> fitting procedure, R2 isn't even constrained to the interval [0,1],
> since ResSS might exceed TSS.
>
> Here
> > X Y YHat Y-Yhat
> > 1 101 97 4
> > 2 102 99 3
> > 3 103 101 2
> > 4 104 103 1
> > 5 105 105 0
> > 6 106 107 -1
> > 7 107 109 -2
> > 8 108 111 -3
> > 9 109 113 -4
> > 10 110 115 -5
>
> Here, TSS=82.5 and ResSS=85, so R^2 = 1-85/82.5 = -0.03, and the fitted
> line predicts worse than always using the sample mean.
I should have added that there was an earlier version of this post that
was in error, claiming that ResSS=TSS. We don't want the corrected SS,
but the actual SS, which differs here because the mean of the residuals
is not 0. I've canceled my earlier post, but given the way cancels
propagate, some copies of the original will survive. So, for the
record, keep this post and the one with R^2= -0.03, and ignore the one
with R^2=0.
Reef Fish
Jerry Dallal wrote:
> I wrote:
>
> > It depends how you defined R2. If you define it as the square of the
> > correlation between observed and predicted, then it's a weakness.
What do you mean "a weakness"?
For OLS fitted regression, R^2 is ALWAYS the correlation between
the observed Y and the fitted Y.
What a tangled web we weave, when we stray from the standard linear
model methods in statistics to nonstatistical methods such as the
"arbitrary model fitting procedure" as measuring errors orthogonal
to the fitted line? What good is the new-gangled-or-mangled R^2
from a statistical point of view? A NEGATIVE value for R^2 is
second only to the "expected sign" fallacy in regression abuses.
-- Bob.
Jerry Dallal
>
> Jerry Dallal wrote:
>
>>I wrote:
>>
>>
>>>It depends how you defined R2. If you define it as the square of the
>>>correlation between observed and predicted, then it's a weakness.
>
>
> What do you mean "a weakness"?
>
> For OLS fitted regression, R^2 is ALWAYS the correlation between
> the observed Y and the fitted Y.
You have to read the thread.
The values "Y-Yhat" below are my own calculation. The rest is from the
person posing the question, to wit: The correlation between Y and Yhat
is 1. If one defines R^2 for any model (not necessarily linear LS) as
the square of the correlation between observed and predicted, then R^2
for this example is 1. Was that an indication of a weakness in R^2 as a
summary measure?
My point was that while, as you say, "For OLS fitted regression, R^2 is
ALWAYS the [square of the] correlation between the observed Y and the
fitted Y.", R^2 is not defined that way. Rather it is usually defined
as 1-ResSS/TSS (or RegSS/TSS), which, for OLS, *happens* to be the
square of the correlation between Y and Yhat. [I realize there are
*many* way to approach R^2.] If one uses the formal definition of R^2
to calculate it for this example, R^2 turns out to be -0.03, which says
the problem is with the model, not R^2.
Reef Fish
Jerry Dallal wrote:
> Reef Fish wrote:
> >
> > Jerry Dallal wrote:
> >
> >>I wrote:
> >>
> >>
> >>>It depends how you defined R2. If you define it as the square of the
> >>>correlation between observed and predicted, then it's a weakness.
> >
> >
> > What do you mean "a weakness"?
> >
> > For OLS fitted regression, R^2 is ALWAYS the correlation between
> > the observed Y and the fitted Y.
>
> You have to read the thread.
Didn't think it necessary, even now. Reading what YOU wrote sufficed.
>
> The values "Y-Yhat" below are my own calculation. The rest is from the
> person posing the question, to wit: The correlation between Y and Yhat
> is 1. If one defines R^2 for any model (not necessarily linear LS) as
> the square of the correlation between observed and predicted, then R^2
> for this example is 1. Was that an indication of a weakness in R^2 as a
> summary measure?
The perception of "weakness" was your OWN, as seen below:
>
> My point was that while, as you say, "For OLS fitted regression, R^2 is
> ALWAYS the [square of the] correlation between the observed Y and the
> fitted Y.", R^2 is not defined that way.
That's correct. It is defined as RegSS/TotSS, (see Neter, Kutner
et al or any regression textbook will do).
That's where the "proportion of variation fitted by the regresssion"
interpretation comes from. R-square ranges from 0 (random scatter
fitted by a horizontal line) to 1 for a "perfect linear fit.
> Rather it is usually defined
> as 1-ResSS/TSS (or RegSS/TSS),
No. But it's equivalent to the usual RegSS/TotSS because
RegSS + SSE (your ResSS) = TotSS.
> which, for OLS, *happens* to be the
> square of the correlation between Y and Yhat.
That's correct.
That's also WHY the correlation coefficient has DIFFERENT
interpretations in a simple regression context, depending on
which UNIT it is expressed. The correlation r is a SIGNED
measure of linear association. The Multiple R which is the
absolute value of r, is the correlation between the observed
Y and the fitted Y, and R-squared has still a THIRD
interpretation, as defined by RegSS/TotSS.
I wrote about this on June 15:
RF> On the topic related to Correlation and Causation, Harry wrote
RF> (page 17-21) on explaining the interpretation of R-square in a
RF> regression (where R is exactly the same as the correlation
RF> coefficient |r| between X and Y in a simple regression:
RF>
RF> 1. The word "explained" is sometimes erroneous thought to
RF> connote causation whereas it refers only to deviations
RF> of fitted values from the overall mean, without any
RF> implication that the regression model that produced
RF> these fitted values has captured any causal scheme
RF> underlying the data.
> [I realize there are
> *many* way to approach R^2.]
I included ALL of the correct ways above, on R and R^2.
> If one uses the formal definition of R^2
> to calculate it for this example, R^2 turns out to be -0.03, which says
> the problem is with the model, not R^2.
This is your ERROR, Jerry.
The definition of Multiple R^2 CANNOT lead to a negative value!
It is what some economists messed with and called Adjusted R^2 that
can take on negative values, to minus infinity, I think.
That is NOT statistics. That's Quackery of the Social Sciences.
A NEGATIVE R^2 should have WARNED you that it's Quackery.
Furthermore, there is NO adjustment of R^2 necessary nor does it
gain anything in the adjustment!
Jerry, I think you've been OVER-EXPOSED to social scientists to
have picked up the Quackery of a negative R^2.
Learn your regression from STATISTICS books!
-- Bob.
Jerry Dallal
>
> Jerry Dallal wrote:
>
>>Reef Fish wrote:
>>
>>>Jerry Dallal wrote:
>>Rather it is usually defined
>>as 1-ResSS/TSS (or RegSS/TSS),
>
>
> No. But it's equivalent to the usual RegSS/TotSS because
> RegSS + SSE (your ResSS) = TotSS.
Isn't that what "or" means, as in "3/6 or 1/2"?
>
>>If one uses the formal definition of R^2
>>to calculate it for this example, R^2 turns out to be -0.03, which says
>>the problem is with the model, not R^2.
>
>
> This is your ERROR, Jerry.
>
> The definition of Multiple R^2 CANNOT lead to a negative value!
>
I'm not sure what the issue is here. R^2 cannot lead to a negative
value in the land of sanity and least squares.
The poster was getting an R^2 of 1 for his ill-fitting model, not
obtained by any least squares procedure, by calculating it as the square
of the correlation between observed and predicted and thought it showed
a weakness in R^2 as a summary measure.
The problem was not with R^2, but with the poster's definition of it.
One can calculate RegSS, ResSS, and TotalSS. The poster's model was
worse (in terms of least squared errors) than no model at all, that is,
ResSS was greater than TotalSS. If one blindly plugs these numbers into
a formula for R^2 one gets -0.03. The point is that R^2 is not, in
fact, deficient for suggesting the model is perfect. Rather, it is
saying that something is very wrong with the model because it gives a
negative value where such a thing should be impossible. One would hope
that a measure of goodness-of-fit would go off the scale when assessing
a model that(a) was derived under methods different from those the
measure was designed to assess and (b) is worse than no model at all.
Reef Fish
Jerry Dallal wrote:
> Reef Fish wrote:
> >
> > Jerry Dallal wrote:
> >
> >>Reef Fish wrote:
> >>
> >>>Jerry Dallal wrote:
>
> >>Rather it is usually defined
> >>as 1-ResSS/TSS (or RegSS/TSS),
> >
> >
> > No. But it's equivalent to the usual RegSS/TotSS because
> > RegSS + SSE (your ResSS) = TotSS.
>
> Isn't that what "or" means, as in "3/6 or 1/2"?
My "no" was referring to "it is usually defined as".
I probably never read the book from which you got your
definition, because I've NEVER seen R^2 DEFINED as "1-ResSS/TSS)".
>
> >
> >>If one uses the formal definition of R^2
> >>to calculate it for this example, R^2 turns out to be -0.03, which says
> >>the problem is with the model, not R^2.
> >
> >
> > This is your ERROR, Jerry.
> >
> > The definition of Multiple R^2 CANNOT lead to a negative value!
> >
>
> I'm not sure what the issue is here. R^2 cannot lead to a negative
> value in the land of sanity and least squares.
Excuse me. Are we discussing statistics in Alice in Wonderland?
>
> The poster was getting an R^2 of 1 for his ill-fitting model, not
> obtained by any least squares procedure, by calculating it as the square
> of the correlation between observed and predicted and thought it showed
> a weakness in R^2 as a summary measure.
>
> The problem was not with R^2, but with the poster's definition of it.
Then why not tell it in Plain English that R^2 is a mathematical
quantity that CANNOT possibly take on a negative value UNLESS
someone is mangling it by introducing something improper! I mentioned
the economist's use Adjusted R^2 as another example of Quackery.
> One can calculate RegSS, ResSS, and TotalSS. The poster's model was
> worse (in terms of least squared errors) than no model at all, that is,
> ResSS was greater than TotalSS. If one blindly plugs these numbers into
> a formula for R^2 one gets -0.03. The point is that R^2 is not, in
> fact, deficient for suggesting the model is perfect. Rather, it is
> saying that something is very wrong with the model because it gives a
> negative value where such a thing should be impossible. One would hope
> that a measure of goodness-of-fit would go off the scale when assessing
> a model that(a) was derived under methods different from those the
> measure was designed to assess and (b) is worse than no model at all.
Your follow-up did not clarify or rectify the issue that whatever
the OP did, it was statistical NONSENSE.
-- Bob.
Jerry Dallal
>
> Jerry Dallal wrote:
>
>>Reef Fish wrote:
>>
>>>Jerry Dallal wrote:
>>>
>>>
>>>>Reef Fish wrote:
>>>>
>>>>
>>>>>Jerry Dallal wrote:
>>
>>>>Rather it is usually defined
>>>>as 1-ResSS/TSS (or RegSS/TSS),
>>>
>>>
>>>No. But it's equivalent to the usual RegSS/TotSS because
>>>RegSS + SSE (your ResSS) = TotSS.
>>
>>Isn't that what "or" means, as in "3/6 or 1/2"?
>
>
> My "no" was referring to "it is usually defined as".
>
> I probably never read the book from which you got your
> definition, because I've NEVER seen R^2 DEFINED as "1-ResSS/TSS)".
>
I'm willing to concede the point, but for the fun of it I pulled four
texts from my shelf:
Draper & Smith, 2nd: RegSS/TotSS, as "Percentage Variation Explained"
Netter et al., latest ed: R^2 = RegSS/TSS = 1-ResSS/TSS
Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
Searle: the square of the cc between observed and predicted!
>>>>If one uses the formal definition of R^2
>>>>to calculate it for this example, R^2 turns out to be -0.03, which says
>>>>the problem is with the model, not R^2.
>>>
>>>
>>>This is your ERROR, Jerry.
>>>
>>>The definition of Multiple R^2 CANNOT lead to a negative value!
>>>
>>
>>I'm not sure what the issue is here. R^2 cannot lead to a negative
>>value in the land of sanity and least squares.
>
>
> Excuse me. Are we discussing statistics in Alice in Wonderland?
In this instance, yes!
> Then why not tell it in Plain English that R^2 is a mathematical
> quantity that CANNOT possibly take on a negative value UNLESS
> someone is mangling it by introducing something improper! I mentioned
> the economist's use Adjusted R^2 as another example of Quackery.
>
> Your follow-up did not clarify or rectify the issue that whatever
> the OP did, it was statistical NONSENSE.
You might look at it that way. You might also look at it as answering
the question, "How does this measure work if applied to arbitrary
models?" and leaving it to the reader to draw his/her own inference
about R^2=-0.03.
Jerry Dallal
> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
Should have been
> Kleinbaum et al,, latest: (TotSS-ResSS)/TotSS
Reef Fish
Jerry Dallal wrote:
> Reef Fish wrote:
> >
> > Jerry Dallal wrote:
> >
> >>Reef Fish wrote:
> >>
> >>>Jerry Dallal wrote:
> >>>
> >>>
> >>>>Reef Fish wrote:
> >>>>
> >>>>
> >>>>>Jerry Dallal wrote:
> >>
> >>>>Rather it is usually defined
> >>>>as 1-ResSS/TSS (or RegSS/TSS),
> >>>
> >>>
> >>>No. But it's equivalent to the usual RegSS/TotSS because
> >>>RegSS + SSE (your ResSS) = TotSS.
> >>
> >>Isn't that what "or" means, as in "3/6 or 1/2"?
> >
> >
> > My "no" was referring to "it is usually defined as".
> >
> > I probably never read the book from which you got your
> > definition, because I've NEVER seen R^2 DEFINED as "1-ResSS/TSS)".
> >
>
> I'm willing to concede the point, but for the fun of it I pulled four
> texts from my shelf:
>
> Draper & Smith, 2nd: RegSS/TotSS, as "Percentage Variation Explained"
So THEY contributed to the misconception and TWO ERRORS ("Percentage"
and "Explained") I wrote about.
> Netter et al., latest ed: R^2 = RegSS/TSS = 1-ResSS/TSS
I've taught from Neter et al (several editions) and R^2 was
always DEFINED as RegSS/TotSS. Yours must've been some "Netter". :-)
> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
your copying error.
>
> Searle: the square of the cc between observed and predicted!
That's a baddy, as a definition.
>
> >>>>If one uses the formal definition of R^2
What formal definition, Jerry? Now that you've listed three (and
one typo) from statistics textbooks?
> >>>>to calculate it for this example, R^2 turns out to be -0.03, which says
> >>>>the problem is with the model, not R^2.
> >>>
> >>>
> >>>This is your ERROR, Jerry.
> >>>
> >>>The definition of Multiple R^2 CANNOT lead to a negative value!
> >>>
> >>
> >>I'm not sure what the issue is here. R^2 cannot lead to a negative
> >>value in the land of sanity and least squares.
> >
> >
> > Excuse me. Are we discussing statistics in Alice in Wonderland?
>
> In this instance, yes!
Actually Beyond Alice in Wonderland! :-) See above references to
Kleinbaum, your "formal definition of R^2" and R^2 = -.03.
>
> > Then why not tell it in Plain English that R^2 is a mathematical
> > quantity that CANNOT possibly take on a negative value UNLESS
> > someone is mangling it by introducing something improper! I mentioned
> > the economist's use Adjusted R^2 as another example of Quackery.
>
> >
> > Your follow-up did not clarify or rectify the issue that whatever
> > the OP did, it was statistical NONSENSE.
>
> You might look at it that way.
There's no other valid way to look at it, Jerry.
> You might also look at it as answering
> the question, "How does this measure work if applied to arbitrary
> models?" and leaving it to the reader to draw his/her own inference
> about R^2=-0.03.
How does WHAT measure work? There is some weak excuse for using
the Searle-like "definition" to get some correlation, but even
Searle's definition would NOT yield a NEGAGIVE number, unless
Searle can get an complex number i*sqrt(-0.3) as a correlation.
-- Bob.
Jerry Dallal
>
> Jerry Dallal wrote:
>>Netter et al., latest ed: R^2 = RegSS/TSS = 1-ResSS/TSS
>
>
> I've taught from Neter et al (several editions) and R^2 was
> always DEFINED as RegSS/TotSS. Yours must've been some "Netter". :-)
Need a big net to catch a big fish.
I am copying verbatim from the third edition, (the latest is at the office)
p 100:
"Thus SSTO is a measure of uncertainty in predicting Y when X is not
considered. Similarly, SSE measures the variation in the Y(i) when a
regression model using the independent variable X is employed. A
natural measure of the effect of X in reducing the variation in Y, i.e.,
the uncertainty in predicting Y, is therefore:
(3.71) r^2 = (SSTO-SSE)/SSTO = SSR/SSTO = 1-SSE/SSTO "
Also, p 241:
"The coefficient of multiple determination, denoted R^2, is defined as
follows:
(7.35) R^2 = SSR/SSTO = 1 - SSE/SSTO
It measures the proportionate reduction of total variation..."
Reef Fish
Jerry Dallal wrote:
> Reef Fish wrote:
> >
> > Jerry Dallal wrote:
> >>Netter et al., latest ed: R^2 = RegSS/TSS = 1-ResSS/TSS
> >
> >
> > I've taught from Neter et al (several editions) and R^2 was
> > always DEFINED as RegSS/TotSS. Yours must've been some "Netter". :-)
>
> Need a big net to catch a big fish.
LOL! http://www.ivydene1.co.uk/doug/dive/images/grouper.gif
>
> I am copying verbatim from the third edition, (the latest is at the office)
As you know, I am recalling EVERYTHING from my big Soft Disk as I had
given away all my statistics books to 10 libraries in China years ago.
>
> p 100:
>
> "Thus SSTO is a measure of uncertainty in predicting Y when X is not
> considered. Similarly, SSE measures the variation in the Y(i) when a
> regression model using the independent variable X is employed. A
> natural measure of the effect of X in reducing the variation in Y, i.e.,
> the uncertainty in predicting Y, is therefore:
>
> (3.71) r^2 = (SSTO-SSE)/SSTO = SSR/SSTO = 1-SSE/SSTO "
I must have recalled the MUCH BETTER definition of R^2 in the 1st
and 2nd editions, as SSReg/SSTot.
>
>
> Also, p 241:
>
> "The coefficient of multiple determination, denoted R^2, is defined as
> follows:
> (7.35) R^2 = SSR/SSTO = 1 - SSE/SSTO
That's better, as the definition.
Ah, this came FIRST, didn't it? (7.35). You were putting (7.71)
first in this post as if it were the definition when Neter et al
were just relating some of the ANOVA table entries to little r^2,
in the SIMPLE regression chapter, I presume, because the relation
applies ONLY to simple regression.
> It measures the proportionate reduction of total variation..."
That's an odd way to put it. To use the parallel of "% of
variation explained", but correcting the two errors, the better
expression would have been,
"It measures the proportion of total variation fitted by the
regression".
So, what happened to this:
JD> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
RF> IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
RF> your copying error.
or how YOU and the others got the R^2 = -.03 ?
I assume it's typo and carelessness respectively, but wanted to know
if otherwise.
-- Bob.
Jerry Dallal
No, not a typo. The page numbers and equation numbers are correct. r^2
is defined for simple linear regression; R^2 for multiple regression.
>
>>It measures the proportionate reduction of total variation..."
>
>
> That's an odd way to put it. To use the parallel of "% of
> variation explained", but correcting the two errors, the better
> expression would have been,
>
> "It measures the proportion of total variation fitted by the
> regression".
That's why I like your suggestion of "variation fitted". No text that
I've read has an equally suitable replacement for "explained by". It's
all mumbo-jumbo.
>
> So, what happened to this:
>
> JD> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
>
> RF> IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
> RF> your copying error.
>
> or how YOU and the others got the R^2 = -.03 ?
>
>
> I assume it's typo and carelessness respectively, but wanted to know
> if otherwise.
>
> -- Bob.
>
Typo, yes; but not completely careless because I posted a correction
right after I'd sent the initial message. On my site the original is
stamped 5:21 pm, the correction at 5:33, and your question about it 7:12
pm. Perhaps you missed the 5:33 post. It *was* in the same thread, a
reply to my own post.
--Jerry
Reef Fish
Jerry Dallal wrote:
> Reef Fish wrote:
> >
> > Jerry Dallal wrote:
> >
> >>Reef Fish wrote:
> >>
> >>>Jerry Dallal wrote:
< Snip >
> >>Also, p 241:
> >>
> >>"The coefficient of multiple determination, denoted R^2, is defined as
> >>follows:
> >>(7.35) R^2 = SSR/SSTO = 1 - SSE/SSTO
> >
> >
> > That's better, as the definition.
> >
> > Ah, this came FIRST, didn't it? (7.35). You were putting (7.71)
> > first in this post as if it were the definition when Neter et al
> > were just relating some of the ANOVA table entries to little r^2,
> > in the SIMPLE regression chapter, I presume, because the relation
> > applies ONLY to simple regression.
>
> No, not a typo. The page numbers and equation numbers are correct. r^2
> is defined for simple linear regression; R^2 for multiple regression.
Never said THAT was a typo. Read what I wrote again. I said you
chose to show (7.71) FIRST, instead of the definition (7.35).
> >
> > "It measures the proportion of total variation fitted by the
> > regression".
I've been using that for DECADES in my Lecture Notes.
>
> That's why I like your suggestion of "variation fitted". No text that
> I've read has an equally suitable replacement for "explained by". It's
> all mumbo-jumbo.
I am quite sure others have used much less misleading terms than
"percent variation explained". My co-author Harry Roberts did use
the word "explain" but immediately explained at length that it
must NOT be taken to mean causal or other meaning of "explain".
In retrospect, I should have suggested the simple, unambiguous
wording of "variation fitted" because that's all it is, no more,
no less.
> > So, what happened to this:
> >
> > JD> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
> >
> > RF> IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
> > RF> your copying error.
> >
> > or how YOU and the others got the R^2 = -.03 ?
> >
> >
> > I assume it's typo and carelessness respectively, but wanted to know
> > if otherwise.
> >
> > -- Bob.
> >
>
> Typo, yes; but not completely careless
Sorry, the "respctively" did not make it clear that the typo was
referring to ONLY
> > JD> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
> >
> > RF> IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
> > RF> your copying error.
which you posted for the first time. So, what was ACTUALLY in
Kleinbaum's book?
> because I posted a correction
> right after I'd sent the initial message. On my site the original is
> stamped 5:21 pm, the correction at 5:33, and your question about it 7:12
> pm. Perhaps you missed the 5:33 post. It *was* in the same thread, a
> reply to my own post.
>
> --Jerry
The "careless" was referring to
RF> > or how YOU and the others got the R^2 = -.03 ?
In Google, you made THREE consecutive posts, at 8:08 pm, 8:16 pm and
8:27 om of July 12.
Your correction of your own post (8:27 pm) was this:
JD> I've canceled my earlier post, but given the way cancels
JD> propagate, some copies of the original will survive. So, for the
JD> record, keep this post and the one with R^2= -0.03, and ignore the
one
JD> with R^2=0.
You KEPT the R^2 = -.03,
which certainly did not follow from any of the definitions you cited.
-- Bob.
Jerry Dallal
Reef Fish wrote:
>
> Jerry Dallal wrote:
>
>>Reef Fish wrote:
>>
>>>Jerry Dallal wrote:
>>>
>>>
>>>>Reef Fish wrote:
>>>>
>>>>
>>>>>Jerry Dallal wrote:
>
>
> < Snip >
>
>>>>Also, p 241:
>>>>
>>>>"The coefficient of multiple determination, denoted R^2, is defined as
>>>>follows:
>>>>(7.35) R^2 = SSR/SSTO = 1 - SSE/SSTO
>>>
>>>
>>>That's better, as the definition.
>>>
>>>Ah, this came FIRST, didn't it? (7.35). You were putting (7.71)
>>>first in this post as if it were the definition when Neter et al
>>>were just relating some of the ANOVA table entries to little r^2,
>>>in the SIMPLE regression chapter, I presume, because the relation
>>>applies ONLY to simple regression.
>>
>>No, not a typo. The page numbers and equation numbers are correct. r^2
>>is defined for simple linear regression; R^2 for multiple regression.
>
>
> Never said THAT was a typo. Read what I wrote again. I said you
> chose to show (7.71) FIRST, instead of the definition (7.35).
But it's not 7.71. It's 3.71!
>
>>>"It measures the proportion of total variation fitted by the
>>>regression".
>
>
> I've been using that for DECADES in my Lecture Notes.
>
>>That's why I like your suggestion of "variation fitted". No text that
>>I've read has an equally suitable replacement for "explained by". It's
>>all mumbo-jumbo.
>
>
> I am quite sure others have used much less misleading terms than
> "percent variation explained". My co-author Harry Roberts did use
> the word "explain" but immediately explained at length that it
> must NOT be taken to mean causal or other meaning of "explain".
> In retrospect, I should have suggested the simple, unambiguous
> wording of "variation fitted" because that's all it is, no more,
> no less.
Less misleading, yes. Concise, no. The language is often so tortured
as to be unintelligible to a naive audience, hence my descriptor
"mumbo-jumbo".
>
>
>>>So, what happened to this:
>>>
>>>JD> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
>>>
>>>RF> IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
>>>RF> your copying error.
>>>
>>>or how YOU and the others got the R^2 = -.03 ?
>>>
>>>
>>>I assume it's typo and carelessness respectively, but wanted to know
>>>if otherwise.
>>>
>>>-- Bob.
>>>
>>
>>Typo, yes; but not completely careless
>
>
> Sorry, the "respctively" did not make it clear that the typo was
> referring to ONLY
>
>
>>>JD> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
>>>
>>>RF> IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
>>>RF> your copying error.
>
>
> which you posted for the first time. So, what was ACTUALLY in
> Kleinbaum's book?
As I posted in my earlier correction, (TotSS-ResSS)/TotSS
>
> The "careless" was referring to
>
> RF> > or how YOU and the others got the R^2 = -.03 ?
>
> In Google, you made THREE consecutive posts, at 8:08 pm, 8:16 pm and
> 8:27 om of July 12.
>
> Your correction of your own post (8:27 pm) was this:
>
> JD> I've canceled my earlier post, but given the way cancels
> JD> propagate, some copies of the original will survive. So, for the
> JD> record, keep this post and the one with R^2= -0.03, and ignore the
> one
> JD> with R^2=0.
>
> You KEPT the R^2 = -.03,
>
> which certainly did not follow from any of the definitions you cited.
I gave the data!
> X Y YHat Y-Yhat
> 1 101 97 4
> 2 102 99 3
> 3 103 101 2
> 4 104 103 1
> 5 105 105 0
> 6 106 107 -1
> 7 107 109 -2
> 8 108 111 -3
> 9 109 113 -4
> 10 110 115 -5
X and Y are the data. Yhat is the fit of the model proposed by the
poster. The values were given by him. They have a correlation
coefficient of 1 with Y. Hence, the square of the correlation between
observed and expected values is 1, even thought the fit is far from
perfect. This is why he was asking whether it was a "defect" in R^2.
*I* calculated the residuals: Y-Yhat
ResSS is the sum of their (residuals) squares = 85. However, TSS =
Sum[(Y-105.5)^2] is only 82.5.
I plug those numbers into 1-ResSS/TotSS and get -0.03. Do you get
something different?
One might also "argue" that since the model does worse than no model at
all, that the RegSS is negative (the net amount it accounts for is
negative) and get at it that way.
Given these Ys and Yhats, -0.03 is what you get when you plug the
numbers into the formula! It's like assigning code numbers to subjects'
ethnicity and calculating the mean. It's *worse* than meaningless
(because the result is ennobled by having gone through a statistics
program), but a number pops out nonetheless.
Hey, this *is* Alice in Wonderland! The whole point is that the result
is nonsensical. But it *is* -0.03. :-)
--Jerry
Reef Fish
Jerry Dallal wrote:
> Reef Fish wrote:
< big snip >
> > Never said THAT was a typo. Read what I wrote again. I said you
> > chose to show (7.71) FIRST, instead of the definition (7.35).
>
> But it's not 7.71. It's 3.71!
You are right! I stand corrected on the order of the equations.
So the resolution of this mystery is that Neter et al did NOT
introduce R^2 until Chapter 7 (multiple regression), but made
some observations about r^2 to the simple regression ANOVA
quantities in Chapter 3.
>
>
> >
> >>>"It measures the proportion of total variation fitted by the
> >>>regression".
> >
> >
> > I've been using that for DECADES in my Lecture Notes.
> >
> >>That's why I like your suggestion of "variation fitted". No text that
> >>I've read has an equally suitable replacement for "explained by". It's
> >>all mumbo-jumbo.
> >
> >
> > I am quite sure others have used much less misleading terms than
> > "percent variation explained". My co-author Harry Roberts did use
> > the word "explain" but immediately explained at length that it
> > must NOT be taken to mean causal or other meaning of "explain".
> > In retrospect, I should have suggested the simple, unambiguous
> > wording of "variation fitted" because that's all it is, no more,
> > no less.
>
>
> Less misleading, yes. Concise, no. The language is often so tortured
> as to be unintelligible to a naive audience, hence my descriptor
> "mumbo-jumbo".
Agreed.
> >>>JD> Kleinbaum et al,, latest: (RegSS-ResSS)/TotSS
> >>>
> >>>RF> IMPOSSIBLE! It's WRONG. That's not R^2 at all. I assume it's
> >>>RF> your copying error.
> >
> >
> > which you posted for the first time. So, what was ACTUALLY in
> > Kleinbaum's book?
>
>
> As I posted in my earlier correction, (TotSS-ResSS)/TotSS
Ok. The appearance of that post was delayed by google. So, that
definition is like the rest of them.
No, I didn't even look at your data! :-) I was looking ONLY
at the definitions you gave, and saw that:
(a) all had RegSS/TotSS which cannot be negative,
(b) even Searle's def. had the "square" of a correlation,
so that can't be negative either.
>
> One might also "argue" that since the model does worse than no model at
> all, that the RegSS is negative (the net amount it accounts for is
> negative) and get at it that way.
But the RegSS is SUM OF SQUSRES which can't be negative either!
>
> Given these Ys and Yhats, -0.03 is what you get when you plug the
> numbers into the formula! It's like assigning code numbers to subjects'
> ethnicity and calculating the mean. It's *worse* than meaningless
> (because the result is ennobled by having gone through a statistics
> program), but a number pops out nonetheless.
>
> Hey, this *is* Alice in Wonderland! The whole point is that the result
> is nonsensical. But it *is* -0.03. :-)
This is Alice's nightmare in not-so-Wonderland. It's past my bed time,
so I'll settle for
"the result is nonsensical"
which was what I had STARTED with (in my FIRST post in this thread)
WITHOUT going through any of the nonsensical calculations you did! :-)
-- Bob.
Richard Ulrich
<Large_Nass...@Yahoo.com> wrote:
[ snip, much]
[Jerry]> >
> > *I* calculated the residuals: Y-Yhat
> >
> > ResSS is the sum of their (residuals) squares = 85. However, TSS =
> > Sum[(Y-105.5)^2] is only 82.5.
> >
> > I plug those numbers into 1-ResSS/TotSS and get -0.03. Do you get
> > something different?
RF >
> No, I didn't even look at your data! :-) I was looking ONLY
> at the definitions you gave, and saw that:
>
> (a) all had RegSS/TotSS which cannot be negative,
>
Bob Ling reads badly.
Bob Ling is clueless when it comes to definitions
of R^2 that are useful for nonlinear regression -- or,
for non-OLS or -ML solutions to something that might
otherwise be called a linear regression problem, which
is the topic of this thread.... As Jerry wrote,
RegSS= 85, TSS= 82.5.
Thus, RegSS/TotSS is greater than 1.0, not "negative."
Thus, "1 minus the fraction" is what is negative.
[ ... ]
--
Rich Ulrich, wpi...@pitt.edu
http://www.pitt.edu/~wpilib/index.html
Reef Fish
Richard Ulrich wrote:
Richard, I was discussing the matter with Jerry. We had understood
each other. We don't need an ignorant buffoon of this newsgroup to
make YOUR own mistakes here, because YOU can't read.
> RF >
> > No, I didn't even look at your data! :-) I was looking ONLY
> > at the definitions you gave, and saw that:
> >
> > (a) all had RegSS/TotSS which cannot be negative,
> >
>
> Bob Ling reads badly.
>
> Bob Ling is clueless when it comes to definitions of R^2
Richard Ulrich does not even know what a LINEAR regression is, when
everyone else in the group except him and Bob O'Hara knew.
Jerry had already "conceded" (the word he used) that the common
definition of R^2 is
R^2 = RegSS/TotSS,
> RegSS= 85, TSS= 82.5.
>
> Thus, RegSS/TotSS is greater than 1.0, not "negative."
That would be correct. And that would have been R^2, because
RegSS cannot be negative, as *I* said.
> --
> Rich Ulrich, wpi...@pitt.edu
> http://www.pitt.edu/~wpilib/index.html
So who is it that can't read?
I had said,
RF> But the RegSS is SUM OF SQUSRES which can't be negative either!
UR> Thus, RegSS/TotSS is greater than 1.0, not "negative."
which is ANOTHER Ulrich error, BTW, from Jerry's figure. But why
would Richard Ulrich know? Finally, the definitions Jerry gave for R^2
RF> (a) all had RegSS/TotSS which cannot be negative,
Why is Richard Ulrich making a gnat and a pest of himself arguing
that R^2 = -.03?
What is your affiliation with Pitt? The Chair of the Department
where you were formerly employed said you are no longer a member
of that department.
Your errors and statistical Quackery is giving Pitt a bad name.
Go away, you pest, unless you something useful to post in STATISTICS.
You haven't done so for quite awhile now.
-- Bob.
Anon.
>
> Richard Ulrich wrote:
>
> Richard, I was discussing the matter with Jerry. We had understood
> each other. We don't need an ignorant buffoon of this newsgroup to
> make YOUR own mistakes here, because YOU can't read.
>
>
>>RF >
>>
>>>No, I didn't even look at your data! :-) I was looking ONLY
>>>at the definitions you gave, and saw that:
>>>
>>>(a) all had RegSS/TotSS which cannot be negative,
>>>
>>
>>Bob Ling reads badly.
>>
>>Bob Ling is clueless when it comes to definitions of R^2
>
>
> Richard Ulrich does not even know what a LINEAR regression is, when
> everyone else in the group except him and Bob O'Hara knew.
>
several texts, and none of them gave the definition that Bob Ling gave:
they definie a model as being linear in its parameters, whereas Bob Ling
claims that they only have to be linear in functions of the parameters.
Indeed, Draper & Smith (Applied Regression Ananlysis, 1998) discuss
models of a form y = f(a1) X1 + g(a2) as being non-linear (htey describe
them as "intrinsically linear").
So, even if Bob Ling might have been a big fish in the statistical pond
many years ago, I'm not accepting his authority on this matter.
Bob
--
Bob O'Hara
Department of Mathematics and Statistics
P.O. Box 68 (Gustaf Hällströmin katu 2b)
FIN-00014 University of Helsinki
Finland
Telephone: +358-9-191 51479
Mobile: +358 50 599 0540
Fax: +358-9-191 51400
WWW: http://www.RNI.Helsinki.FI/~boh/
Journal of Negative Results - EEB: www.jnr-eeb.org
Reef Fish
Anon. wrote:
> Reef Fish wrote:
> >
> > Richard Ulrich wrote:
> >
> > Richard, I was discussing the matter with Jerry. We had understood
> > each other. We don't need an ignorant buffoon of this newsgroup to
> > make YOUR own mistakes here, because YOU can't read.
> >
> >
> >>RF >
> >>
> >>>No, I didn't even look at your data! :-) I was looking ONLY
> >>>at the definitions you gave, and saw that:
> >>>
> >>>(a) all had RegSS/TotSS which cannot be negative,
> >>>
> >>
> >>Bob Ling reads badly.
> >>
> >>Bob Ling is clueless when it comes to definitions of R^2
> >
> >
> > Richard Ulrich does not even know what a LINEAR regression is, when
> > everyone else in the group except him and Bob O'Hara knew.
> >
> For the record, I checked the definition of a linear regression in
> several texts, and none of them gave the definition that Bob Ling gave:
For the record, the definition I gave was exactly the same definition
Kendall and Stuart gave, and also exactly the same definition given
by every author or Linear regression!
> they definie a model as being linear in its parameters, whereas Bob Ling
> claims that they only have to be linear in functions of the parameters.
That part was left as unspecified as was the CONSTRAINTS on the
coefficients. For example, everyone in the world knows that
Y = b X (with the intercept constrained to zero) is a LINEAR model,
except Richard Ulrich who us the only one IN THE WORLD who says that
is a NONLINEAR model.
Even Bob O'Hara did not make THAT mistake.
> Indeed, Draper & Smith (Applied Regression Ananlysis, 1998) discuss
> models of a form y = f(a1) X1 + g(a2) as being non-linear (htey describe
> them as "intrinsically linear").
Silly man! An intrinsically LINEAR model is NOT a nonlinear model --
because it can be linearized by a transformation. That's why those
models are called intrinsically LINEAR, by Draper and Smith, Neter
et al, and other textbooks on LINEAR regression models.
They are not the STANDARD Linear models, as used by Kendall and Stuart,
Reef Fish Bob, and all other authors of Linear regression models.
They are, nevertheless, LINEAR models, because they are intrisically
so.
A non-linear model are only those that are NOT expressible in the
STANDARD form, with or without constraints on the parameters; AND
they are not intrisically linear.
> Bob O'Hara
> Department of Mathematics and Statistics
> P.O. Box 68 (Gustaf Hällströmin katu 2b)
> FIN-00014 University of Helsinki
> Finland
You and Richard Ulrich remain the ONLY two in sci.stat.* who still
don't know what a LINEAR Regression model is; and Richard Ulrich
is the ONLY ONE IN THE WORLD who thinks Y = b X is a nonlinear model.
Glad you help set THAT record straight.
-- Bob.
Richard Ulrich
On Fri, 15 Jul 2005 13:56:51 -0400, Richard Ulrich
<Rich....@comcast.net> wrote:
> On 14 Jul 2005 21:40:57 -0700, "Reef Fish"
> <Large_Nass...@Yahoo.com> wrote:
> [ snip, much]
>
> [Jerry]> >
>
> > > *I* calculated the residuals: Y-Yhat
> > >
> > > ResSS is the sum of their (residuals) squares = 85. However, TSS =
> > > Sum[(Y-105.5)^2] is only 82.5.
Jerry, *here*, is referring to ResSS and not RegSS --
> > >
> > > I plug those numbers into 1-ResSS/TotSS and get -0.03. Do you get
> > > something different?
>
> RF >
> > No, I didn't even look at your data! :-) I was looking ONLY
> > at the definitions you gave, and saw that:
> >
> > (a) all had RegSS/TotSS which cannot be negative,
- Where Bob is writing RegSS, not ResSS -
>
> Bob Ling reads badly.
>
> Bob Ling is clueless when it comes to definitions
> of R^2 that are useful for nonlinear regression -- or,
> for non-OLS or -ML solutions to something that might
> otherwise be called a linear regression problem, which
> is the topic of this thread.... As Jerry wrote,
>
> RegSS= 85, TSS= 82.5.
- I wrote wrong. Jerry said ResSS.
> Thus, RegSS/TotSS is greater than 1.0, not "negative."
>
> Thus, "1 minus the fraction" is what is negative.
> [ ... ]
But Jerry ended by writing,
'One might also "argue" that since the model does worse than no model
at all, that the RegSS is negative (the net amount it accounts for is
negative) and get at it that way.
'Given these Ys and Yhats, -0.03 is what you get when you plug the
numbers into the formula! .... '
The point remains that Bob seems clueless when it comes
to what is useful for definitions of R^2.
--
Rich Ulrich, wpi...@Pitt.edu
http://www.pitt.edu/~wpilib/index.html
Reef Fish
Richard Ulrich wrote:
> Dang! I blew it, partly,
You blew it TOTALLY, ever since you started posting your ERRORS,
misinformation, and statistical Quackery in sci.stat.* groups.
>
> On Fri, 15 Jul 2005 13:56:51 -0400, Richard Ulrich
> <Rich....@comcast.net> wrote:
> > > (a) all had RegSS/TotSS which cannot be negative,
>
> - Where Bob is writing RegSS, not ResSS -
because that is the DEFINITION of R^2 Jerry and I and three of
the books Jerry cited all agreed,
> > Bob Ling is clueless when it comes to definitions
After an entire post of frivol in which Richard Ulrich was WRONG,
so obviously wrong as I explained in the follow-up, Richard
finally woke up in his stupor to say,
> - I wrote wrong. Jerry said ResSS.
and Richard Ulrich had the unadulterated GALL to say now,
> The point remains that Bob seems clueless when it comes
> to what is useful for definitions of R^2.
>
> --
> Rich Ulrich, wpi...@Pitt.edu
> http://www.pitt.edu/~wpilib/index.html
What is your affiliation with Pitt? The Chair of the Department
where you were formerly employed said you are no longer a member
of that department.
Your errors and statistical Quackery is giving Pitt a bad name.
Go away, you pest, unless you something useful to post in STATISTICS.
You have posted NOTHING but your own statistical errors and
vacuous frivol.
-- Bob.
Richard Ulrich
responded to by a Reply following my own post.
Here, I just respond to one ad-hominem slur.
I imagine that by explaining this one here, I might remove
this it from his arsenal.
On 16 Jul 2005 06:07:28 -0700, "Reef Fish"
<Large_Nass...@Yahoo.com> wrote:
[ ...]
> Richard Ulrich does not even know what a LINEAR regression is, when
> everyone else in the group except him and Bob O'Hara knew.
[ ... ]
If that is *fair*, then
I suppose it would be just-as-fair if I described Bob Ling
as arguing that Smoking does not cause cancer.
- After all, it is only a few steps away from his actual
argument that the evidence is poor (which he makes
without any indication of familiarity with the evidence).
- And Bob is a few steps away from describing good evidence
that "Richard Ulrich does not even know what a LINEAR
regression is."
Let's see -- I wrote *one* post in the thread on distinguishing
"linear" regression. A neat brain-storming session was
disintegrating, without any order emerging as to *who*
wanted a definition of "linear" or why, or what it would
serve. No taxonomy was emerging, no hierarchy of terms.
Bob Ling was pushing for the furthest extension of the
reach of "what can be linear", while ignoring published
definitions and requirements that others cited. And not
necessarily without self-contradiction, I think I remember.
I don't remember anyone saying that Bob Ling had expressed
himself totally intelligibly, or convincingly.
I suggested, somewhat tongue-in-cheek for the basic term,
something like this: that maybe only the simplest model
should be called a "linear regression model" and the others
could be "<adjective> linear regression model". Several
times, I think, I asked for *terms* to use. I was hoping that
someone else might be prompted to organize the terminology
or build a taxonomy. (That didn't happen.)
- My own notion, now, is that the simplest thing be
called an "abecedarian linear regression model". That uses
a neat word (referring to abc's) that isn't used much.
An "abcd linear model", for short --
y = a + b*x + c*y + d*z + e(i) .
For greatest simplicity, no constraints would apply.
And this describes a vast majority of regressions as
taught and performed.
For engineers, "linear" requires that the x,y,z terms
cannot be x, x^2, x^3 -- For their own reasons, which I
do not know, their "linear regression model" must be
linear (both) in the variables and in the parameters. I figure
that it helps to organize their textbooks and their practice.
And that's probably what we want to do in accepting or
spreading terminology.
In math-stat, we say "linear in the parameters."
The abcd linear model is the simplest. Matrix notation
suggests a wider scope; described in common notation;
and allowing common solutions; with familiar properties
for the error terms. Solving matrices including "linear
constraints" -- Now, this gets into the widening of
definitions beyond the abcd model, but seems like
something statisticians will widely accept as the
canonical "linear regression model". Beyond that,
I don't say that we should not call something some
*sort* of linear regression model, but sometimes it is
going to be advisable to keep the adjective-description
attached to it.
Saying that something "can be written as a linear
regression model" suggests to me a natural distinction --
It is NOT one as presently written, but it *can* be.
What is the cost, or the loss, of doing so? That can
be something interesting to explore. If there is no
cost or loss at all, or cost is small, then the alternate
form should be fully accepted: It *is* already a
linear model, written in another form; this other form
is another way to write some linear models.
Similarly, saying that something "can be solved"
as a linear regression model implies that it is *not*,
necessarily, precisely the same. What gets included
by this? What is the cost or loss of doing so? ... and
so on.
- Now, after this, if Bob repeats the same ad-hominem,
I think I may feel justified in replying, briefly, by citing this
post and noting that Bob Ling must be clueless when it
comes to discussing terminology.
Reef Fish
Richard Ulrich wrote:
> There was a valid point within Bob's note, which I have
> responded to by a Reply following my own post.
>
> Here, I just respond to one ad-hominem slur.
>
> [ ...]
> > Richard Ulrich does not even know what a LINEAR regression is, when
> > everyone else in the group except him and Bob O'Hara knew.
> [ ... ]
That's an ad hominem slur? The FACT had been so thoroughly
documented in various threads relating to the definition of LINEAR
regression models that only the BLIND would not have known it.
<--------- excerpted from a post summarizing responses -------->
DEFINITION of Linear models> PE Anon Bob Ulrich Dallal
6/14 6/10 6/10-11
(2) linear linear linear -- linear
(3) linear linear* linear* nonlin --
(4) linear witn b>0 constr. <wrong> linear nonlin linear
(5) linear with b^2>0 constr. lin/non lin/non nonlin --
(6) linear with 0 intercept linear linear nonlin linear
(7) linear with 0 same as (5) as (5) as (5) nonlin --
(8) linear with constraint linear linear nonlin linear
* both gave same answer of linear in b^3 but nonlinear in b.
(see MY explanation) that the definition does not depend on
the linear FORM of the parameter. Partial credit.
<------------------ end excerpt ------------------------------->
Richard Ulrich was the ONLY one whose responses were ALL WRONG.
Richard Ulrich was also the ONLY person in the world who said
Y = b X is a NONLINEAR regression model.
Now, what's ad hominem about the above FACTS in the archives?
> - And Bob is a few steps away from describing good evidence
> that "Richard Ulrich does not even know what a LINEAR
> regression is."
What I excerpted above is just ONE of dozens of posts which supported
> that "Richard Ulrich does not even know what a LINEAR
> regression is."
< ... lengthy excuses and obfuscation by Ulrich snipped ... >
> --
> Rich Ulrich, wpi...@pitt.edu
> http://www.pitt.edu/~wpilib/index.html
Why did you NOT answer the question about using your Pitt sig
when you are no longer affiliated with Pitt, or Pitt's
Psychiatry Department, as confirmed by the Chair of the Dept.?
You are a fraud, a statistical Quack, a statistical malpractitioner,
who is a DISGRACE to Pitt and the field of Statistics!
-- Bob.
Erkki.Ko...@helsinki.fi.invalid
> X Y YHat Y-Yhat
> 1 101 97 4
> 2 102 99 3
> 3 103 101 2
> 4 104 103 1
> 5 105 105 0
> 6 106 107 -1
> 7 107 109 -2
> 8 108 111 -3
> 9 109 113 -4
> 10 110 115 -5
r is obviously 1. Data not ok? Perhaps X is rownumber?
Erkki
040-5024491 <http://www.helsinki.fi/people/Erkki.Komulainen/>
Jerry Dallal
Please reread the entire thread. The data are fine. Yhat was NOT
obtained by least squares (or any other rational model fitting procedure
I know of). It is of the OP's invention to give something that has a
correlation of 1 with Y but is not Y itself. (He might just as well
have used X!)
Anon.
>
> Anon. wrote:
>
>>Reef Fish wrote:
>>
>>>Richard Ulrich wrote:
>>>
>>>Richard, I was discussing the matter with Jerry. We had understood
>>>each other. We don't need an ignorant buffoon of this newsgroup to
>>>make YOUR own mistakes here, because YOU can't read.
>>>
>>>
>>>
>>>>RF >
>>>>
>>>>>No, I didn't even look at your data! :-) I was looking ONLY
>>>>>at the definitions you gave, and saw that:
>>>>>
>>>>>(a) all had RegSS/TotSS which cannot be negative,
>>>>>
>>>>
>>>>Bob Ling reads badly.
>>>>
>>>>Bob Ling is clueless when it comes to definitions of R^2
>>>
>>>
>>>Richard Ulrich does not even know what a LINEAR regression is, when
>>>everyone else in the group except him and Bob O'Hara knew.
>>>
>>
>>For the record, I checked the definition of a linear regression in
>>several texts, and none of them gave the definition that Bob Ling gave:
>
>
> For the record, the definition I gave was exactly the same definition
> Kendall and Stuart gave, and also exactly the same definition given
> by every author or Linear regression!
>
>
>
>>they definie a model as being linear in its parameters, whereas Bob Ling
>>claims that they only have to be linear in functions of the parameters.
>
>
> That part was left as unspecified as was the CONSTRAINTS on the
> coefficients. For example, everyone in the world knows that
>
claiming that everyone else has missed something off their definitions.
<snip>
>
>> Indeed, Draper & Smith (Applied Regression Ananlysis, 1998) discuss
>>models of a form y = f(a1) X1 + g(a2) as being non-linear (htey describe
>>them as "intrinsically linear").
>
>
> Silly man! An intrinsically LINEAR model is NOT a nonlinear model --
Draper & Smith discuss intrinsically linear model is in their chapter on
non-linear models. Note the non. Yes, it can be linearised, but if you
simply estimate the parameters of the linearised model, and
back-transform, then you'll get biased estimates. Hence, in general
it's not a good approach to fitting these models, and so they are
treated as non-linear (as indeed they are, according to the definition
used everywhere but Clemson College).
> because it can be linearized by a transformation. That's why those
> models are called intrinsically LINEAR, by Draper and Smith, Neter
> et al, and other textbooks on LINEAR regression models.
>
> They are not the STANDARD Linear models, as used by Kendall and Stuart,
> Reef Fish Bob, and all other authors of Linear regression models.
> They are, nevertheless, LINEAR models, because they are intrisically
> so.
>
Clearly Kendall & Stuart, and Draper & Smith think that they are
non-linear: K&S give an example of such a model, and state that it is
non-linear, and D&S discuss these models as being a class of non-linear
models.
Either provide positive evidence that they are wrong or incompetent, or
admit that your definition is not a standard one.
Bob
--
Bob O'Hara
Department of Mathematics and Statistics
P.O. Box 68 (Gustaf Hällströmin katu 2b)
FIN-00014 University of Helsinki
Finland
Telephone: +358-9-191 51479
Richard Ulrich
<Large_Nass...@Yahoo.com> wrote:
>
> Richard Ulrich wrote:
> > There was a valid point within Bob's note, which I have
> > responded to by a Reply following my own post.
> >
> > Here, I just respond to one ad-hominem slur.
> >
> > [ ...]
> > > Richard Ulrich does not even know what a LINEAR regression is, when
> > > everyone else in the group except him and Bob O'Hara knew.
> > [ ... ]
>
> That's an ad hominem slur? The FACT had been so thoroughly
> documented in various threads relating to the definition of LINEAR
> regression models that only the BLIND would not have known it.
Oh, I see. You are a practicing advocate of the BIG LIE.
- repeat the same garbage often enough, and it becomes true.
I'll point to what I posted yesterday in sci.stat.math,
Message-ID: <0knid1du91602updl...@4ax.com>
(use the part between brackets in the advanced search, google-groups.)
And my conclusion, there: if he were to bring that up again --
Bob Ling is clueless when it comes to discussing terminology.
[snip, Bob's own table]
>
> Richard Ulrich was also the ONLY person in the world who said
>
> Y = b X is a NONLINEAR regression model.
>
>
> Now, what's ad hominem about the above FACTS in the archives?
A conclusion based on an out-of-context line of one post?
I suppose the main reason that I *suggested* the above
(not my favorite definition) was that it could suggest a
superior *rational* alternative to Bob's push-to-the-extreme.
Devising terminology. Laying out *reasons* to object to
the above could be a way to build to, "Who is using this
definition? What do we say, to which students?"
[ ... ]
>
> What I excerpted above is just ONE of dozens of posts which supported
>
> > that "Richard Ulrich does not even know what a LINEAR
> > regression is."
I think that should say "asserted" rather than "supported."
Saying the same (empty) thing over and over?
>
> Why did you NOT answer the question about using your Pitt sig
> when you are no longer affiliated with Pitt, or Pitt's
> Psychiatry Department, as confirmed by the Chair of the Dept.?
The e-mail is a useful forwarding address, which it will
remain for a time, even after the account is closed some day.
The stats-FAQ is still occasionally useful to folks.
Other details are none of your business.
I took some early retirement on good terms with my
department. Being "non-affiliated" is no big deal for me.
Can Bob say the same? The details he has provided,
amid the discussions of teaching, sure do sound
like "forcing out a tenured professor." Does he imagine
I left in disgrace?
>
> You are a fraud, a statistical Quack, a statistical malpractitioner,
> who is a DISGRACE to Pitt and the field of Statistics!
I figure that "fraud" and Quack and malpractitioner have never
had much content except that old ad-hominem intent. Oh,
they *might* boil down to this case: I am familiar with the
epidemiology data and arguments about "smoking causes
cancer" and I think they are pretty good; whereas Bob is
unfamiliar with data and arguments, and thinks they can't
be good, on a-priori logical grounds.
(I keep trying to "represent" Bob accurately, but every
re-framing re-opens the question of whether I got it.)
Bob, is that the basis of my Quackery and fraud? or may
I regard it as purely ad-hominem with no valid content?
I think Bob Ling is newly trying to be specific in his slurs.
Tentatively, I'm willing to reply to the "meaningful" ones.
State of employment does not seem meaningful, and
it seems a discredit to Bob that he should think it is.
Reef Fish
Richard Ulrich wrote:
> On 16 Jul 2005 20:05:48 -0700, "Reef Fish"
> <Large_Nass...@Yahoo.com> wrote:
>
> >
> > Richard Ulrich wrote:
> > >
> > > Here, I just respond to one ad-hominem slur.
> > >
> > > [ ...]
> > > > Richard Ulrich does not even know what a LINEAR regression is, when
> > > > everyone else in the group except him and Bob O'Hara knew.
> > > [ ... ]
> >
> > That's an ad hominem slur? The FACT had been so thoroughly
> > documented in various threads relating to the definition of LINEAR
> > regression models that only the BLIND would not have known it.
>
> Oh, I see. You are a practicing advocate of the BIG LIE.
> - repeat the same garbage often enough, and it becomes true.
Are you DENYING that what I posted (and you snipped) are NOT said
by Richard Ulrich in the archives which are easily retrievable by
ANYONE? All the Ulrich "nonlinear" claims of 6/10, and Ulrich's
assertion that Y = b X is a NONLINEAR model?
========= these are the FACTS about Ulrich, confirmable in Ulrich's
posts, in the archives!
<--------- excerpted from a post summarizing responses -------->
DEFINITION of Linear models> PE Anon Bob Ulrich Dallal
6/14 6/10 6/10-11
(2) linear linear linear -- linear
(3) linear linear* linear* nonlin --
(4) linear witn b>0 constr. <wrong> linear nonlin linear
(5) linear with b^2>0 constr. lin/non lin/non nonlin --
(6) linear with 0 intercept linear linear nonlin linear
(7) linear with 0 same as (5) as (5) as (5) nonlin --
(8) linear with constraint linear linear nonlin linear
* both gave same answer of linear in b^3 but nonlinear in b.
(see MY explanation) that the definition does not depend on
the linear FORM of the parameter. Partial credit.
<------------------ end excerpt ------------------------------->
Richard Ulrich was the ONLY one whose responses were ALL WRONG.
Richard Ulrich was also the ONLY person in the world who said
Y = b X is a NONLINEAR regression model.
=============== the above are FACTS about Ulrich.
> > Richard Ulrich was also the ONLY person in the world who said
> >
> > Y = b X is a NONLINEAR regression model.
> >
> > Now, what's ad hominem about the above FACTS in the archives?
>
> A conclusion based on an out-of-context line of one post?
Out of WHAT context? It remains Ulrich was the ONLY ONE in the WORLD
who could have said that NONSENSE, in any context. It was in the
context of LINEAR regression models. Only Richard Ulrich could
mistaken that model to be NONLINEAR.
> > Why did you NOT answer the question about using your Pitt sig
> > when you are no longer affiliated with Pitt, or Pitt's
> > Psychiatry Department, as confirmed by the Chair of the Dept.?
>
> The e-mail is a useful forwarding address, which it will
> remain for a time, even after the account is closed some day.
>
> The stats-FAQ is still occasionally useful to folks.
Not to anyone who has a modicum of skills in using keywords in
groups.google to retrieve BETTER and up-to-date posts.
> I took some early retirement on good terms with my
> department. Being "non-affiliated" is no big deal for me.
>
> Can Bob say the same? The details he has provided,
> amid the discussions of teaching, sure do sound
> like "forcing out a tenured professor."
Would you like to put up $1,000 of your money against
$1,000,000 of mine on any evidence YOU can provide for
your SLUR of my profession and retirement? If you can
provide the SLIGHTEST shread of evidence that I was "forced
out a tenureed professor" you'll win your million from me.
If you can't, then you can pay me $1,000 and SHUT UP!
Do you want to PUT UP or SHUT UP?
And if I were you, I would be very careful about making
statements that may be construed as you being LIBELOUS
against me.
> Does he imagine I left in disgrace?
I don't imagine. If you didn't, you SHOULD have. You ARE a
disgrace to Pitt, and to the statistics profession, on the
basis of your ERRORS, MISINFORMATIONS, and MALPRACTICE that
are proven in the groups.google.com archives, as evidence
AGAINST you in all those respects.
-- Bob.
Radford Neal
Reef Fish <Large_Nass...@Yahoo.com> wrote:
>Why is Richard Ulrich making a gnat and a pest of himself arguing
>that R^2 = -.03?
It's just speculation on my part, but maybe (just a possibility, mind
you), Richard Ulrich might perhaps (I know it's silly, but bear with
me), have in his mind that the right definition of R^2 to use is one
that actually, you know, means something useful.
From this (no doubt misguided) viewpoint, one could be lead to believe
that one should choose a definition that preserves the (naive) idea
that a bigger R^2 is better, and a smaller R^2 is worse. So if you
somehow obtained a REALLY bad model, that's worse than just predicting
the overall mean, you would on this basis (rather silly, I realize)
expect to get an R^2 that's less than zero, rather than an R^2 that's
greater than one.
Of course, this is ridiculous. Everyone who's not a complete QUACK
knows that keeping the property that R^2 is non-negative is much, much
more important than keeping the property that R^2 is of some use in
telling you whether your model fits well.
----------------------------------------------------------------------------
Radford M. Neal rad...@cs.utoronto.ca
Dept. of Statistics and Dept. of Computer Science rad...@utstat.utoronto.ca
University of Toronto http://www.cs.utoronto.ca/~radford
----------------------------------------------------------------------------