Negative r, positive B? regression
eleaticus
analysis a number of losers "in" a player's hand, where a loser was any ace,
king, or queen not in the hand.
The number of tricks taken by the partners in NTcorrelated quite highly and
negatively with this loser count, yet each such missing honor should
militate against trick taking.
But, one way or the other, how does a negative .75 r enter a multiple
regression with a positive load, not to mention forcing its way into the
regression in first place?
I don't get it.
???
thanks
eleaticus
no intercept
positive dependent variable
"eleaticus" <elea...@bellsouth.net> wrote in message
news:hh9r84$cnf$1...@news.eternal-september.org...
Rich Ulrich
1) Would you understand what it means if the program told you
that you had a negative R *squared* ? - that is one of the
possibilities for no-intercept regression. Until you don't know
what that means, *never* use a regression with the "no intercept"
option unless you are following a cookbook.
2) Google groups for <regression suppressor author:ulrich > .
If there is just one variable predicting, its regression coefficient
will be the same sign as the zero-order correlation. But these are
called "partial regression coefficients" because they are to be
interpreted in the prediction equation when taken all together.
--
Rich Ulrich
eleaticus
"Rich Ulrich" <rich....@comcast.net> wrote in message
news:jl2ij51e6qcd28hl8...@4ax.com...
> On Mon, 28 Dec 2009 03:02:59 -0600, "eleaticus"
> <elea...@bellsouth.net> wrote:
>
> >Forgot to specify:
> >
> >no intercept
> >positive dependent variable
> >
> >
> >"eleaticus" <elea...@bellsouth.net> wrote in message
> >news:hh9r84$cnf$1...@news.eternal-september.org...
> >> In exploring a large bridge (game) database I decided to include in an
> >> analysis a number of losers "in" a player's hand, where a loser was any
> >ace,
> >> king, or queen not in the hand.
> >>
> >> The number of tricks taken by the partners in NTcorrelated quite highly
> >and
> >> negatively with this loser count, yet each such missing honor should
> >> militate against trick taking.
> >>
> >> But, one way or the other, how does a negative .75 r enter a multiple
> >> regression with a positive load, not to mention forcing its way into
the
> >> regression in first place?
> >>
> >> I don't get it.
> >>
> >> ???
> >>
Thanks in general, but ...
> 1) Would you understand what it means if the program told you
> that you had a negative R *squared* ? - that is one of the
> possibilities for no-intercept regression. Until you don't know
> what that means, *never* use a regression with the "no intercept"
> option unless you are following a cookbook.
ALWAYS use no intercept when the dependent variable is ratio scale (true
zero, etc) and all the independent variables are ratio scale.
>
> 2) Google groups for <regression suppressor author:ulrich > .
> If there is just one variable predicting, its regression coefficient
> will be the same sign as the zero-order correlation. But these are
> called "partial regression coefficients" because they are to be
> interpreted in the prediction equation when taken all together.
As I tried to say, the variable in question was negatively and greatly
correlated with the dependent variable and by itself and/or with other
variables ALWAYS had a positive regression coefficient.
Therefore, my wonderment.
Hmmm. The first variable in a stepwise IS the only variable before the
second variable enters the regression.
oren
>
> --
> Rich Ulrich
Paul
>
>
> ALWAYS use no intercept when the dependent variable is ratio scale (true
> zero, etc) and all the independent variables are ratio scale.
Y (response variable) = time to produce a batch of widgets (ratio
scale).
X (predictor) = number of widgets in a batch (also ratio scale).
So you're saying the regression of Y on X should have no intercept?
There cannot be a fixed time component to set up the production of the
batch?
>
>
> As I tried to say, the variable in question was negatively and greatly
> correlated with the dependent variable and by itself and/or with other
> variables ALWAYS had a positive regression coefficient.
Consider the model Y = 1 - 2*X + e where e is noise independent of X.
Picture a cluster of points in the first quadrant of (X,Y) space,
generated using that model. The cluster will exhibit a negative
slope. Now picture a regression line *through the origin* for that
cluster of points. It will have a positive slope.
/Paul
eleaticus
"Paul" <paul_...@att.net> wrote in message
news:ee8c9ce4-487f-4fb4...@q2g2000yqd.googlegroups.com...
> eleaticus wrote:
> >
> >
> > ALWAYS use no intercept when the dependent variable is ratio scale (true
> > zero, etc) and all the independent variables are ratio scale.
>
> Y (response variable) = time to produce a batch of widgets (ratio
> scale).
> X (predictor) = number of widgets in a batch (also ratio scale).
>
> So you're saying the regression of Y on X should have no intercept?
> There cannot be a fixed time component to set up the production of the
> batch?
Your scenario is not as specified fromt two viewpoints:
A fixed time component is a constant, and not a variable at all,
There is no true zero to the Y variable if you consider the set up time as
part of the variable, it is rather an intervalvariable and not the ratio
scale variable I specified.
> >
> >
>
> > As I tried to say, the variable in question was negatively and greatly
> > correlated with the dependent variable and by itself and/or with other
> > variables ALWAYS had a positive regression coefficient.
>
> Consider the model Y = 1 - 2*X + e where e is noise independent of X.
> Picture a cluster of points in the first quadrant of (X,Y) space,
> generated using that model. The cluster will exhibit a negative
> slope. Now picture a regression line *through the origin* for that
> cluster of points. It will have a positive slope.
Again, your model of Y is NOT a ratio scale variable as I specified.
>
> /Paul
Paul
> "Paul" <paul_...@att.net> wrote in message
> news:ee8c9ce4-487f-4fb4...@q2g2000yqd.googlegroups.com...
> > eleaticus wrote:
> > >
> > >
> > > ALWAYS use no intercept when the dependent variable is ratio scale (true
> > > zero, etc) and all the independent variables are ratio scale.
> >
> > Y (response variable) = time to produce a batch of widgets (ratio
> > scale).
> > X (predictor) = number of widgets in a batch (also ratio scale).
> >
> > So you're saying the regression of Y on X should have no intercept?
> > There cannot be a fixed time component to set up the production of the
> > batch?
>
> Your scenario is not as specified fromt two viewpoints:
>
> A fixed time component is a constant, and not a variable at all,
> There is no true zero to the Y variable if you consider the set up time as
> part of the variable, it is rather an intervalvariable and not the ratio
> scale variable I specified.
>
not occur (due to a presumably positive constant term)? So the Kelvin
temperature scale is actually interval, not ratio, because zero
degrees Kelvin will never happen?
Ok, fine, new example: Y is the net interchange, in a given month, of
IP packets between two long-distance telephone carriers A and B. Y is
positive if A passed more traffic to B than B passed to A, negative if
B sent more to A than vice versa, and zero if they balanced out.
(Long-distance carriers make periodic payments among themselves to
compensate imbalances of this sort, although I don't know if they base
the payments on packets, connections, connection-minutes or what.) X
is the difference between A's customer charge per minute and B's
customer charge per minute (positive if A charges more). If you
prefer, you can make X the log of the ratio of A's rate to B's rate.
Either way, X = 0 implies equal rates, and as X increases (A gets more
expensive relative to B), A's customers spend less time talking
compared to B's customers (and B may gain customers from A if they
operate in the same markets), and so Y is negatively correlated to X.
Y is ratio scale (and this time the zero value can actually occur).
In an ordinary linear regression of Y on X, the constant term would be
the average difference in traffic when A and B have the same rate (X =
0). That average difference need not be zero; for instance, A might
serve a larger customer base. If that's the case (constant term
positive, say), forcing the regression through the origin can flip the
sign of the slope coefficient.
/Paul
eleaticus
That's a lot better example en re ratio scale data, paul, but don't you
think it would be pretty silly trying to find the difference Y by intercept
regression when you already know that the average difference is the
difference of the averages?
[ Sigma( w - x)/n ] = Sigma(w)/n - Sigma(x)/n
But all this avoids my question:
How in L can a variable correlated high negative with the dep variable be
first in and with a positive load?
I don't get it.
And, the same Paul as on rgb?, what I haven't said there I think is that
with intercept the r^2 is around 40%, say, in some cases, and without? 90%.
Same variables and cases but only the w/o is valid. with a minimum of 62000
cases in the by-far smallest data subset, the increase is as close to
impossible as possible by chance.
Back when I used regression with strange variables for NFL sports betting,
the w/ and w/o were approximately the same.
oren
>
> /Paul
Rich Ulrich
<elea...@bellsouth.net> wrote:
>
>"Rich Ulrich" <rich....@comcast.net> wrote in message
>news:jl2ij51e6qcd28hl8...@4ax.com...
>> On Mon, 28 Dec 2009 03:02:59 -0600, "eleaticus"
>> <elea...@bellsouth.net> wrote:
[snip, previous]
>ALWAYS use no intercept when the dependent variable is ratio scale (true
>zero, etc) and all the independent variables are ratio scale.
>
If you show me a reference that prescribes that,
I will tell you that that source is one to be avoided.
>
>>
>> 2) Google groups for <regression suppressor author:ulrich > .
>> If there is just one variable predicting, its regression coefficient
>> will be the same sign as the zero-order correlation. But these are
>> called "partial regression coefficients" because they are to be
>> interpreted in the prediction equation when taken all together.
>
>As I tried to say, the variable in question was negatively and greatly
>correlated with the dependent variable and by itself and/or with other
>variables ALWAYS had a positive regression coefficient.
>
>Therefore, my wonderment.
>
>Hmmm. The first variable in a stepwise IS the only variable before the
>second variable enters the regression.
>
You have not done the reading that I recommended last time.
2) Google groups for <regression suppressor author:ulrich > .
You might also look for <negative R2 author:ulrich > .
To search "Google groups", you can put in groups.google.com or
if you are at the main Google page, click on "more" and select
Groups.
--
Rich Ulrich
Rich Ulrich
<elea...@bellsouth.net> wrote:
[snip]
>
>And, the same Paul as on rgb?, what I haven't said there I think is that
>with intercept the r^2 is around 40%, say, in some cases, and without? 90%.
>Same variables and cases but only the w/o is valid. with a minimum of 62000
Now, if you know the basics of regression, you would know that
adding a term to a regression *has* to increase the sum-of-squares
that is explained by the regression -- and the "intercept" is an
additional term. In terms of least-squares fit, it can never do
worse.
So, how can the regression without intercept show an R^2
of 90% where it was 40% before? - because R^2 has a
DIFFERENT DEFINITION in the two cases. For no-intercept,
a common convention is to use the TOTAL (uncorrected)
sum of squares instead of the mean-corrected SS; and when
your two terms incidentally "account for" the mean, the SS
is included in both the numerator and the denominator -- instead
of intentionally omitting it from both -- and the reported R^2
reflects that SS, which is often enormous but unhelpful.
- A good general stats program, in my opinion, should warn you that
the two versions of R^2 are not at all comparable. SPSS warns.
(A program for experts might assume that the user knows all this.)
- Using the mean-corrected SS, it is readily possible to obtain
an R^2 which is negative for the no-intercept case, especially
with only one variable. That's why I warned that you should
not use no-intercept regression (especially) if you did not
understand this "feature".
- The residual SS is the guide for describing the effectiveness
of two regressions, or of the effect of removing (say) the
intercept term.
>cases in the by-far smallest data subset, the increase is as close to
>impossible as possible by chance.
>
>Back when I used regression with strange variables for NFL sports betting,
>the w/ and w/o were approximately the same.
>
If the naturally fitted intercept is truly close to zero (which
is what you expected), that will happen. If not, then not.
If still in doubt, Google up some earlier discussions.
--
Rich Ulrich
Paul
>
> But all this avoids my question:
>
> How in L can a variable correlated high negative with the dep variable be
> first in and with a positive load?
>
I was trying to get at this with my example(s), but we got side-
tracked. A negative Pearson correlation coefficient says that E[Y|X]
is a decreasing function of X; it doesn't say/imply/require that E[Y|X
= 0] = 0. Regression through the origin, however, *forces* E[Yhat|
X=0] to be 0. Suppose that your sample lives entirely in the first
quadrant. A regression line through the origin with a negative slope
would never enter the first quadrant; it would run through the second
and fourth quadrants. So in order to get the line somewhere near the
sample, it would have to have a positive slope.
In other words, forcing the regression through the origin in a
situation where it is not warranted (where E[Y|X=0] is not zero) can
cause the slope coefficient to attain the wrong sign. There may be
other explanations; for instance, multicollinearity can cause a sign
reversal (but you indicated the sign was bollixed even in a simple
regression, so that's not likely the culprit). I'd go back and
investigate carefully the reasons for running the regression through
the origin. In particular, Y being ratio scale is not a fortiori a
reason for regressing through the origin.
/Paul
eleaticus
> On Tue, 29 Dec 2009 13:45:42 -0600, "eleaticus"
> <elea...@bellsouth.net> wrote:
>
> [snip]
>
> >
> >And, the same Paul as on rgb?, what I haven't said there I think is that
> >with intercept the r^2 is around 40%, say, in some cases, and without?
90%.
> >Same variables and cases but only the w/o is valid. with a minimum of
62000
>
> Now, if you know the basics of regression, you would know that
> adding a term to a regression *has* to increase the sum-of-squares
> that is explained by the regression -- and the "intercept" is an
> additional term. In terms of least-squares fit, it can never do
> worse.
>
> So, how can the regression without intercept show an R^2
> of 90% where it was 40% before? - because R^2 has a
> DIFFERENT DEFINITION in the two cases.
I didn't ask anything about that. You are beating on a strawman.
I just pointed out that in the situations I have been examining, the real
stuff is without intercept and the "with" doesn't even come close to showing
us the real relationship quality/quantity of the dep vs indep variables.
> >Back when I used regression with strange variables for NFL sports
betting,
> >the w/ and w/o were approximately the same.
> >
>
> If the naturally fitted intercept is truly close to zero (which
> is what you expected), that will happen. If not, then not.
Y'all haven't even come close to addressing the answer to the basic
question: how can it be?
Well, you did think you were when in effect you said the results couldn't be
what they actually are, which is how I feel about it, too.
But it has been replicated now on a second data set.
(First was responder's hand characteristics when opener was a 12-14 NT;
second was opposite a 15-17 NT.)
And, the two different r^2 results back then were all approximately 33%, and
the intercept was a considerable fraction of the mean dep var.
BTW, in these cases the adjusted r^2 was minutely different that the
unadjusted.
eleaticus
On Dec 29, 2:45 pm, "eleaticus" <eleati...@bellsouth.net> wrote:
>
> But all this avoids my question:
>
> How in L can a variable correlated high negative with the dep variable be
> first in and with a positive load?
>
======
I was trying to get at this with my example(s), but we got side-
tracked. A negative Pearson correlation coefficient says that E[Y|X]
is a decreasing function of X; it doesn't say/imply/require that E[Y|X
= 0] = 0. Regression through the origin, however, *forces* E[Yhat|
X=0] to be 0. Suppose that your sample lives entirely in the first
quadrant. A regression line through the origin with a negative slope
would never enter the first quadrant; it would run through the second
and fourth quadrants. So in order to get the line somewhere near the
sample, it would have to have a positive slope.
In other words, forcing the regression through the origin in a
situation where it is not warranted (where E[Y|X=0] is not zero) can
cause the slope coefficient to attain the wrong sign. There may be
other explanations; for instance, multicollinearity can cause a sign
reversal (but you indicated the sign was bollixed even in a simple
regression, so that's not likely the culprit). I'd go back and
investigate carefully the reasons for running the regression through
the origin. In particular, Y being ratio scale is not a fortiori a
reason for regressing through the origin.
==============
Hmmm.
You do cause me to think about the fact that the strange positive load means
that with this indep variable at zero, the hand i question has all 12 As,
Ks, Qs, and thus should take 12-13 tricks, with - by itself - a Y (# of
tricks taken) of thus 12-13.
On the other hand, when using N and S hcp counts as indeps, the loser count
comes into use third, I think it was, (not with count system efficacy) but
still with a positive load, which is indefensible!
I think, as a result of your post, given that the loser count should be
negative re tricks taken, I will transform them to negatives and see what
happens. 12 losers should result in approx zero tricks-effect of responder's
hand.
oren
/Paul
Rich Ulrich
<elea...@bellsouth.net> wrote:
>
>"Rich Ulrich" <rich....@comcast.net> wrote in message
>news:edpkj5tc9amtoqgbf...@4ax.com...
>> On Tue, 29 Dec 2009 13:45:42 -0600, "eleaticus"
>> <elea...@bellsouth.net> wrote:
>>
[snip]
>>
>> So, how can the regression without intercept show an R^2
>> of 90% where it was 40% before? - because R^2 has a
>> DIFFERENT DEFINITION in the two cases.
>
>I didn't ask anything about that. You are beating on a strawman.
The primary relevance of this point is that your choice of
regressions shows another piece of fundamental ignorance
about the tool that you are using.
Are you planning to show your results anywhere?
No one knowledgable is going to be impressed if you
mis-use regression from the start.
[snip; down to a question about the suppressor variable]
>
>Y'all haven't even come close to addressing the answer to the basic
>question: how can it be?
"Suppressor" -- now I have given you the keyword three times.
And you have had another poster illustrate it.
You can ask questions about old-time posts, and there are
plenty of posts. If you don't care enough to search on it....
--
Rich Ulrich
eleaticus
> On Tue, 29 Dec 2009 15:58:11 -0600, "eleaticus"
> <elea...@bellsouth.net> wrote:
>
> >
> >"Rich Ulrich" <rich....@comcast.net> wrote in message
> >news:edpkj5tc9amtoqgbf...@4ax.com...
> >> On Tue, 29 Dec 2009 13:45:42 -0600, "eleaticus"
> >> <elea...@bellsouth.net> wrote:
> >>
> [snip]
> >>
> >> So, how can the regression without intercept show an R^2
> >> of 90% where it was 40% before? - because R^2 has a
> >> DIFFERENT DEFINITION in the two cases.
> >
> >I didn't ask anything about that. You are beating on a strawman.
>
> The primary relevance of this point is that your choice of
> regressions shows another piece of fundamental ignorance
> about the tool that you are using.
>
> Are you planning to show your results anywhere?
R U nuts? I have repeatedly stated that the r= -.75 variable (w/dep var)
entered the regression with a positive load, and was first-in in a multiple,
and with two variables that did enter before it on another analysis, entered
third also with a positive load.
Remember? That was the question subject.
>
> No one knowledgable is going to be impressed if you
> mis-use regression from the start.
>
>
> [snip; down to a question about the suppressor variable]
> >
> >Y'all haven't even come close to addressing the answer to the basic
> >question: how can it be?
>
> "Suppressor" -- now I have given you the keyword three times.
> And you have had another poster illustrate it.
And my last previous response to Paul - in this sub-thread - showed he had
said things to which I attended.
eleaticus
> And my last previous response to Paul - in this sub-thread - showed he had
> said things to which I attended.
oops. other sub-thread
eleaticus
As I said in the previous, you got me thinking about the relatively strange
variable that was causing the problem.
So, eventually I came to see the point, and to realize that all those
models - besides the sports betting - I had worked with extensively were
full models with by far the greater bulk of the variance of the independent
variables being positively related to the dep var.
In this case the strange variable was essentially - as actually compiled -
an ipsative with respect to the sum of actual, real variables that amongst
themselves accounted for darn "near" all the variance of the dep var.
And the Regression w/o intercept forces the resultant formula result to an
ipsative of the dep var.
The problem here is a lack of a full model with independent variables
directly related to the dep var.
When you walk into a restaurant without plastic or check, the meal you can
purchase depends on the cash you have, not the cash you don't have.
True, the meals you might prefer may be beyond your immediate means but the
amount you lack is ipsative with respect to what you have, and useless for
determining which of the meals of lesser cost you choose.
Just so a loser count is useful for recognizing a limit to what you can make
but not necessary given a good measure of what you can make. That is what
our point counts are pluses, and only super-nits try to take fractional
trick counts from the high card count.
It might correctly be thought that a near-pure negative/reverse count would
be useful.
oren
Rich Ulrich
<elea...@bellsouth.net> wrote:
>
>"Rich Ulrich" <rich....@comcast.net> wrote in message
>news:5agnj55nf2l49302k...@4ax.com...
>> On Tue, 29 Dec 2009 15:58:11 -0600, "eleaticus"
>> <elea...@bellsouth.net> wrote:
>>
[snip, some]
>
>R U nuts? I have repeatedly stated that the r= -.75 variable (w/dep var)
>entered the regression with a positive load, and was first-in in a multiple,
Sorry, I must have misread that. That's a silly mistake on your
part -- putting in any variables with them scaled in the wrong
direction. I wasn't expecting that one.
Yeah, what it means -- It is a sign that you have mis-stated
your model, or that your data disagree with it. It is a clear
case where the usual OLS R^2 would be negative at that step,
since the *mean* would make a much better fit, in terms of
residual, than the model that is forced through zero.
- more on why attending to the definition of R^2 matters.
I think you have figured out since then that if you call the
variables "ratio", then they all have to be scaled the same way.
When you do an OLS regression, with an intercept, the
"significance" of the intercept might be considered a test of
whether the model does have a zero intercept.
The magnitude of the intercept -- if, for instance, you saw that
it was near "13 tricks" -- should further clue you in to your problem
of the mis-imagined model.
[snip, rest]
--
Rich Ulrich