Multinomial Regression Warning

[Sheryl]

I received the following warning:

There are 134 (59.6%) cells (i.e., dependent variable levels by
subpopulations) with zero frequencies.

I also received the following comment:
The dependent variable has only one value observed in 62 (82.7%)
subpopulations.

This topic of the warning (but not the comment) came up in 2003 - the
exact link is
http://groups.google.com/group/comp.soft-sys.stat.spss/browse_thread/thread/32223432735f0f57/b808a43f858a2bd6?lnk=gst&q=I+have+73+observations+with+5+&rnum=1#b808a43f858a2bd6

It was suggested there (mostly by Brendan Halpin) regarding the case
of the zero frequency warning,
"the deviance (-2*Likelihood Ratio) can no
longer be assumed to have a chi-sq distribution, so it can no
longer be used as an overall goodness of fit indicator.

More strictly speaking, if the number of "settings" (combinations
of values of explanatory variables) is large (or dependent on the
sample, as it will be if continuous variables are present) you
can't use the -2LR test for goodness of fit. If so, the
Hosmer-Lemeshow statistic is an apparently effective fudge - as I
recall it splits the sample into deciles according to predicted
probability, and makes a calculation based on residuals in these
groups -- details should be available in SPSS, H&S's own book, and
Agresti's _Intro to Categ Data Analysis_, none of which I have to
hand ATM. "

A practical question: The Hosmer-Lemeshow option is available in
Binary Logistic Regression, but not in Multinomial. What can I do for
multinomial logistic regression?

TIA

Sheryl

[Richard Ulrich]

On Wed, 22 Aug 2007 13:06:31 -0700, Sheryl <riley....@gmail.com>
wrote:

Okay. Your warnings tell you that, probably, you have
insufficient data for whatever you are trying to model,
the way you are trying to model it.

After that -- What is your question?

--
Rich Ulrich, wpi...@pitt.edu
http://www.pitt.edu/~wpilib/index.html

[Sheryl]

On Aug 23, 3:32 am, Richard Ulrich <Rich.Ulr...@comcast.net> wrote:
> On Wed, 22 Aug 2007 13:06:31 -0700, Sheryl <riley.she...@gmail.com>

> wrote:
>
>
>
>
>
> > I received the following warning:
>
> > There are 134 (59.6%) cells (i.e., dependent variable levels by
> > subpopulations) with zero frequencies.
>
> > I also received the following comment:
> > The dependent variable has only one value observed in 62 (82.7%)
> > subpopulations.
>
> > This topic of the warning (but not the comment) came up in 2003 - the
> > exact link is

> >http://groups.google.com/group/comp.soft-sys.stat.spss/browse_thread/...

>
> > It was suggested there (mostly by Brendan Halpin) regarding the case
> > of the zero frequency warning,
> > "the deviance (-2*Likelihood Ratio) can no
> > longer be assumed to have a chi-sq distribution, so it can no
> > longer be used as an overall goodness of fit indicator.
>
> > More strictly speaking, if the number of "settings" (combinations
> > of values of explanatory variables) is large (or dependent on the
> > sample, as it will be if continuous variables are present) you
> > can't use the -2LR test for goodness of fit. If so, the
> > Hosmer-Lemeshow statistic is an apparently effective fudge - as I
> > recall it splits the sample into deciles according to predicted
> > probability, and makes a calculation based on residuals in these
> > groups -- details should be available in SPSS, H&S's own book, and
> > Agresti's _Intro to Categ Data Analysis_, none of which I have to
> > hand ATM. "
>
> > A practical question: The Hosmer-Lemeshow option is available in
> > Binary Logistic Regression, but not in Multinomial. What can I do for
> > multinomial logistic regression?
>
> Okay. Your warnings tell you that, probably, you have
> insufficient data for whatever you are trying to model,
> the way you are trying to model it.
>
> After that -- What is your question?
>
> --

> Rich Ulrich, wpi...@pitt.eduhttp://www.pitt.edu/~wpilib/index.html- Hide quoted text -
>
> - Show quoted text -

That is not what I understood from the previous discussion. What I
understood is that in this case, Hosmer-Lemeshow is still an
appropriate statistic. Of course, it's possible that the reason SPSS
doesn't offer this option in multinomial is that it's not appropriate.
Does anyone have more information about this?

[Art Kendall]

use CROSSTABS to see how many cells there should be and if in fact that
many cells are empty.
Is it possible you are treating a continuous variable as nominal? Or
that you have nominal level variables with many values?

Art Kendall
Social Research Consultants

[Bruce Weaver]

Nor did I.

> understood is that in this case, Hosmer-Lemeshow is still an
> appropriate statistic. Of course, it's possible that the reason SPSS
> doesn't offer this option in multinomial is that it's not appropriate.
> Does anyone have more information about this?

No information on how to do the H-L test for multinomial logistic
regression, no. However, you might want to take a look at this post by
Frank Harrell (and the associated thread). Do an advanced Google groups
search on this message ID:

20030610142629.2...@virginia.edu

--
Bruce Weaver
bwe...@lakeheadu.ca
www.angelfire.com/wv/bwhomedir
"When all else fails, RTFM."

[Bruce Weaver]

Art Kendall wrote:
> use CROSSTABS to see how many cells there should be and if in fact that
> many cells are empty.
> Is it possible you are treating a continuous variable as nominal? Or
> that you have nominal level variables with many values?

Art, it has been my experience in the past that NOMREG gives the warning
about empty cells even if you have *only* continuous variables (treated
as continuous) in the model. For that reason, I always did as you
suggest for any categorical variables in the model--i.e., crosstabulate
them to see how many cells really are empty.

Cheers,
Bruce

[agnese....@gmail.com]

Hi Bruce, I'm also having the same problem. Even when the only independent variables are continuous and entered as covariates, I get the empty cell warning and it seems like SPSS is trying to crosstabultate them. So are you suggesting to ignore the warning if the only independent variables are continuous, or if the independent variabels are categorical and crosstabulations of each categorical independent variable with the dependent variable show that the expected frequencies are ok (none that are 0 and less than 20% that are below 5)? Also, what post by Frank Harrel were you referring to? I could not follow through the link. Thanks so much!
Agnese

[Ray Koopman]

On May 3, 5:30 am, agnese.marin...@gmail.com wrote:
> Il giorno giovedì 23 agosto 2007 15:23:29 UTC+2, Bruce Weaver ha scritto:
>
>
>
> > Art Kendall wrote:
> > > use CROSSTABS to see how many cells there should be and if in fact that
> > > many cells are empty.
> > > Is it possible you are treating a continuous variable as nominal?  Or
> > > that you have nominal level variables with many values?
>
> > Art, it has been my experience in the past that NOMREG gives the warning
> > about empty cells even if you have *only* continuous variables (treated
> > as continuous) in the model.  For that reason, I always did as you
> > suggest for any categorical variables in the model--i.e., crosstabulate
> > them to see how many cells really are empty.
>
> > Cheers,
> > Bruce
>
> > --
> > Bruce Weaver

> > bwea...@lakeheadu.ca

> >www.angelfire.com/wv/bwhomedir
> > "When all else fails, RTFM."
>
> Hi Bruce, I'm also having the same problem. Even when the only independent variables are continuous and entered as covariates, I get the empty cell warning and it seems like SPSS is trying to crosstabultate them. So are you suggesting to ignore the warning if the only independent variables are continuous, or if the independent variabels are categorical and crosstabulations of each categorical independent variable with the dependent variable show that the expected frequencies are ok (none that are 0 and less than 20% that are below 5)? Also, what post by Frank Harrel were you referring to? I could not follow through the link. Thanks so much!
> Agnese

http://groups.google.com/group/comp.soft-sys.sas/msg/9cdd75080498697e?hl=en

[Rich Ulrich]

On Fri, 3 May 2013 10:15:35 -0700 (PDT), Ray Koopman <koo...@sfu.ca>
wrote:

>On May 3, 5:30�am, agnese.marin...@gmail.com wrote:

Okay, Ray, how did you find that?

That's an interesting thread, though the subject is not
the warning about cells.

I used Google's advanced search for groups (which is
now somewhat well hidden) to find the post by
Bruce in 2007. But that one included a message ID
in a form that I my old stats FAQ used to report; and
which (it seems) that Google no longer accepts as a
"message ID".

Is there a way to search on that?

--
Rich Ulrich

[Bruce Weaver]

Rich, I thought the answer to your question was straight forward until I
tried it! In the OLD Google Groups, it was obvious how to get to the
Advanced Search Page, where you can search on message ID. In the NEW
Google Groups, it is not so obvious how to get to the Advanced Search
page. Rather than spending too much time looking for it, I Googled
<Google Groups advanced search>, and clicked on the link that came up in
the results. ;-)

--
Bruce Weaver
bwe...@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/Home

[Ray Koopman]

On May 3, 1:33 pm, Bruce Weaver <bwea...@lakeheadu.ca> wrote:
> In the OLD Google Groups, it was obvious how to get to the
> Advanced Search Page, where you can search on message ID.

In the old google groups (which I use because my browser won't run
the new version), the only way I know to get to the Advanced Search,
which has a [Lookup Message ID] button, is to search for something
-- it doesn't matter what -- using the [Search Groups] button,
not the [Search this group] button. That brings up a page that
has Advanced Search as an option, next to the [Search] button.
That's "obvious"?

[Bruce Weaver]

Believe it or not, Ray, that's more obvious than what I see in the NEW
(and not so improved) Google Groups! I just did a search on <spss>, and
see this in the results window:

Search results for spss
Sorted by relevance | Sort by date

To be sure I wasn't just missing an Advanced Search link, I hit CTL-F
and searched the page. "Advanced Search" is not found (with or without
capital letters)! So it would appear that you can't get there from
here. My Google Search on <Google Groups advanced search> seems to be
the best way. You have to love progress. :-)

[Rich Ulrich]

[see lower down, for Retry-Report]

But, fellas, that wasn't my problem! Bruce's search gets me
https://groups.google.com/a/euroaquae.eu/advanced_search
- which includes a message that old Google Groups is going away,
which I don't see with the search that I have been using.

I've been regularly using a less compact, slightly wordier version
than that. This is at
https://groups.google.com/advanced_search?hl=en&q=&hl=en&

I type "advanced" in my Firefox address pane, and it "recalls" the
last time that I used it. I don't know how long that memory is, but
I've been okay so far.

Both pages have a box at the bottom for Message ID and a
button for Lookup message. Both of them show an example.
But the example of the ID is not like the one from Bruce's post
in 2007. It looks different, and when *I* tried it, nothing was
found. I also tried putting the ID into the general seach field,
just for hope, but that didn't work either.



Retry-Report. It does seem that I have just been running into
some of the flakiness of Group searches. (I've seen reports in
alt.usage.english of other sporadic failures. I didn't know how
common they were, but it now seems like "somewhat." )

I repeated what I had searched earlier today-- I found Bruce's 2007
note by using Advanced search on <Multinomial Regression> in Subject,
and then I pasted 20030610142629.2...@virginia.edu
into the Message ID window. It worked it this time - on a second try.
On the second try, I checked the "Any time" box instead of setting
a year range. But that upper part of the form probably should not
be relevant. Earlier in the day, neither way worked.

So I tried the same thing with the search page from Bruce.
That page failed to find (even) the 2007 thread. Then, pasting in
the Message ID that I found before, that page failed to return the
note by its Message ID.

I assume that if I try a few more times, I will sometimes find
the post, and sometimes not.

But it is good to know that the old Message ID form still works,
even if it won't be all the time.


--
Rich Ulrich

[parveenif...@gmail.com]

i have the same problem dear.have you got any solution for this problem.kindly share with me about it.

[dudipr...@gmail.com]

There are 2042 (78.2%) cells (i.e., dependent variable levels by subpopulations) with zero frequencies.
i got this warning while doing with multinomial logit model. i am using 705 no. customers data. is it okay? or have to increase or decrease the data? what is the min. and max. no. to be used?
some ordinal variables and dummy variables are there, and i considered them as covariates such as income, age, education, platoon size,and another variable taking from strong dissatisfaction to strong satisfaction. is it correct or have to change them into factors?

[dudipr...@gmail.com]

and also, in the final intercepts, and coefficients table, i'm getting the significance value >.05 for max of variables. what t do?

[dudipr...@gmail.com]

This pivot table will not be produced because the total number of pivot table cells (including the empty cells) exceeds 5407505. This limit can be changed by using the SET MXCELLS command.

[Rich Ulrich]

On Wed, 3 Feb 2016 20:55:38 -0800 (PST), dudipr...@gmail.com wrote:

>On Friday, October 17, 2014 at 11:35:49 PM UTC-7, parveenif...@gmail.com wrote:
>> On Wednesday, August 22, 2007 1:06:31 PM UTC-7, Sheryl wrote:
>> > I received the following warning:
>> >
>> > There are 134 (59.6%) cells (i.e., dependent variable levels by
>> > subpopulations) with zero frequencies.
>> >
>> > I also received the following comment:
>> > The dependent variable has only one value observed in 62 (82.7%)
>> > subpopulations.
>> >
>> > This topic of the warning (but not the comment) came up in 2003 - the

.
.

>> > Sheryl
>>
>> i have the same problem dear.have you got any solution for this problem.kindly share with me about it.
>
>and also, in the final intercepts, and coefficients table, i'm getting the significance value >.05 for max of variables. what t do?

[reply that I sent by email, too]

Subject: Re: Multinomial Regression Warning
From: dudipr...@gmail.com

You posted 3 messages to the Usenet SPSS group, each with
the small fraction of a proper question.

Since those were in Reply to posts from 2007, I suspect that
you found the original posts by Googling, and have no idea
what a Usenet group is. In short: If you post there, any
Reply will be posted there; you will only see it if you are
reading the group for answers, over the next week or two.

It is NOT a service with email replies.

If you do want an answer, in the group, please post one
message that is coherent. But I expect that your analysis
should drop all higher order interaction terms, assuming
that your cells arise from that problem.

--
Rich Ulrich