pystatsmodels
https://groups.google.com/d/forum/pystatsmodels
We are discussing the implementation of statistical and econometrics models and methods of data handling in Python using statsmodels.enRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/m9-mD-4ZAgAJ
Better to not think about the election, when you have nice python packages to think about! The groups are settings of explanatory variables. The parameters are not inf, but some of them are enormous (10**36), and obviously the fit to the data is very bad, so that's what I mean by failed. If Ihttps://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
Rachel MelamedThu, 20 Oct 2016 20:38:16 UTCRe: [pystatsmodels] Looping through regressions
https://groups.google.com/d/msg/pystatsmodels/IAuJD3aB8lc/Y7oZCrzdAQAJ
On Wed, Oct 19, 2016 at 8:10 PM, Robert Garrison II <garrison...@gmail.com> wrote: > I understand that this may be an elementary question, but bear with me. > > I have a 16M record dataframe, and I want to perform regressions on 60 > subsets of the dataframe defined by Variable_A. I created a listhttps://groups.google.com/d/topic/pystatsmodels/IAuJD3aB8lc
josefpktdThu, 20 Oct 2016 02:15:11 UTCLooping through regressions
https://groups.google.com/d/msg/pystatsmodels/IAuJD3aB8lc/4ZFlCu3WAQAJ
I understand that this may be an elementary question, but bear with me. I have a 16M record dataframe, and I want to perform regressions on 60 subsets of the dataframe defined by Variable_A. I created a list of the subsets (e.g. subsets=list(set(df['Variable_A'])) ), that I want to iteratehttps://groups.google.com/d/topic/pystatsmodels/IAuJD3aB8lc
Robert Garrison IIThu, 20 Oct 2016 00:10:25 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/-l6v86buAAAJ
On Wed, Oct 12, 2016 at 11:04 AM, Rachel Melamed <rachel....@gmail.com> wrote: > Hi Josef, > I'm not sure if this is the same issue, but I ran into a case where the > parameter estimation totally failed, and the deviance is again nan, and the > pearson chi2 value is super high > > Dep. Variable:https://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
josefpktdMon, 17 Oct 2016 01:13:57 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/-Lo2CUsPEQAJ
Hi Josef, I'm not sure if this is the same issue, but I ran into a case where the parameter estimation totally failed, and the deviance is again nan, and the pearson chi2 value is super high Dep. Variable: ['success', 'fail']No. Observations: 16069 Model: GLMDf Residuals: 15481 Model Family:https://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
Rachel MelamedWed, 12 Oct 2016 15:04:56 UTCRe: [pystatsmodels] ARMA parameters + sigma2 covariance matrix
https://groups.google.com/d/msg/pystatsmodels/oXofQ6L-dr8/tybQw7iTEAAJ
Yes, for better or worse we don't concentrate out the scale parameter in state space, so all state space models, e.g. SARIMAX, will compute the scale via MLE and will return the full covariance matrix.https://groups.google.com/d/topic/pystatsmodels/oXofQ6L-dr8
Chad FultonTue, 11 Oct 2016 01:20:28 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/06IIkOaVDwAJ
On Fri, Oct 7, 2016 at 3:21 PM, Rachel Melamed <rachel....@gmail.com> wrote: > I'm happy to send you the data if it will help. > Thanks, not necessary, I found an old testcase for when we fixed the mu=0 corner case. There is also already a PR for this where I don't know why it got ignored (norhttps://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
josefpktdFri, 07 Oct 2016 19:49:09 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/UrQzJV-UDwAJ
I'm happy to send you the data if it will help. On Friday, October 7, 2016 at 2:45:47 PM UTC-4, josefpktd wrote: > > > > On Fri, Oct 7, 2016 at 2:21 PM, Rachel Melamed <rachel....@gmail.com> > wrote: > >> Ah, I took out that one point and no more nan. Ok, I guess that answers >> my question,https://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
Rachel MelamedFri, 07 Oct 2016 19:21:08 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/X9jDbS-UDwAJ
14 of the parameters have a change that is > 10**-3, but none of those have a change that is greater than 10**-2. All of my predictors are categorical, and parameters affected are for very sparsely observed categories. Rachel On Friday, October 7, 2016 at 2:45:47 PM UTC-4, josefpktd wrote: > >https://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
Rachel MelamedFri, 07 Oct 2016 19:17:43 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/Knf9dnGSDwAJ
On Fri, Oct 7, 2016 at 2:21 PM, Rachel Melamed <rachel....@gmail.com> wrote: > Ah, I took out that one point and no more nan. Ok, I guess that answers > my question, sort of. > follow-up to this: Did the parameters change by more than in the 3rd decimal or so? I'm trying to figure out ifhttps://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
josefpktdFri, 07 Oct 2016 18:45:47 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/8WdX-D6SDwAJ
On Fri, Oct 7, 2016 at 1:58 PM, Rachel Melamed <rachel....@gmail.com> wrote: > Thank you for the reply. > Version: 0.6.1 > Min: 2.649e-10 > Max: 1.0 > > There is one observation where the mu = 1.0, and this has [success, fail] > = [1, 0] > Do you think that is the problem? > Yes, when I lookedhttps://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
josefpktdFri, 07 Oct 2016 18:42:10 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/hPEs3SKRDwAJ
Ah, I took out that one point and no more nan. Ok, I guess that answers my question, sort of. Thanks On Friday, October 7, 2016 at 1:58:29 PM UTC-4, Rachel Melamed wrote: > > Thank you for the reply. > Version: 0.6.1 > Min: 2.649e-10 > Max: 1.0 > > There is one observation where the mu = 1.0,https://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
Rachel MelamedFri, 07 Oct 2016 18:21:50 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/WIJgjdyPDwAJ
Thank you for the reply. Version: 0.6.1 Min: 2.649e-10 Max: 1.0 There is one observation where the mu = 1.0, and this has [success, fail] = [1, 0] Do you think that is the problem? Thanks again! On Friday, October 7, 2016 at 12:30:46 PM UTC-4, josefpktd wrote: > > > > On Fri, Oct 7, 2016 athttps://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
Rachel MelamedFri, 07 Oct 2016 17:58:29 UTCRe: [pystatsmodels] nan deviance in binomial model
https://groups.google.com/d/msg/pystatsmodels/B4M2Jr3yPFc/aL64JROLDwAJ
On Fri, Oct 7, 2016 at 12:19 PM, Rachel Melamed <rachel....@gmail.com> wrote: > Hello, > > I'm running a binomial regression and I can't figure out why the fitted > model deviance is "nan" > > Model = sm.GLM(endog=success_fail, exog=design, > family=sm.families.Binomial()).fit() > > Model.summahttps://groups.google.com/d/topic/pystatsmodels/B4M2Jr3yPFc
josefpktdFri, 07 Oct 2016 16:30:46 UTCRe: [pystatsmodels] ARMA parameters + sigma2 covariance matrix
https://groups.google.com/d/msg/pystatsmodels/oXofQ6L-dr8/rGd5woaKDwAJ
wrote: >> Hello, >> >> I am fitting data to an AR model and I would like to get estimates of errors >> and covariance matrix of the parameters. >> >> For an AR(5) model I do something like this: >> >> arma_mod = sm.tsa.ARMA(y, order=(5,0)) >> arma_res = arma_mod.fit(trend='nc') >> >> andhttps://groups.google.com/d/topic/pystatsmodels/oXofQ6L-dr8
josefpktdFri, 07 Oct 2016 16:20:43 UTC