In some cases, the maximum likelihood estimator might not exist, parameters might be infinite or not unique (e.g. (quasi-)separation in models with binary endogenous variable). Under the default settings, statsmodels will print a warning if the optimization algorithm stops without reaching convergence. However, it is important to know that the convergence criteria may sometimes falsely indicate convergence (e.g. if the value of the objective function converged but not the parameters). In general, a user needs to verify convergence."
How does one check for convergence? What is driving whether a model converges or not?
On May 11, 2021, at 12:30 PM, jordan....@gmail.com <jordan....@gmail.com> wrote:
Hello,
--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/201d5264-7393-46bb-aa04-203d02a50558n%40googlegroups.com.
There is an iterative process.For IRLS fits, You can either check that the deviance converges or the Params converge. Note I think default convergence criterion assumes a smaller dataset. Not sure how big yours is.I’d recommend trying method=‘newton’ or method=‘lbfgs’ along with optim_hessian=‘eim’. Experiment with the options… depends on the data …optim_hessian=‘oim’ assumes the data follow the family selected. Crazy corner cases might hurt you.On May 11, 2021, at 12:30 PM, jordan....@gmail.com <jordan....@gmail.com> wrote:Hello,At my place of employment, we're trying to put a proprietary software to bed and just use statsmodels.Between the two (willis towers watson and pystatsmodels), using the same model (Poisson GLM), and same data, we're getting convergence warnings in pystatsmodels.
I noticed the following on the documents page:"ncomplete convergence in maximum likelihood estimation¶In some cases, the maximum likelihood estimator might not exist, parameters might be infinite or not unique (e.g. (quasi-)separation in models with binary endogenous variable). Under the default settings, statsmodels will print a warning if the optimization algorithm stops without reaching convergence. However, it is important to know that the convergence criteria may sometimes falsely indicate convergence (e.g. if the value of the objective function converged but not the parameters). In general, a user needs to verify convergence."
How does one check for convergence? What is driving whether a model converges or not?
----
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/201d5264-7393-46bb-aa04-203d02a50558n%40googlegroups.com.
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/13DD278B-17AC-4AB8-B639-2DCFBB91DE70%40gmail.com.
On May 11, 2021, at 3:07 PM, josef...@gmail.com wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/CAMMTP%2BCpmdS-g6qvVFy7SfLGq2r-q8AmciRRRuAq-LN7PWgeOQ%40mail.gmail.com.
I’m not picking the method so I guess whatever is the default.
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/908D4C09-DCF0-41EE-B773-B409E638211E%40gmail.com.