Specify heteroscedastic noise model

[william....@gmail.com]

Is there a means of specifying a particular heteroscedastic noise model? For example, if I want a linear model with linear heteroscedastic noise such as:

Y_i = \beta*X_i + \epsilon_i

\epsilon_i ~ N(0, X_i*\gamma)

is there a way to specify that noise model within a statsmodels OLS function?

Thank you!

[josef...@gmail.com]

WLS has the weights option with the interpretation of inverse variance for heteroscedasticity, but it takes the weights as given and doesn't estimate any variance parameters.

There is a statsmodels.regression.feasible_gls.GLSHet model that alternates between estimating the WLS model with fixed weights and estimating the variance function `res_resid = OLS(self.link(results.resid**2), self.exog_var).fit()`

It has zero unit test coverage, mainly because at the time I didn't find any equivalent model in R or Stata. The implementation is a standard econometrics text book version. In R I only saw a full gaussian MLE version.

We currently don't have a full MLE version, MixedLM and some statespace models use MLE for variance estimation, but I think none of them would have this as special case.

abbreviated historical note

At the time (2011) my background was almost exclusively econometrics where full MLE is considered to be too restrictive.

In the mean time it became apparent that the trend is going towards heteroscedasticity robust standard errors which became higher priority

I am reading up and making plans for adding heteroscedasticity models through GLM as prototypical case, double exponential or similar models, with both full MLE and estimating equation/WLS/GLS versions, we are slowly working our way to get there. (Slow because there are many other directions for enhancements and improvements and so far there wasn't a large demand.)

Josef

 

Thank you!

[william....@gmail.com]

Thanks Josef!

Since the linear heteroscedastic model has an analytic likelihood, could I create a custom Likelihood Model, as described in http://www.statsmodels.org/stable/examples/notebooks/generated/generic_mle.html? 

[josef...@gmail.com]

On Wednesday, July 19, 2017, <william....@gmail.com> wrote:

Thanks Josef!

Since the linear heteroscedastic model has an analytic likelihood, could I create a custom Likelihood Model, as described in http://www.statsmodels.org/stable/examples/notebooks/generated/generic_mle.html? 

 

Yes, there should might still be a draft version somewhere for normal mle without heteroscedasticity.

Note, computationally its easier and more robust to concentrate out the linear part with WLS and use nonlinear optimization for the variance parameters.

The complete mle would require less code.

Josef

[josef...@gmail.com]

I just saw

New in Stata 15: hetregress with least squares and MLE (assuming normality and correct specification)

http://www.stata.com/manuals/rhetregress.pdf

(I have Stata 14 and will skip 15)

aside: Stata 15 has some other goodies where we are not able to keep up.

it also looks like they finally got more serious nonparametric regression, and they have some areas where they try to catch up to open source.

Extended regression models, ERM, looks like a nice new super class for models.

unpaid advertising: I like Stata especially with open access manuals (as long as the limitations of Stata are not seriously binding)

Josef