WLS (weighted least squares) error independent of weights

[Dan Parshall]

I'm using Python's statsmodels to perform a weighted linear regression. Since this is my first time with this module, I ran some basic tests.

Doing these, I find that errors on the parameters are independent of the weight. This doesn't fit my naive expectation (that larger error bars would produce a more uncertain result), or the definition I've seen in nonlinear fitting, which generally uses the weighted Hessian to report the uncertainty of the parameters.

I couldn't find an explicit description of the calculation of the parameter-errors (either in the statsmodels docs or support pages).  I've posted on 

Minimal code:

import numpy as np

import statsmodels.api as sm

escale = 1

xvals = np.arange(0,11)

yvals = xvals**2                         # dummy nonlinear y-data

evals = escale*np.sqrt(yvals + escale)   # errorbars

X = sm.add_constant(xvals)               # to get intercept term

model = sm.WLS(yvals, X, 1.0/evals**2) # weight inverse to error

result = model.fit()

print "escale = ", escale

print "fit = ", result.params

print "err = ", result.bse        # same value independent of escale

[josef...@gmail.com]

(Reply by ipad with no proper keyboard)

Wls params and cov_params are independent of the scale of the weights because the scale cancels for params and the cov_params contains rescaling by an estimate of the error variance.

Current master (iirc since 0.7) has a cov_type option in fit to fix the scale so that the scale is given by the weights. There is a notebook for it in the docs.

If needed, i can find the details and usage tonight central european time.

Josef

[josef...@gmail.com]

On Mon, Jul 18, 2016 at 10:09 AM, <josef...@gmail.com> wrote:
> (Reply by ipad with no proper keyboard)
>
> Wls params and cov_params are independent of the scale of the weights
> because the scale cancels for params and the cov_params contains rescaling
> by an estimate of the error variance.
>
> Current master (iirc since 0.7) has a cov_type option in fit to fix the
> scale so that the scale is given by the weights. There is a notebook for it
> in the docs.
>
> If needed, i can find the details and usage tonight central european time.

http://www.statsmodels.org/devel/examples/notebooks/generated/chi2_fitting.html
It looks like this is missing from the index of notebook examples

it looks like two spellings are allowed

if cov_type in ['fixed scale', 'fixed_scale']

and here's an example as test case
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/regression/tests/test_robustcov.py#L853


I hope that helps

Josef