Decorrelate data

[chrims]

I employ ordinary least squares to model the relationship between to variables.
The Durbin-Watson test shows however strong evidence of positively correlated residuals.

1) How can I access the eigenvectors of the covariance matrix? (to manually decorrelate data)
2) Is there an easy, straightforward way to decorrelate data in statsmodels?

[josef...@gmail.com]

On Thu, Jun 18, 2015 at 6:14 PM, chrims <stabi...@web.de> wrote:

I employ ordinary least squares to model the relationship between to variables.
The Durbin-Watson test shows however strong evidence of positively correlated residuals.

1) How can I access the eigenvectors of the covariance matrix? (to manually decorrelate data)

I don't think we have this in any of the code.

The problem is that the covariance matrix for all observations is (nobs, nobs) which is usually too large, and we couldn't estimate it from the sample without additional assumptions.

 

2) Is there an easy, straightforward way to decorrelate data in statsmodels?

kind of, decorrelation can be done by fitting an AR model, and look at the residuals.

But if you assume that the errors are autocorrelated, then you would need to use the same AR coefficients to filter endogenous and all explanatory variables.

This is exactly what the `whiten` method of GLSAR does.

Backing up a bit:

OLS is still a consistent estimator of the mean parameters even if there is autocorrelation, but we need to use autocorrelation AC or heteroscedasticity and autocorrelation HAC robust standard errors for the parameter estimates to get those correct. (available with fit(cov_type='hac'

ARX and ARMAX estimate a model with autocorrelated errors that follow either a AR or a ARMA model.

GLSAR is essentially a "cheaper" version of ARX, with slightly different behavior in small sample 

(because GLSAR is a 2-stage estimator and conditions on initial conditions. I never compared results since ARMA gained the "X".) 

So, I'm not sure we expose the pieces to do it manually, but we have the models for autocorrelated errors.

Josef

[josef...@gmail.com]

On Thu, Jun 18, 2015 at 6:39 PM, <josef...@gmail.com> wrote:

On Thu, Jun 18, 2015 at 6:14 PM, chrims <stabi...@web.de> wrote:

I employ ordinary least squares to model the relationship between to variables.
The Durbin-Watson test shows however strong evidence of positively correlated residuals.

1) How can I access the eigenvectors of the covariance matrix? (to manually decorrelate data)

I don't think we have this in any of the code.

The problem is that the covariance matrix for all observations is (nobs, nobs) which is usually too large, and we couldn't estimate it from the sample without additional assumptions.

To be a bit more precise:

The AR filter in GLSAR represents the Cholesky decomposition of the (nobs, nobs) covariance matrix (ignoring initial conditions in our version IIRC) but never constructs the matrix. This is under the assumption that the process is an AR process.

[josef...@gmail.com]

On Thu, Jun 18, 2015 at 6:44 PM, <josef...@gmail.com> wrote:

On Thu, Jun 18, 2015 at 6:39 PM, <josef...@gmail.com> wrote:

On Thu, Jun 18, 2015 at 6:14 PM, chrims <stabi...@web.de> wrote:

I employ ordinary least squares to model the relationship between to variables.
The Durbin-Watson test shows however strong evidence of positively correlated residuals.

1) How can I access the eigenvectors of the covariance matrix? (to manually decorrelate data)

I don't think we have this in any of the code.

The problem is that the covariance matrix for all observations is (nobs, nobs) which is usually too large, and we couldn't estimate it from the sample without additional assumptions.

To be a bit more precise:

The AR filter in GLSAR represents the Cholesky decomposition of the (nobs, nobs) covariance matrix (ignoring initial conditions in our version IIRC) but never constructs the matrix. This is under the assumption that the process is an AR process.

I'm faster in typing than thinking.

IIRC, it represents the Cholesky decomposition of the inverse of the covariance matrix.

Jose

[chrims]

OLS is still a consistent estimator of the mean parameters even if there is autocorrelation, but we need to use autocorrelation AC or heteroscedasticity and autocorrelation HAC robust standard errors for the parameter estimates to get those correct. (available with fit(cov_type='hac'

Yes. I applied HAC, but expected to see a change in the DW test statistic?
How do I see then any "effect" of employing HAC robust standard errors besides the change in the standard errors themselves?

[josef...@gmail.com]

Changing to robust standard errors doesn't change the parameter estimates, and so it won't change prediction, nor residuals, and it will not remove the autocorrelation in the residuals.

The only thing that is supposed to change are standard errors and the related inference, like t_test and confidence intervals.

The point is that it doesn't matter for consistency and correct inference that there is autocorrelation, and a significant DW statistic or other autocorrelation test.

If we want to improve the estimator (increase efficiency, lowering variance of parameter estimates), then we need to use either ARX, ARMAX or GLSAR. With those estimator the parameter estimates will change and the residuals of the whitened, transformed model should not have the autocorrelation left.

(Another possibility is to use a dynamic model, for example add some lagged explanatory variables which might capture the autocorrelation in the data directly instead of treating it through the residual.)

Josef

[chrims]

The point is that it doesn't matter for consistency and correct inference that there is autocorrelation, and a significant DW statistic or other autocorrelation test.

Then if I apply (H)AC how do I decide which number of maxlags is best? Solely on the minimization of the standard errors?

[josef...@gmail.com]

On Fri, Jun 19, 2015 at 9:52 AM, chrims <stabi...@web.de> wrote:

The point is that it doesn't matter for consistency and correct inference that there is autocorrelation, and a significant DW statistic or other autocorrelation test.

Then if I apply (H)AC how do I decide which number of maxlags is best? Solely on the minimization of the standard errors?

I'm not sure, or don't remember the details

It needs to be large enough that it captures all the residual correlation. 

It uses a kernel that downweights the correlation at large lags and might/should not be very sensitive to the exact choice.

Stata's newey requires the user to specify the number of lags, and doesn't say how large.

matlab recommends/uses maxLag = floor(4*(T/100)^(2/9));

http://www.mathworks.com/help/econ/hac.html

which we use also in some other places (hypothesis tests for autocorrelation)

There are some procedures to estimate or autoselect it from the data, IIRC, but that's not implemented.

Newer literature goes all the way to maxlag = nobs - 1 in some cases, which has in general better properties, but the covariance might not be positive definite (with a flat top kernel that they also recommend).

(I read the literature and small sample simulation studies carefully for HC and cluster robust standard errors, but not for HAC.)

How long are your time series and what kind of data is it?

Josef