What do the ARMAX.fit method options mean?

[yo lo]

Hello everyone,

I am trying to fit an ARMAX model (more of a MAX model, I don't have autoregression) to some data. There are 6 options :

• statespace
• innovations_mle
• hannan_rissanen
• burg
• innovations
• yule_walker

We're told that yule_walker only works on pure AR models so I didn't try it. I tried the other 5 options and the two that stood out and gave me the results I expected were innovations and innovations_mle. Burg failed because it is also unsuited to exogenous input.

I would like to understand why some methods work and some other don't so my question is : what do the innovations, innovations-MLE and statespace methods do? Because in all honesty it is pretty hard to find a description of an algorithm with such generic key-words.

[Chad Fulton]

Hello,

The primary reference for most of these is the book "Introduction to Time Series and Forecasting" by Brockwell and Davis (2016) (although these are common methods that you could find in most time series textbooks); the section numbers are below.

Four of the methods are essentially moment estimators (i.e. they use sample means and covariances as estimators for population moments) that use a least-squares principle.  They differ in exactly which moments are estimated, and as a result some are only applicable to certain models.

- Yule-Walker estimation: section 5.1.1 (only for AR(p) models)

- Burg's method: section 5.1.2 (only for AR(p) models)

- Innovations method: section 5.1.3 (only for MA(q) models)

- Hannan-Rissanen method: section 5.1.4. (allows for ARMA(p, q) models)

Compared to the maximum likelihood estimators (described below), these four estimators can be less efficient, they can be numerically problematic in some cases, and they do not allow for estimating as large a class of models.  However, if they are applicable, they can be fast and provide good estimates.

Except for the the Hannan-Rissanen method, these methods do not allow incorporating regression effects (i.e. exogenous variables).  So if you specify one of these methods for a model with exogenous variables, Generalized Least Squares (GLS) will be used in combination with the estimator (see e.g. section 6.6.1 of the Brockwell and Davis book). In addition, these models do not allow for non-stationary models, so if you specify a integrated model, then the data will be differenced to produce a stationary model before the estimators are applied.

The other two methods are likelihood-based estimators - i.e. they attempt to find parameters that maximize the likelihood function.  Because the likelihood function is not available analytically, we have to numerically maximize the likelihood function.  The two methods differ in how they evaluate the likelihood function for a given set of parameters:

- Innovations MLE (maximum likelihood estimation) method: section 5.2

- "statespace": this method casts the ARIMA model into the form of a linear gaussian state space model, and uses the Kalman filter to compute the likelihood function (see the book "Time series analysis by state space methods" by Durbin and Koopman (2012) for details about these models).

The innovations method does not allow for exogenous variables or non-stationary models, so the above notes (using GLS and differenced data) apply here.

The "statespace" method is the most general approach, and, for example, allows directly incorporating non-stationary models and exogenous variables without resorting to differencing the data or using GLS.

One drawback of the maximum likelihood based estimators is that since they are numerically maximizing the likelihood function, they require a "first guess" about the parameters, which they then iteratively improve on.  If the "first guess" is bad (i.e. far away from the maximum of the likelihood function) or the likelihood function is either flat or has lots of local maxima, then these methods can fail - in that sense, they can be less robust than the simpler estimators described earlier.

The default approach is to use the Hannan-Rissanen method to compute a set of "first guess" parameters, and then use the "statespace" method to improve the parameters by numerically maximizing the likelihood function.

In summary, there is no single estimator that "always works" or is "always the best", and specific issues with certain estimators, such as what you've described, can only be analyzed on a case-by-case basis.

Best,

Chad

--
You received this message because you are subscribed to the Google Groups "pystatsmodels" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pystatsmodel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pystatsmodels/f2fc277d-00be-4559-ad92-3f38241f28f2n%40googlegroups.com.