AR(1) Modeling

[Vishv Jeet]

Consider the following simple AR(1) model;

data {

  int T;

  ...

}

parameters {

  vector eps[T];

  real<lower=0> phi;

  real<lower=0> sigma;

}

model {

  eps[1] ~ normal(0, sigma);

  tail(eps, T-1) ~ normal(phi * head(eps, T-1), sigma); // AR(1) modeling suggested in STAN reference

  ....

}

My question is: phi * head(eps, T-1) is a nonlinear function of two parameters, so even though I am not directly sampling it but shouldn't this need a Jacobian adjustment because eps is not data but a parameter itself.

regards,

vishy

[Ben Goodrich]

That is technically correct about the Jacobian but that is not generally how we would suggest to model an AR(1) process for _parameters_ (as opposed to data). It is usually better to do

data {

 int T;

 ...

}

parameters {

  real eps_0;
  vector[T - 1] eps_raw;

  real<lower=0> phi;

 real<lower=0> sigma;

}

model {
  vector[T] eps;
  eps[1] = eps_0;
  for (t in 2:T)
    eps[t] = phi * eps[t - 1] + sigma * eps_raw[t - 1];

  target += normal_lpdf(eps_0 | 0, sigma * sqrt(1 - square(phi)));
  target += normal_lpdf(eps_raw | 0, 1);
  ...
}

In this parameterization, the primitive parameters eps_raw are iid standard normal a priori, which makes it much easier for Stan to move around than if eps were a primitive parameter. The reason to treat eps_0 separately is that sigma is the standard deviation of the errors _conditional on on eps[t - 1]_. The marginal standard deviation is deflated by sqrt(1 - square(phi)).

Ben

[Vishv Jeet]

Thank you Ben. This is very helpful. Just curious, was the way I had written the model wrong or computationally inefficient?

[Ben Goodrich]

On Thursday, December 15, 2016 at 2:47:01 PM UTC-5, Vishv Jeet wrote:

Thank you Ben. This is very helpful. Just curious, was the way I had written the model wrong or computationally inefficient?

It was only wrong in the prior for the initial period was inconsistent with the generative model for the remaining periods. But it presumably would have had very low effective sample sizes for the reason I gave
 

In this parameterization, the primitive parameters eps_raw are iid standard normal a priori, which makes it much easier for Stan to move around than if eps were a primitive parameter.

For large values of phi, it would be very difficult for the individual elements of eps to move freely throughout the posterior distribution, which does not occur if eps_raw is the primitive parameter.

Ben