HMM for animal movement spending too long in each state

175 views
Skip to first unread message

Ben Lambert

unread,
Jul 19, 2016, 2:02:04 PM7/19/16
to Stan users mailing list
Hello,

I have been working with a HMM model where animal movement (distance moved, and relative angle) depends on a latent state. The first state is 'foraging' where the animal moves short distances, and frequently changes its movement direction between steps. The other state is 'migrating' where the animal moves with purpose in a given direction, characterised by long steps and small changes in its relative angle.

Bob very kindly helped us to get a model up and running using dynamic programming. I have been comparing this model with fake data, to see how it fares. At the moment the model seems to be (quite strongly) overestimating the time spent in state 1 (Bob mentioned this in his original posting of the model.) For some reason the switching probabilities for state 2 appear to be estimated fairly faithfully.

My question is: is there anything we can do to remedy this issue?

At the moment we have tried the following:

a. Setting priors for the switching probabilities that are concentrated towards 0.5. This helps a bit, but the priors need to be very concentrated for us to get estimates that are close to the true values.
b. If instead we supply the true switching probs as data, we still see that the estimated latent vector tends to get stuck. 
c. Supplying (as data) the true values for rho, alpha and mu, and using the resultant model to estimate the latent state vector. It still seems to get stuck in a given state!

Does anyone have any other ideas? I include the model below:

functions{
  ## Function that returns the log pdf of the wrapped-Cauchy
  real wrappedCauchy(real aPhi, real aRho, real aMu) {
    return(- log(2*pi()) + log((1-aRho^2)/(1+aRho^2-2*aRho*cos(aPhi-aMu))));
  }
  
}


data {
  int<lower=0> N;
  vector<lower=-pi(), upper=pi()>[N] phi;
  vector<lower=0>[N] l;
  int<lower=1> K;
  vector<lower=0>[K] theta_priors;
}
parameters {
  simplex[K] theta[K]; ## Switching parameter
  vector<lower=-pi(), upper=pi()>[K] mu; ## Location of wrapped Cauchy
  vector<lower=0,upper=1>[K] rho; ## Scale of wrapped Cauchy
  positive_ordered[K] alpha; ## Exponential parameters
}  
model {
  vector[K] log_theta[K];
  vector[K] lp;

  for (k in 1:K) log_theta[k] = log(theta[k]);
  lp = rep_vector(-log(K), K); ## Initial state log-probability
  for (n in 1:N) {
    for (k in 1:K)
      lp[k]
        = log_sum_exp(log_theta[k] + lp)
        + wrappedCauchy(phi[n], rho[k], mu[k])
        + exponential_lpdf(l[n] | alpha[k]);
  }
  target += log_sum_exp(lp);

  alpha[1] ~ normal(0.001,0.0001); ## Priors concentrated near to true values
  alpha[2] ~ normal(0.005,0.0001);

  ## Supply theta prior parameters as data
  for(i in 1:K){
    theta[i] ~ dirichlet(theta_priors);
  }
  
}


generated quantities {
  int<lower=1,upper=K> y_star[N];
  real log_p_y_star;
  {
    int back_ptr[N, K];
    real best_logp[N, K];
    real best_total_logp;
    for (k in 1:K)
      best_logp[1, K] = wrappedCauchy(phi[1], rho[k], mu[k])
                      + exponential_lpdf(l[1] | alpha[k]);
    for (t in 2:N) {
      for (k in 1:K) {
      best_logp[t, k] = negative_infinity();
        for (j in 1:K) {
          real logp;
          logp = best_logp[t-1, j]
          + log(theta[j, k]) + wrappedCauchy(phi[t], rho[k], mu[k])
                      + exponential_lpdf(l[t] | alpha[k]);
          if (logp > best_logp[t, k]) {
            back_ptr[t, k] = j;
            best_logp[t, k] = logp;
          }
        }
      }
    }
    log_p_y_star = max(best_logp[N]);
    for (k in 1:K)
      if (best_logp[N, k] == log_p_y_star)
      y_star[N] = k;
      for (t in 1:(N - 1))
      y_star[N - t] = back_ptr[N - t + 1,
      y_star[N - t + 1]];
  }
}

Bob Carpenter

unread,
Jul 19, 2016, 3:40:30 PM7/19/16
to stan-...@googlegroups.com
I found the same thing during simulation using a von Mises
distribution for the change in direction.

I wonder if it may have to do with the fact that if we
set up a constrained variable

real<lower = - pi() / 2, upper = pi() / 2> theta;

that there's a discontinuity, because in terms of angles,
pi()/2 == -pi()/2.

I'm curious what happens with simulated data in packages
like moveHMM, which I would assume suffer the same problem,
because I saw the same thing with optimization in Stan.

I'm pretty sure the coding of the HMM isn't the issue.

- Bob
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Michael Betancourt

unread,
Jul 19, 2016, 3:58:38 PM7/19/16
to stan-...@googlegroups.com
I tried to warn about this earlier — taking real<lower = - pi() / 2, upper = pi() / 2> theta;
pushes a huge amount of probability mass at the discontinuity to infinity which the
sampler will have difficulty reaching in a reasonable amount of time. I’m not sure
how this would exactly manifest, but it ultimately reduces to the fact that our HMC
sampler can only handle spaces that are globally homeomorphic to R^{n}.

Ben Goodrich

unread,
Jul 19, 2016, 4:47:52 PM7/19/16
to Stan users mailing list
On Tuesday, July 19, 2016 at 3:40:30 PM UTC-4, Bob Carpenter wrote:
I found the same thing during simulation using a von Mises
distribution for the change in direction.

I wonder if it may have to do with the fact that if we
set up a constrained variable

  real<lower = - pi() / 2, upper = pi() / 2> theta;

that there's a discontinuity, because in terms of angles,
pi()/2 == -pi()/2.  

It may work better if you reparameterize the correlation as a product of two unit-vectors of size p = 2

https://en.wikipedia.org/wiki/Von_Mises%E2%80%93Fisher_distribution

Ben



Bob Carpenter

unread,
Jul 19, 2016, 5:31:03 PM7/19/16
to stan-...@googlegroups.com
Yes, your mentioning it and realizing that there are values near
the boundaries when I simulated is what led me to point it out.

I want to try the parameterization with

parameters {
unit_vector[2] xy; // xy[1]^2 + xy[2]^2 = 1

transformed parameters {
real<lower=-pi()/2, upper=pi()/2> theta;
theta = atan2(xy[1], xy[2]) - pi();

and see if that works. The underlying unit vector
parameterization involves two parameters with an implicit
prior for identification.

- Bob

Bob Carpenter

unread,
Jul 19, 2016, 5:42:09 PM7/19/16
to stan-...@googlegroups.com
You'll need the Jacobian for the transform:

(x, y) -> (x, theta)

- Bob

Ben Lambert

unread,
Jul 19, 2016, 5:59:20 PM7/19/16
to Stan users mailing list
Hello all,

Thanks for your posts. 

Bob - by 'theta' here I'm guessing you mean what I call 'phi' (the angle distribution)? For me 'theta' represents the transition probabilities...

Also, my angle distribution goes from -pi() to pi() - Bob how does this affect the transformation you suggest above? (Sorry for being a bit slow, but I don't quite understand your suggestion so don't know how to change it.)

Finally, I have tried estimating the model using 'moveHMM' - on first looks it does not appear to suffer from the same issues...the transition probability matrix appears to be a much better estimate of the actual transition matrix.

Thanks again!

Ben

Bob Carpenter

unread,
Jul 19, 2016, 6:01:59 PM7/19/16
to stan-...@googlegroups.com



> On Jul 19, 2016, at 5:59 PM, 'Ben Lambert' via Stan users mailing list <stan-...@googlegroups.com> wrote:
>
> Hello all,
>
> Thanks for your posts.
>
> Bob - by 'theta' here I'm guessing you mean what I call 'phi' (the angle distribution)?

Right.

> For me 'theta' represents the transition probabilities...
>
> Also, my angle distribution goes from -pi() to pi() - Bob how does this affect the transformation you suggest above? (Sorry for being a bit slow, but I don't quite understand your suggestion so don't know how to change it.)

Sorry, meant that to be 2 * pi(). The transform's right, just
not the bounds on theta. And indeed, that's the angle in radians.

> Finally, I have tried estimating the model using 'moveHMM' - on first looks it does not appear to suffer from the same issues...the transition probability matrix appears to be a much better estimate of the actual transition matrix.

We should be getting the same answer with the same model if
it's just optimization and not max marginal likelihood. Now
I wonder if everything's coded properly.

- Bob

Ben Lambert

unread,
Jul 19, 2016, 6:12:03 PM7/19/16
to Stan users mailing list
Ok, when I use 'optimizing' I haven't been getting particularly good estimates of the state. In particular, the states appear to get stuck in one 

Instead I've been using MCMC then taking the average state across all samples...Is this ok?

For completeness, the code below is what I've been using to generate the fake data:

## Generate a continuous latent variable that we then use to make a discrete state vector
fARProcess <- function(rho,X0,sigma,aT){
  X <- vector(length=aT)
  X[1] <- X0
  for(t in 2:aT){
    X[t] = rho * X[t-1] + rnorm(1,0,sigma)
  }
  return(X)
}

fState <- function(rho,X0,sigma,aT){
  X <- fARProcess(rho,X0,sigma,aT)
  Z <- ifelse(X<0,0,1)
  return(Z)
}

fGenerateData1 <- function(aAlpha,aBeta0,aBeta1,rho,X0,sigma,aT){
  Z <- fState(rho,X0,sigma,aT)
  l <- vector(length=aT)
  for(t in 1:aT){
    if(Z[t]==0){
      l[t] <- rweibull(1,aAlpha,aBeta0)
    }else{
      l[t] <- rweibull(1,aAlpha,aBeta1)
    }
  }
  return(list(l=l,Z=Z))
}

## Angle data
library(CircStats)
fGenerateAngle <- function(Z,aMu0,aMu1,aRho0,aRho1){
  lAngle <- sapply(Z, function(x) ifelse(x==0,rwrpcauchy(1,aMu0+pi,aRho0)-pi,rwrpcauchy(1,aMu1+pi,aRho1)-pi))
  return(lAngle)
}

## Fit model to fake data
N <- 400
L <- fGenerateData1(1,200,1000,0.99,0,1,N)
lAngle <- fGenerateAngle(L$Z,0,0,0.3,0.9)

plot(L$Z,type = 'l')
plot(L$l,type='l')
plot(lAngle,type = 'l')

Ben

Bob Carpenter

unread,
Jul 19, 2016, 10:13:37 PM7/19/16
to stan-...@googlegroups.com

> On Jul 19, 2016, at 6:12 PM, 'Ben Lambert' via Stan users mailing list <stan-...@googlegroups.com> wrote:
>
> Ok, when I use 'optimizing' I haven't been getting particularly good estimates of the state. In particular, the states appear to get stuck in one

One mode?

> Instead I've been using MCMC then taking the average state across all samples...Is this ok?

Depends what you're doing. The problem is that the
mixture labels aren't identifiable. So you
either need to insure the model's identifiable (say by
ordering means of components or something) or just use
a single chain or just do inference that doesn't depend on
the identity of mixture components.

- Bob

Ben Lambert

unread,
Jul 20, 2016, 6:02:35 AM7/20/16
to Stan users mailing list
Yes, sorry - I meant one mode. I know these types of model suffer from identifiability issues, but I would have thought that moveHMM (which uses MLE) would have performed similarly to the model as we've coded it up...?

Can you confirm the transform you were suggesting is as follows:

parameters { 
    unit_vector[2] xy;  // xy[1]^2 + xy[2]^2 = 1 

 transformed parameters { 
    real<lower=-pi(), upper=pi()> theta; 

Bob Carpenter

unread,
Jul 20, 2016, 8:51:30 AM7/20/16
to stan-...@googlegroups.com

> On Jul 20, 2016, at 6:02 AM, 'Ben Lambert' via Stan users mailing list <stan-...@googlegroups.com> wrote:
>
> Yes, sorry - I meant one mode. I know these types of model suffer from identifiability issues, but I would have thought that moveHMM (which uses MLE) would have performed similarly to the model as we've coded it up...?

If the model's the same, we should get the same MLE. I'm not
sure what moveHMM does, but I believe they use max marginal
likelihood for some estimates.

That's why I'm worried that either they're fitting a different
model or using a different estimator, or there's something wrong
with the way I coded the HMM.

> Can you confirm the transform you were suggesting is as follows:
>
> parameters {
> unit_vector[2] xy; // xy[1]^2 + xy[2]^2 = 1
>
> transformed parameters {
> real<lower=-pi(), upper=pi()> theta;
> theta = atan2(xy[1], xy[2]) - pi();

Yes. There's an implicit uniform prior on unit vectors,
which means there will be implicitly a uniform prior on theta.

If you want to add a more informative prior, then you'll need
to increment the log density with the log Jacobian determinant
of the transform.

- Bob

Ben Lambert

unread,
Jul 20, 2016, 10:05:43 AM7/20/16
to Stan users mailing list
Hi,

I have now implemented (what I hope is what Bob was thinking, although please correct me if wrong). The code for this is below - it seems to suffer from a lot of divergent iterations, which makes me think perhaps I've misinterpreted what Bob had in mind...

Also, I'm not sure that the problem with identifying the latent state has to do with the divergences at mu = -pi(); when I supply - as data - the mean angle at zero, the estimator still suffers from the same issues. In other words - it estimates a transition matrix with too much weight towards staying in a given mode.

What do you think?

functions{
  ## Function that returns the log pdf of the wrapped-Cauchy
  real wrappedCauchy(real aPhi, real aRho, real aMu) {
    return(- log(2*pi()) + log((1-aRho^2)/(1+aRho^2-2*aRho*cos(aPhi-aMu))));
  }
  
  ## Function that returns a random sample from wrapped-Cauchy by first
  ## sampling from the Cauchy, then using modular arithmetic
  real wrappedCauchy_rng(real aRho, real aMu){
    real aCauchyRand;
    real aWCRand;
    
    aCauchyRand = cauchy_rng(pi()+aMu,-log(aRho));
    if (aCauchyRand>0){
      aWCRand = fmod(aCauchyRand,2*pi()) - pi();
    } else{
      aWCRand = -fmod(-aCauchyRand,2*pi()) + pi();   
    }

    return(aWCRand);
  }
  
}

data {
  int<lower=0> N;
  vector<lower=-pi(), upper=pi()>[N] phi;
  vector<lower=0>[N] l;
  int<lower=1> K;
}
parameters {
  simplex[K] theta[K];
  unit_vector[2] xy[K];  // xy[1]^2 + xy[2]^2 = 1
  vector<lower=0,upper=1>[K] rho;
  positive_ordered[K] alpha;
}  
transformed parameters{
  real<lower=-pi(), upper=pi()> mu[K];
  for(i in 1:K){
    vector[2] aXY;
    aXY = xy[i];
    mu[i] = atan2(aXY[1], aXY[2]) - pi();
  }
  
}

model {
  vector[K] log_theta[K];
  vector[K] lp;

  for (k in 1:K) log_theta[k] = log(theta[k]);
  lp = rep_vector(-log(K), K); ## Initial state log-probability
  for (n in 1:N) {
    for (k in 1:K)
      lp[k]
        = log_sum_exp(log_theta[k] + lp)
        + wrappedCauchy(phi[n], rho[k], mu[k])
        + exponential_lpdf(l[n] | alpha[k]);
  }
  target += log_sum_exp(lp);

  alpha[1] ~ normal(0.001,0.0001);
  alpha[2] ~ normal(0.005,0.0001);
  rho[1] ~ normal(0.95,0.01);
  rho[2] ~ normal(0.3,0.01);

Ben Lambert

unread,
Jul 20, 2016, 10:28:24 AM7/20/16
to Stan users mailing list
Another question I had that is somewhat related, was how to best make the transition probabilities depend on a covariate, X? I know Bob mentioned that this was possible using a 'multi_logit' transform, but I was unsure what exactly you meant by this...

I can see that essentially we want something like (for a two-state model):

for(i in 1:2){
logit(theta[i,1]) = alpha + beta * X
theta[i,2] = 1 - theta[i,1];
}

Does this seem like a reasonable approach?

Forgive my fumbling around here, I am new to these types of models.

Best,

Ben

Bob Carpenter

unread,
Jul 20, 2016, 3:48:10 PM7/20/16
to stan-...@googlegroups.com
These are probably causing your divergent transitions
as the scale is very constrained:

alpha[1] ~ normal(0.001,0.0001);
alpha[2] ~ normal(0.005,0.0001);
rho[1] ~ normal(0.95,0.01);
rho[2] ~ normal(0.3,0.01);

We recommend converting anything you can to unit scale. It
takes a lot of the pressure off Stan's adaptation.

- Bob

Bob Carpenter

unread,
Jul 20, 2016, 3:48:49 PM7/20/16
to stan-...@googlegroups.com
Check out categorical_logit. What you really want is
softmax. There's a discussion in the manual.

- Bob

Andrew Gelman

unread,
Jul 20, 2016, 8:05:12 PM7/20/16
to stan-...@googlegroups.com
Never "just use a single chain." If you can't identify the mixture components by ordering their means, you can identify them by locking in particular observations to particular components. Or there are other ways of doing it.

Bob Carpenter

unread,
Jul 21, 2016, 12:47:29 AM7/21/16
to stan-...@googlegroups.com
What I found in running multiple chains is that they
did indeed converge, but the label-switching problem remains.
The ordering of means for the distance traveled indeed identified
this simple model. But, you have to be aware that doing the
ordering changes the model's posterior (perhaps not significantly,
but I'm not sure how to test that without doing things Andrew
won't approve of).

For locking in particular observations in an HMM, do you mean
something like assuming it always starts in state 1 or something
like that? We could code that, and it won't change the model.
I'm not sure how well it'll identify things given the long-running
chains. It doesn't seem to work so well with ideal-point models.

- Bob

Andrew Gelman

unread,
Jul 21, 2016, 9:49:07 PM7/21/16
to stan-...@googlegroups.com
I think that for any of these models, you need to do something to identify the separate components. I don't know of any completely general way to do this if the models are more than one-dimensional. Different methods of identification seem to work for different problems.
A
Reply all
Reply to author
Forward
0 new messages