Seeking an overview on "Hessian is Singular" warning message

[Andrew Cox]

Hi,

  I wonder if one of the resident gurus could answer a few questions about the "Hessian is Singular" warning.  They are:

1.  Is this usually due to linear dependencies in the data, or are there other reasons I might get the error?  

2. Why is it that I sometimes get the error on a simpler model that does not generate it when all the terms are nested in a more complex model?  For example, I got the error here:

HZ_StateYearLand<-distsamp(~state+year+land_detect~1,keyfun="hazard", unitsOut="ha", output="density",data=umf)

but not here:

HZ_Global<-distsamp(~state+year+min_since_sunrise+land_detect~1,keyfun="hazard", unitsOut="ha", output="density",data=umf)

3. Why is it that a model with one detection function may run fine, but the same model run with another may not?  For example, I get the error here:

 HN_State<-distsamp(~state~1,keyfun="halfnorm", unitsOut="ha", output="density",data=umf)

but not here:

  HZ_State<-distsamp(~state~1,keyfun="hazard", unitsOut="ha", output="density",data=umf)

4. Is there a good way of estimating starting values with distance data to try to overcome this problem?

5. Who is this Hessian person, and why does he hate me so?

Thanks in advance!

Andrew Cox

University of Missouri - Columbia 

[Jeffrey Royle]

hi Andrew

 I think this is the Hessian guy:
http://en.wikipedia.org/wiki/Ludwig_Otto_Hesse

 The basic problem of a singular Hessian is almost always due to a non-identifiable parameter, so that the p-dimensional objective function really lives in (p-1) (or fewer) dimensions. could be linear or nonlinear dependencies, extremely sparse data causing the algorithm to fail, bad starting values, etc..

 

 Estimating starting values: nothing automatic, as it would depend on the specific function.  I could imagine doing this by some kind of a coarse grid search initially, and maybe it wouldn't be too hard to cook up a helper function to do this. Seems like a good thing for someone to contribute to the project!

 

regards

andy

[Andrew Cox]

Just a quick update for other users who may encounter similar issues.

1. Setting starting values helped with some models. I either used parameter values from other models that successfully converged or set se=FALSE as an option and used those.  I was a little confused on how to order the starting values, but I believe it goes: starts=c(abundance intercept, abundance covariates...,detection intercept,detection covariates...).  And for hazard function, the scale parameter precedes the detection intercept.  Someone should correct me if I am wrong.

2. In one case, we had an observer who never detected one of the focal species.  We had to either delete this user or merge him with another for the model to converge.  But this led us to go back and review potential dependencies among other covariates to make sure we wouldn't run into similar problems elsewhere.  Never a bad idea!

3.  This whole process reinforced the need to carefully examine the output from each model we ran.  Some models successfully converged but still exhibited unrealistic estimates and SEs.

Many thanks to Andy and Richard and Marc and others for their continued support on this board.

- Andrew

[Paul J Taillie]

Andrew,

I ran into some similar problems with distsamp() and went through the same procedure of just trying start values from other models that did run.  Also, a lot of times I got models to run (similar to your case 3) and then used the estimates from those models as a basis for start values, reran the model, and got some better output (sometimes).  Anyway, I offer no useful advice, just confirming that I had your problem as well. 

Good luck.

Paul

[Kery Marc]

Hello again,

I believe we have always to be on the lookout for spurious convergence to the MLE when using Maximum Likelihood for more complex models. Makes me think that some people bashing the bayesians for problems with convergence assessment of the Markov chains should perhaps clean in front of their own door first ;) -- When I ran multistate models in MARK years ago I frequently got solutions that were not the MLEs. This is often easy to diagnose when you have a progression of models of increasing complexity: the negative log-likelihood (NLL) must go down, at least a little bit, when you add in structure to the model. If the program does not, then you have not succeeded in finding the global maximum of the likelihood. So it's good to monitor the NLL for series of fitted models and see whether they make sense. --- In unmarked I have had such challenges with the colext function.

Re. the ordering of params in the list of starting values: you can see that by typing this (for the help file example of gdistsamp):
> coef(m1)
lambda(Int)    phi(Int)      p(Int)
  1.5265430   0.5192659   3.4448748

So if you want to use the solutions as inits for a fit of a model with one covariate on lambda, you could use these starting values:
> c(coef(m1)[1], NA, coef(m1)[2:3])
lambda(Int)                phi(Int)      p(Int)
  1.5265430          NA   0.5192659   3.4448748

Regards  --  Marc

--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.