Cannot find solution for the error: Hessian is singular. Try using fewer covariates or providing starting values

[Ana Gonzalez]

Hi everyone,

I’m trying to fit some models using distsamp but I get the error “"Hessian is singular. Try using fewer covariates or providing starting values." 

I read many of the answers to this error but couldn’t find any that applies to my case. I’ll appreciate if someone can help me out.

I’m running simple models without detection covariates for now; just density covariates. 

My covariates are Habitat (two level: Forest, coffee), and Site (three level: Fragua, Vientos, Vuelta). These are just my 2015 data, eventually I’ll have year as an additional density covariate.

SWTH_Null <- distsamp(~1~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

SWTH_1 <- distsamp(~ 1 ~Habitat+Site, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

SWTH_2<- distsamp(~ 1 ~Habitat, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

SWTH_3<- distsamp(~ 1 ~Site, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

Models SWTH_1 and SWTH_3 run fine (results below). But I get the error message when I run the model SWTH_2 (only habitat with two levels) which is simpler than the other two. 

I cannot really scale my covariates, or include the parameters of a simpler model as start values  since it’s the simplest. 

I’ll appreciate any advice.

Many thanks,

Ana

SWTH 1_Habitat and Site

Call:
distsamp(formula = ~1 ~ Habitat + Site, data = ltUMF, keyfun = "hazard", 
    output = "density", unitsOut = "kmsq")

Density:
              Estimate     SE     z  P(>|z|)
(Intercept)     5.8534 0.1034 56.59 0.00e+00
HabitatForest   0.6692 0.0706  9.48 2.53e-21
SiteVientos    -0.2570 0.0837 -3.07 2.13e-03
SiteVuelta      0.0929 0.0806  1.15 2.49e-01

Detection:
 Estimate    SE    z  P(>|z|)
     2.44 0.175 13.9 3.68e-44

Hazard-rate(scale):
 Estimate    SE     z P(>|z|)
     0.12 0.147 0.813   0.416

AIC: 2304.222 
              Estimate     SE     z  P(>|z|)
(Intercept)     5.8534 0.1034 56.59 0.00e+00
HabitatForest   0.6692 0.0706  9.48 2.53e-21
SiteVientos    -0.2570 0.0837 -3.07 2.13e-03
SiteVuelta      0.0929 0.0806  1.15 2.49e-01

Detection:
 Estimate    SE    z  P(>|z|)
     2.44 0.175 13.9 3.68e-44

Hazard-rate(scale):
 Estimate    SE     z P(>|z|)
     0.12 0.147 0.813   0.416

AIC: 2304.222 

SWTH 3_Site

Call:
distsamp(formula = ~1 ~ Site, data = ltUMF, keyfun = "hazard", 
    output = "density", unitsOut = "kmsq")

Density:
            Estimate     SE      z P(>|z|)
(Intercept)    6.245 0.0922 67.730 0.00000
SiteVientos   -0.272 0.0837 -3.245 0.00117
SiteVuelta     0.048 0.0804  0.597 0.55069

Detection:
 Estimate    SE    z  P(>|z|)
     2.44 0.175 13.9 3.39e-44

Hazard-rate(scale):
 Estimate    SE     z P(>|z|)
     0.12 0.147 0.814   0.416

AIC: 2396.094 
            Estimate     SE      z P(>|z|)
(Intercept)    6.245 0.0922 67.730 0.00000
SiteVientos   -0.272 0.0837 -3.245 0.00117
SiteVuelta     0.048 0.0804  0.597 0.55069

Detection:
 Estimate    SE    z  P(>|z|)
     2.44 0.175 13.9 3.39e-44

Hazard-rate(scale):
 Estimate    SE     z P(>|z|)
     0.12 0.147 0.814   0.416

AIC: 2396.094 
> 

[Kery Marc]

Hi Ana,

you can also try to use solutions from a more complex model as starting values for a simpler one. So, you could try to use the coefficients from model 1 (without the two for site) as inits for model 2. Or, even better: average about the three site effects in model 1 and use that as the intercept init for at attempt at getting solutions for model 2.

Best regards  --- Marc

---

From: unma...@googlegroups.com [unma...@googlegroups.com] on behalf of Ana Gonzalez [ana.gonzale...@gmail.com]
Sent: 31 May 2017 01:10
To: unmarked
Subject: [unmarked] Cannot find solution for the error: Hessian is singular. Try using fewer covariates or providing starting values

--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ana Gonzalez]

Hi Marc,

Thanks so much for your quick response, it was very helpful.

I used coef(fm) to obtain the coefficients of  complex models and used them in my simple models.I also combined coefficients of simple models and used them as starting values of complex models that gave me the error message. All worked!

After model selection I wanted to check the goodness of fit of my top model but my results are quite poor.

I run two set of models and in both cases I get poor fit for the top model. I'm not quite sure how to proceed when the fit of my selected model is poor. I'll appreciate any advice. I included my results below in case it's helpful.

Many thanks,

Ana

I have three study sites and have line transects in Forest and shade coffee in each site

#Set 1 (13 models)

SWTH_Null <- distsamp(~1~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

SWTH_H_S <- distsamp(~ 1 ~Habitat+Site, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

SWTH_H <- distsamp(~ 1 ~Habitat, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq",starts=c(5.85335811,0.66917417,2.43728483,0.11994592))

SWTH_S<- distsamp(~ 1 ~Site, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

#Models with detectability

Det_H <- distsamp(~Habitat~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq", starts=c(6.2007080,2.2588355,0.7765481,0.2339953))

Det_S <- distsamp(~Site~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq", starts=c(6.2007080,2.2588355,-0.8403309,0.1936365 ,0.2339953))

Det_HabSite<- distsamp(~Habitat+Site~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

#Detectability Habitat

DH_H_S<- distsamp(~Habitat~Habitat+Site,ltUMF, keyfun="hazard", output="density", unitsOut="kmsq"

                  , starts=c(5.5621487,0.9789661,-0.25744416,0.09150561,2.9288883,-0.5765546,0.2276319))

DH_H<- distsamp(~Habitat~Habitat,ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

DH_S<- distsamp(~Habitat~Site, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

#Detectability site

DS_H_S<- distsamp(~Site~Habitat+Site,ltUMF, keyfun="hazard", output="density", unitsOut="kmsq"

                  ,starts=c(7.21469205, 0.6531522, -0.25744416,0.09150561, 5.7695628, -3.2980616, -2.5904657,0.8289918))

DS_H<- distsamp(~Site~Habitat,ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

coef(DS_H)

DS_S<- distsamp(~Site~Site, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq"

                ,starts=c(7.21469205, -0.25744416,0.09150561, 5.7695628, -3.2980616, -2.5904657,0.8289918))

#Top model: DS_H_S< (~Site~Habitat+Site)

#Poor fit!

Parametric Bootstrap Statistics:

               t0 mean(t0 - t_B) StdDev(t0 - t_B) Pr(t_B > t0)

SSE          1463            601             62.2            0

Chisq        1302            443             45.2            0

freemanTukey  430            109             12.8            0

 

t_B quantiles:

              0% 2.5% 25% 50% 75% 97.5% 100%

SSE          711  753 814 858 904   990 1106

Chisq        735  777 830 857 889   949 1009

freemanTukey 282  297 312 320 329   347  366

 

t0 = Original statistic compuated from data

t_B = Vector of bootstrap samples

#Set 2. I didn't include site density in this set since I'm interested just in the density by habitat.

#Models with out detectability

SWTH_Null <- distsamp(~1~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

SWTH_H <- distsamp(~ 1 ~Habitat, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq",starts=c(5.85335811,0.66917417,2.43728483,0.11994592))

#Models with detectability

Det_H <- distsamp(~Habitat~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq", starts=c(6.2007080,2.2588355,0.7765481,0.2339953))

Det_S <- distsamp(~Site~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq", starts=c(6.2007080,2.2588355,-0.8403309,0.1936365 ,0.2339953))

Det_HabSite<- distsamp(~Habitat+Site~1, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

#Detectabily-Habitat

DH_H<- distsamp(~Habitat~Habitat,ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

#Detectability Site

DS_H<- distsamp(~Site~Habitat,ltUMF, keyfun="hazard", output="density", unitsOut="kmsq")

Det_HabSite_H<- distsamp(~Habitat+Site~Habitat, ltUMF, keyfun="hazard", output="density", unitsOut="kmsq"

                       ,starts=c(5.8044394, 0.6625020

                                 ,2.2588355,0.7765481,-0.8403309,0.1936365,0.1199997))

Top model: Det_HabSite_H (~Habitat+Site~Habitat)

                       

Call: parboot(object = Det_HabSite_H, statistic = fitstats, nsim = 1000, report = 1)

#Poor fit!

Parametric Bootstrap Statistics:

               t0 mean(t0 - t_B) StdDev(t0 - t_B) Pr(t_B > t0)

SSE          1494            634             62.8            0

Chisq        1346            491             43.4            0

freemanTukey  435            115             12.1            0

t_B quantiles:

              0% 2.5% 25% 50% 75% 97.5% 100%

SSE          670  742 817 855 904   991 1113

Chisq        717  777 825 854 883   942 1021

freemanTukey 286  298 312 320 328   344  364

t0 = Original statistic compuated from data

t_B = Vector of bootstrap samples