UnmarkedFrameGDS data

793 views
Skip to first unread message

Don Segundo

unread,
Nov 4, 2015, 11:05:04 AM11/4/15
to unmarked
 First off let me say that I'm a relative noob with unmarked.  Ive spent some time trying to figure this out using the unmarked manual however its not been much help.

Im trying to use the above data in unmarkedFrameGDS to get population estimates ala' Sillet et al. 2012 (island scrub jay paper) however it is unclear to me how to code for  multiple visits.  Ive figured out how to upload the data using csvtoUMF however its unclear to me how to use it in unmarkedFrameGDS.

Sillett's code for formatting the data using unmarkedFrameGDS is as follows:

umfFall <- unmarkedFrameGDS(y = Xall.100m, siteCovs=covs, dist.breaks=db,  unitsIn="m", survey="point", numPrimary=1) summary(umfSpr) 

His study had one visit with distance truncated at 100m. I have 4 visits with distance truncated at 100m.

Would the coding look something like this?

 umfbevi2012<-unmarkedFrameGDS(y= y[1:4], siteCovs=data.frame (patch, cp) , yearlySiteCov=list (DATE= date TIME=time),dist.breaks=db, unitsIn="m", survey= "point", numPrimary=1)
BEVI2012.csv

Richard Chandler

unread,
Nov 4, 2015, 1:01:21 PM11/4/15
to Unmarked package
When you start off being critical, you're less likely to get help. No one gets paid to maintain unmarked. There are many things that could be improved, especially in the documentation, but it's very hard to find the time. However, you are welcome to contribute to the project and tidy up the documentation or fill in the gaps or whatever. 

Richard
 

--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Richard Chandler
Assistant Professor
Warnell School of Forestry and Natural Resources
University of Georgia

Don Segundo

unread,
Nov 4, 2015, 9:00:48 PM11/4/15
to unmarked, rcha...@warnell.uga.edu
 Richard,

Im sorry if my slight criticism of the documentation offends you.  That was my experience.  I will make it a goal of mine to contribute to the project once I have figured out how actually use it. Can you please help me now?

Dan Linden

unread,
Nov 4, 2015, 10:44:05 PM11/4/15
to unmarked, rcha...@warnell.uga.edu
Don, usually it is good to start off humble when you are looking for help.  There is a lot information on this forum, but you have to spend some time poking around.  Also, aside from the user manual there are multiple vignettes for using unmarked models, including one on distance sampling:

https://cran.r-project.org/web/packages/unmarked/index.html

Your data do not indicate that distance sampling was actually used.  Did you record distances to each individual, or did you simply count all individuals within a 100m radius?  The latter would be appropriate for gpcount when you have multiple visits.

Don Segundo

unread,
Nov 5, 2015, 9:31:09 AM11/5/15
to unmarked, rcha...@warnell.uga.edu
Dan,

 I was humble in my original post as I said I was a noob. While I may have said that I didn't find the manual helpful I didn't say anything about its relative worth. Richard's response to me was inappropriate. Maybe he was having a bad day. Also, I spent a considerable amount of time going through the forum and the manual before I posted.

Thank you for your help.I did use distance however that data file lumped all the observations into one bin 0-100.  I realize now that the file needs distances.  I set up the data file based on the data and documentation from Sillett's 2012 Island scrub jay paper which has no distances in the data fie.

Do you think I can use gpcount to come up with population estimates similar to the Sillett paper?

Matthew Giovanni

unread,
Nov 5, 2015, 9:45:28 AM11/5/15
to unma...@googlegroups.com, rcha...@warnell.uga.edu

Don, you're missing the point: to state that you found the user manual unhelpful was not only unnecessary but wholly irrelevant to your request for assistance. Case closed. I suggest we end the discussion and focus on your technical request.

Matt

Matt Giovanni
608-320-9331

--

Richard Chandler

unread,
Nov 5, 2015, 9:53:33 AM11/5/15
to Unmarked package
Don,

If your question is simply: "Would someone please explain how to use gdistsamp to fit a model to the jay data", then I'd be happy to help. I took the example from the distsamp() help page and made a few minor edits:

data(issj)
str(issj)

jayumf <- unmarkedFrameGDS(y=as.matrix(issj[,1:3]),
          siteCovs=data.frame(scale(issj[,c("elevation","forest","chaparral")])),
          dist.breaks=c(0,100,200,300), unitsIn="m", survey="point", 
          numPrimary=1)

(fm1jay <- gdistsamp(~chaparral, ~1, ~1, jayumf, mixture="NB"))


Additional details regarding the analysis of the jay data can be found in this vignette, which comes with unmarked:



Best,
Richard

Dan Linden

unread,
Nov 5, 2015, 11:10:22 AM11/5/15
to unmarked, rcha...@warnell.uga.edu
Don, the Sillet et al. 2012 paper does have distance data, but they do not actually estimate phi using multiple replicates.  They used the gdistsamp function to take advantage of the negative binomial distribution for abundance.  This is pointed out in the help file (?gdistsamp) where Richard notes that setting numPrimary=1 will give you the same results as distsamp but allow examination of the NB distribution.  Sillet et al. 2012 used 3 distance bands as indicated in their code:

db <- c(0, 100, 200, 300)

If you have R sites, J distance bands, and T occasions, you need a matrix y that is R x JT with the count of individuals for each band at each occasion. The data files in Sillet et al. 2012 are R=307, J=3, and T=1 (fall and spring data, separate).  As another example, with J=3 and T=2, your column names could look like this:

colnames(y) <- c("band1visit1", "band2visit1","band3visit1","band1visit2", "band2visit2","band3visit2")

The unmarkedFrameGDS object doesn't need column names for y but it might help with your data organization.  Experienced R users simplify their code to increase readability and efficiency so sometimes things aren't obvious.  But it is best to dig into the help files and work with examples, like Richard has provided, to better understand.  The help file for gdistsamp does have everything you need, but you have to spend time with it.

Don Segundo

unread,
Nov 6, 2015, 10:02:17 AM11/6/15
to unmarked, rcha...@warnell.uga.edu
Dan,

Thank you very much for taking the time to actually look at my question and trying to help.  While Sillett et al. does show three distance bands in their coding, the data file does not have any data on distances or categories of detections at different distances unless your seeing something Im not. Anyway, Im in the process of categorizing the detections into different bins among the multiple visits and the way you suggested coding for the column names makes sense.

Don Segundo

unread,
Nov 8, 2015, 2:51:49 PM11/8/15
to unmarked, rcha...@warnell.uga.edu
Dan,

I realize that I cant use unmarkedFrameGDS for my analysis as it doesn't accommodate multiple visits and so I need to use gpcount. Anyway, Ive come up with a new datafile that includes 5 bins per visit (attached) my coding thus far looks like this:

> a<-read.csv("C:/Users/Bryan/Desktop/CREP dissertation/ch.2CREPPopulationestimate/N-mixture data/BEVI_csv/BEVI2012_20m.csv")
> y<-a[,2:20]
> J<-207
> T<-4
> site.cov<-(cp=a[22])
> db<-c(0,20,40,60,80,100)
> obs.covs<-data.frame(time = a[,24:27], date=a[,27:31])
> beviumf<-unmarkedFramePCount(y=y, siteCovs=site.cov, obsCovs=obs.covs)

However, I get an error code after the last line of code: Error in validObject(.Object) :
  invalid class “unmarkedFrame” object: obsCovData does not have M*obsNum rows.

Any ideas? 
BEVI2012_20m.csv

Giancarlo Sadoti

unread,
Nov 10, 2015, 1:00:01 AM11/10/15
to unmarked, rcha...@warnell.uga.edu
Brian, see below

#issue 1:
#your y data are in distance bins; pcount can't use those. you need to collapse them.
#you made an error in your column numbers.
y=cbind(rowSums(a[,2:6],na.rm=T),
        rowSums(a[,7:11],na.rm=T),
        rowSums(a[,12:16],na.rm=T),
        rowSums(a[,17:21],na.rm=T))                           
J<-207
T<-4
site.cov<-(cp=a[22])

#issue 2: you used data.frame() instead of list() and made an error in your column numbers
obs.covs<-list(time = a[,24:27], date=a[,28:31])
beviumf<-unmarkedFramePCount(y=y, siteCovs=site.cov, obsCovs=obs.covs)

Dan Linden

unread,
Nov 10, 2015, 8:10:19 AM11/10/15
to unmarked, rcha...@warnell.uga.edu
Also, unmarkedFrameGDS and gdistsamp do account for multiple visits - it is the whole reason they exist:

?gdistsamp
"To estimate this additional parameter [phi], replicate distance sampling data must be collected at each transect."

Don Segundo

unread,
Nov 10, 2015, 9:48:15 AM11/10/15
to unmarked, rcha...@warnell.uga.edu
Thank you Giancarlo.
Message has been deleted

Don Segundo

unread,
Nov 11, 2015, 10:41:39 AM11/11/15
to unmarked, rcha...@warnell.uga.edu

After much confusion Ive come to the conclusion that I can use unmarkedFrameGDS with my data.  I will have to analyze years separately with the time frames representing within year replicate surveys. New problem,  when I try to backTransform the abundance estimates I get an error message. Currently in R  I have:


> a<-read.csv("C:/Users/Bryan/Desktop/CREP dissertation/ch.2CREPPopulationestimate/N-mixture data/BEVI_csv/BEVI2012_20mNoobscovs.csv")

> y<-a[,2:21]

> R<-206

> T<-4

> site.cov<-(cp=a[22])

> db<-c(0,20,40,60,80,100)

>

> beviumf<-unmarkedFrameGDS(y=y, siteCovs=site.cov, survey="point", dist.breaks=db,unitsIn="m", numPrimar=T)

> summary(beviumf)

unmarkedFrame Object

 

206 sites

Maximum number of observations per site: 20

Mean number of observations per site: 11.6

Number of primary survey periods: 4

Number of secondary survey periods: 1

Sites with at least one detection: 21

 

Tabulation of y observations:

   0    1    2 <NA>

2333   49    8 1730

 

Site-level covariates:

    cp   

 CP22:53 

 CP23:76 

 CP3A:45 

 CP4D:32 

>

> BEVI<- list()

> BEVI$null<-gdistsamp(~1, ~1, ~1, beviumf, output="abund", mixture="P", K=50)

> BEVI$nullNB<-gdistsamp(~1, ~1, ~1, beviumf, output="abund", mixture="NB", K=200)

> BEVI$NBcp<-gdistsamp(~cp, ~1, ~1, beviumf, output="abund", mixture="NB", K=200)

> BEVI$cp<-gdistsamp(~cp, ~1, ~1, beviumf, output="abund", mixture="NB", K=200)

> fitsBEVI<-fitList(fits=BEVI)

> msBEVI<-modSel(fitsBEVI)

> exportBEVI<-cbind(coef(msBEVI), msBEVI@Full$AIC)

> save.image("RbeviData")

> View(exportBEVI)

> backTransform(BEVI$cp, type="phi")

Backtransformed linear combination(s) of Availability estimate(s)

 

 Estimate     SE LinComb (Intercept)

   0.0393 0.0262    -3.2           1

 

Transformation: logistic

> backTransform(BEVI$cp, type="det")

Backtransformed linear combination(s) of Detection estimate(s)

 

 Estimate   SE LinComb (Intercept)

     62.1 11.1    4.13           1

 

Transformation: exp

> backTransform(BEVI$cp, type="lambda")

Error in .local(obj, ...) :

  Cannot directly backTransform an unmarkedEstimate with length > 1


Im guessing it has to do with the categorical nature of the site covariate?  I also tried using the linearComb function but couldnt get it to work.

BEVI2012_20mNoObsCovs.csv

Kery Marc

unread,
Nov 11, 2015, 11:06:53 AM11/11/15
to unma...@googlegroups.com, rcha...@warnell.uga.edu
Dear Don Segundo,

I believe that backTransform only works for an intercept-only model. As soon as you have a covariate or more, you need to use predict.

Kind regards  --- Marc

______________________________________________________________
 
Marc Kéry
Tel. ++41 41 462 97 93
marc...@vogelwarte.ch
www.vogelwarte.ch
Swiss Ornithological Institute | Seerose 1 | CH-6204 Sempach | Switzerland
______________________________________________________________

********* NEW Hierarchical modeling books (due November 2015 and in 2016) *********
*** Kéry & Royle (2016): Applied hierarchical models in ecology, Academic Press, Volume 1, Prelude and Static Models, see here: http://www.amazon.de/Applied-Hierarchical-Modeling-Ecology-Marc/dp/0128013788
Book web site: www.mbr-pwrc.usgs.gov/pubanalysis/keryroylebook/
*** Kéry & Royle (2016): AHM Volume 2,  Dynamic and advanced models, due late 2016

***   Published hierarchical modeling books   ***
(1) Kéry (2010): Introduction to WinBUGS for Ecologists, Academic Press; see www.mbr-pwrc.usgs.gov/pubanalysis/kerybook 
(2) Kéry & Schaub (2012): Bayesian Population Analysis using WinBUGS, Academic Press; see www.vogelwarte.ch/bpa

***   Hierarchical modeling workshops: www.phidot.org/forum/viewforum.php?f=8

***   Hierarchical modeling Google Group mailing lists   ***
(1) unmarked: for questions specific to the R package unmarked
(2) hmecology: for material covered in Royle & Dorazio (2008), Kéry & Schaub (2012), Kéry & Royle (2015); see groups.google.com/forum/?hl=en#!forum/hmecology

From: unma...@googlegroups.com [unma...@googlegroups.com] on behalf of Don Segundo [bryan....@gmail.com]
Sent: 11 November 2015 16:41
To: unmarked
Cc: rcha...@warnell.uga.edu
Subject: Re: [unmarked] UnmarkedFrameGDS data

Don Segundo

unread,
Nov 11, 2015, 3:16:39 PM11/11/15
to unmarked, rcha...@warnell.uga.edu
Thank you Marc.

Im trying to code using the predict function but I cant seem to get it right...

In R Ive tried:
newdata<-
(cp=factor("CP", levels=c("CP22","CP23", "CP3A", "CP4D))
predict(cp, type="lambda", newdata=newdata)


It keeps telling me object is not a matrix.

Kery Marc

unread,
Nov 11, 2015, 3:26:02 PM11/11/15
to unma...@googlegroups.com
Dear Don Segundo,

in R, the first argument in the call to predict should be the name of the fitted model object, not the name of the covariate or factor for which you want to predict. It's the same in the predict method for unmarked.

Kind regards  --  Marc





Sent: 11 November 2015 21:16

Benedikt Schmidt

unread,
Nov 11, 2015, 3:56:21 PM11/11/15
to unma...@googlegroups.com
Dear Don Segundo,
 
The same, or at least a similar problem, was discussed a while ago:
 
 
Hope this helps.
 
Best,
Benedikt
 
 
 
Gesendet: Mittwoch, 11. November 2015 um 21:16 Uhr
Von: "Don Segundo" <bryan....@gmail.com>
An: unmarked <unma...@googlegroups.com>
Cc: rcha...@warnell.uga.edu
Betreff: Re: [unmarked] UnmarkedFrameGDS data

Don Segundo

unread,
Nov 11, 2015, 4:09:54 PM11/11/15
to unmarked
 Marc, 

The name of the model is the same as the name of the factor. Please see my code below:

>  a<-read.csv("C:/Users/Bryan/Desktop/CREP dissertation/ch.2CREPPopulationestimate/N-mixture data/BEVI_csv/BEVI2012_20mNoobscovscorrected.csv")

>  y<-a[,2:21]
>  R<-206
>  T<-4
>  site.cov<-(cp=a[22])
>  db<-c(0,20,40,60,80,100)
>
> beviumf<-unmarkedFrameGDS(y=y, siteCovs=site.cov, survey="point", dist.breaks=db,unitsIn="m", numPrimar=T)
> summary(beviumf)
unmarkedFrame Object

206 sites
Maximum number of observations per site: 20
Mean number of observations per site: 11.6
Number of primary survey periods: 4
Number of secondary survey periods: 1
Sites with at least one detection: 22

Tabulation of y observations:
   0    1    2 <NA>
2297   81   12 1730

Site-level covariates:
    cp   
 CP22:53 
 CP23:76 
 CP3A:45 
 CP4D:32 
>
>
> null<-gdistsamp(~1, ~1, ~1, beviumf, output="abund", mixture="P", K=50)
> nullNB<-gdistsamp(~1, ~1, ~1, beviumf, output="abund", mixture="NB", K=200)
> NBcp<-gdistsamp(~cp, ~1, ~1, beviumf, output="abund", mixture="NB", K=200)
> cp<-gdistsamp(~cp, ~1, ~1, beviumf, output="abund", mixture="NB", K=200)

> fmlist<-fitList(null, nullNB, NBcp, cp)

Warning message:
In fitList(null, nullNB, NBcp, cp) :
  Your list was unnamed, so model names were added as object names

> modSel (fmlist)
       nPars    AIC delta   AICwt cumltvWt
NBcp       7 566.96  0.00 5.0e-01     0.50
cp         7 566.96  0.00 5.0e-01     1.00
nullNB     4 576.45  9.49 4.3e-03     1.00
null       3 656.88 89.92 1.5e-20     1.00

> summary(cp)

Call:
gdistsamp(lambdaformula = ~cp, phiformula = ~1, pformula = ~1,
    data = beviumf, output = "abund", mixture = "NB", K = 200)

Abundance (log-scale):
            Estimate     SE      z  P(>|z|)
(Intercept)    -2.43  0.703 -3.458 0.000544
cpCP23          1.57  0.799  1.960 0.049985
cpCP3A          2.25  0.833  2.700 0.006944
cpCP4D         -9.06 65.337 -0.139 0.889763

Availability (logit-scale):
 Estimate   SE    z P(>|z|)
     7.46 57.3 0.13   0.896

Detection (log-scale):
 Estimate     SE    z P(>|z|)
     4.01 0.0966 41.6       0

Dispersion (log-scale):
 Estimate   SE     z  P(>|z|)
    -2.04 0.35 -5.83 5.65e-09

AIC: 566.9591
Number of sites: 206
optim convergence code: 0
optim iterations: 63
Bootstrap iterations: 0

> backTransform(cp, type="det")

Backtransformed linear combination(s) of Detection estimate(s)

 Estimate   SE LinComb (Intercept)
     55.4 5.35    4.01           1



> newdata<-data.frame(cp=factor("cp23", levels=c("cp23", "cp3d")))
> predict(cp, newdata=newdata, type="state")
Error in nrow(X) : object 'X' not found

Don Segundo

unread,
Nov 11, 2015, 4:57:48 PM11/11/15
to unmarked
 I got it to work by changing the model names and tweaking the coding:

> newdata<-data.frame(cp=factor("cp23", levels=c("cp23", "cp3d", "cp22", "cp4d")))
> predict(BEVIcp, newdata=newdata, type="lambda")

   Predicted         SE      lower     upper
1 0.08801466 0.06185246 0.02220095 0.3489301

Thanks to all who helped!!
Reply all
Reply to author
Forward
0 new messages