Trouble creating umf for stacked data

[Rachel Moseley]

Hi,

I have 3 years of occupancy data from 45 sites. Each of these sites have been surveyed for 7 consecutive nights, 4 times a year, so I have a maximum of 84 observations. Originally I wanted to do a multiple season analysis but it has been suggested on this forum to stack the years of data and do a single season analysis. I have been trying to get my data in to an unmarked data frame, but as many others have mentioned, the csvToUMF function is converting all of my continuous variables to factors. While I have tried the various suggestions such as using "read.csv", removing the site covariates and adding them after the umf is created separately, and joining the y, siteCovs, and obsCovs .csv files, I have not found a solution to what is causing the issue. I believe my data (which has several obscovs) is in long format and while I won't be using every single covariate in the analysis, I'd like to bring everything in to R at once. What steps am I missing? I haven't found any other posts with solutions. I am attaching the .csv files and R file here. Any help is appreciated.

Cheers,

Rachel

[Adam Smith]

Hi Rachel,

I've created a branch that fixes this problem, at least for long data.frames (i.e., `long = TRUE`).  I'll try to get the pull request submitted soon, but for now you can install my copy of unmarked...

devtools::install_github("adamdsmith/unmarked", ref = "fix_csvumf")

Then your conversion will work:

library(unmarked)
bat <- csvToUMF(filename="~/HHB/HHB Analysis/R Analysis/Unmarked/Single Season Analysis/HHB3.csv", long = TRUE, type = "unmarkedFrameOccu")
str(bat)

Formal class 'unmarkedFrameOccu' [package "unmarked"] with 5 slots

  ..@ y       : 'matrix' int [1:45, 1:84] 1 0 0 1 0 0 0 0 0 0 ...

  .. ..- attr(*, "dimnames")=List of 2

  .. .. ..$ : NULL

  .. .. ..$ : NULL

  ..@ obsCovs :'data.frame': 3780 obs. of  19 variables:

  .. ..$ year      : num [1:3780] 3 3 3 3 3 3 3 2 1 2 ...

  .. ..$ jdate     : num [1:3780] -0.706 -0.718 -0.73 -0.741 -0.753 ...

  .. ..$ cycle     : Factor w/ 4 levels "lac","mate","preg",..: 4 4 4 4 4 4 4 4 4 4 ...

  .. ..$ season    : num [1:3780] 11 11 11 11 11 11 11 7 3 7 ...

  .. ..$ ssr       : num [1:3780] 27 27 27 27 27 27 27 7 119 7 ...

  .. ..$ msw       : num [1:3780] 6.16 6.16 6.16 6.16 6.16 ...

  .. ..$ myt       : num [1:3780] 11.9 11.9 11.9 11.9 11.9 ...

  .. ..$ myr       : num [1:3780] 6.64 6.64 6.64 6.64 6.64 ...

  .. ..$ myw       : num [1:3780] 4.66 4.66 4.66 4.66 4.66 ...

  .. ..$ night     : Factor w/ 756 levels "1-Aug-14","1-Aug-15",..: 738 33 61 91 123 153 181 735 29 30 ...

  .. ..$ mmt       : num [1:3780] 9.78 9.78 9.78 9.78 9.78 ...

  .. ..$ mmr       : num [1:3780] 0.0323 0.0323 0.0323 0.0323 0.0323 ...

  .. ..$ mst       : num [1:3780] 12.9 12.9 12.9 12.9 12.9 ...

  .. ..$ month     : num [1:3780] 1 1 1 1 1 1 1 2 2 2 ...

  .. ..$ year.     : num [1:3780] 2017 2017 2017 2017 2017 ...

  .. ..$ mmStudyt  : num [1:3780] 9.17 9.17 9.17 9.17 9.17 ...

  .. ..$ mmStudyr  : num [1:3780] 12.2 12.2 12.2 12.2 12.2 12.2 12.2 4 4 4 ...

  .. ..$ mmStudyw  : num [1:3780] 5.22 5.22 5.22 5.22 5.22 ...

  .. ..$ JulianDate: num [1:3780] 9 10 11 12 13 14 15 40 41 41 ...

  ..@ siteCovs:'data.frame': 45 obs. of  3 variables:

  .. ..$ mStudyt: num [1:45] 11.6 11.6 11.6 11.6 11.6 ...

  .. ..$ mStudyr: num [1:45] 9.52 9.52 9.52 9.52 9.52 ...

  .. ..$ mStudyW: num [1:45] 4.02 4.02 4.02 4.02 4.02 ...

  ..@ mapInfo : NULL

  ..@ obsToY  : num [1:84, 1:84] 1 0 0 0 0 0 0 0 0 0 ...

Note that the function now identifies site covariates automatically so you don't have to add them in manually.  For example, if you combined your observation and additional site data into a single csv, it will (hopefully) distribute obsCovs and siteCovs correctly:

dat <- read.csv("~/HHB/HHB Analysis/R Analysis/Unmarked/Single Season Analysis/HHB3.csv")
sitecovs <- read.csv("~/HHB/HHB Analysis/R Analysis/Unmarked/Single Season Analysis/sitecovs3.csv" )
everything <- cbind(dat, sitecovs)
write.csv(everything, "~/HHB/HHB Analysis/R Analysis/Unmarked/Single Season Analysis/everything.csv", row.names = FALSE)
bat <- csvToUMF(filename = "~/HHB/HHB Analysis/R Analysis/Unmarked/Single Season Analysis/everything.csv", long = TRUE, type = "unmarkedFrameOccu")
str(bat)
Formal class 'unmarkedFrameOccu' [package "unmarked"] with 5 slots
  ..@ y       : 'matrix' int [1:45, 1:84] 1 0 0 1 0 0 0 0 0 0 ...
  .. ..- attr(*, "dimnames")=List of 2
  .. .. ..$ : NULL
  .. .. ..$ : NULL
  ..@ obsCovs :'data.frame': 3780 obs. of  19 variables:
  .. ..$ year      : num [1:3780] 3 3 3 3 3 3 3 2 1 2 ...
  .. ..$ jdate     : num [1:3780] -0.706 -0.718 -0.73 -0.741 -0.753 ...
  .. ..$ cycle     : Factor w/ 4 levels "lac","mate","preg",..: 4 4 4 4 4 4 4 4 4 4 ...
  .. ..$ season    : num [1:3780] 11 11 11 11 11 11 11 7 3 7 ...
  .. ..$ ssr       : num [1:3780] 27 27 27 27 27 27 27 7 119 7 ...
  .. ..$ msw       : num [1:3780] 6.16 6.16 6.16 6.16 6.16 ...
  .. ..$ myt       : num [1:3780] 11.9 11.9 11.9 11.9 11.9 ...
  .. ..$ myr       : num [1:3780] 6.64 6.64 6.64 6.64 6.64 ...
  .. ..$ myw       : num [1:3780] 4.66 4.66 4.66 4.66 4.66 ...
  .. ..$ night     : Factor w/ 756 levels "1-Aug-14","1-Aug-15",..: 738 33 61 91 123 153 181 735 29 30 ...
  .. ..$ mmt       : num [1:3780] 9.78 9.78 9.78 9.78 9.78 ...
  .. ..$ mmr       : num [1:3780] 0.0323 0.0323 0.0323 0.0323 0.0323 ...
  .. ..$ mst       : num [1:3780] 12.9 12.9 12.9 12.9 12.9 ...
  .. ..$ month     : num [1:3780] 1 1 1 1 1 1 1 2 2 2 ...
  .. ..$ year.     : num [1:3780] 2017 2017 2017 2017 2017 ...
  .. ..$ mmStudyt  : num [1:3780] 9.17 9.17 9.17 9.17 9.17 ...
  .. ..$ mmStudyr  : num [1:3780] 12.2 12.2 12.2 12.2 12.2 12.2 12.2 4 4 4 ...
  .. ..$ mmStudyw  : num [1:3780] 5.22 5.22 5.22 5.22 5.22 ...
  .. ..$ JulianDate: num [1:3780] 9 10 11 12 13 14 15 40 41 41 ...
  ..@ siteCovs:'data.frame': 45 obs. of  5 variables:
  .. ..$ mStudyt: num [1:45] 11.6 11.6 11.6 11.6 11.6 ...
  .. ..$ mStudyr: num [1:45] 9.52 9.52 9.52 9.52 9.52 ...
  .. ..$ mStudyW: num [1:45] 4.02 4.02 4.02 4.02 4.02 ...
  .. ..$ region : num [1:45] 1 1 1 1 1 1 1 1 1 2 ...
  .. ..$ treehab: num [1:45] 1 0 0 1 0 0 1 1 0 1 ...
  ..@ mapInfo : NULL
  ..@ obsToY  : num [1:84, 1:84] 1 0 0 0 0 0 0 0 0 0 ...

Best,

Adam

[Richard Chandler]

Thanks Adam, this is great. We'll be sure to merge in your changes before the next release. 

Richard

--
You received this message because you are subscribed to the Google Groups "unmarked" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Richard Chandler

Assistant Professor

Warnell School of Forestry and Natural Resources

University of Georgia

706-542-5815

chandlerlab.uga.edu

[Rachel Moseley]

Thanks so much for your contributions as always, Adam! I will try it out this weekend.

Cheers,

Rachel

[Rachel Moseley]

A summary of the data shows that site and observation covariates are correct and that "bat" is an unmarked frame object but when I run a model I get the following error message:

> fm0 <- occu(~1, ~1, bat)

Error in occu(~1, ~1, bat) : Data is not an unmarkedFrameOccu object.

Any idea why the model is not recognizing the umf correctly?

[Adam Smith]

Hi Rachel,

There's no comma between formulae:

fm0 <- occu(~1 ~1, bat)

A

--
You received this message because you are subscribed to a topic in the Google Groups "unmarked" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/unmarked/_leESWUENZA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to unmarked+unsubscribe@googlegroups.com.

[Rachel Moseley]

Right! Thanks. Although now I have a different error:

> fm0 <- occu(~1 ~1, bat)

Error in rowSums(y, na.rm = TRUE) : 'x' must be numeric

I ran str(dat) which showed that the detection history is being stored as a factor.

$ det     : Factor w/ 3 levels ".","0","1": 2 2 3 2 2 2 3 2 2 2 ...

I ran str(bat) to see the structure of the umf and it looks like the detection data is being stored as a character rather than as a number :

Formal class 'unmarkedFrameOccu' [package "unmarked"] with 5 slots

  ..@ y       : 'matrix' chr [1:45, 1:84] "1" "0" "0" "1" ...

I'm not sure if this is the key issue, but I tried running the code  dat$det<-as.integer(dat$det) but it doesn't change that y is a character. I decided to read in a csv that has all of the data combined and formatted for csvToUMF. I'll attach it here although I get the same error when combining the 2 data sets. I'm not sure why the y matrix is a character rather than an integer even when I create a umf from the original .csv files that I sent you. 

Cheers,

Rachel

To unsubscribe from this group and all its topics, send an email to unmarked+u...@googlegroups.com.

[Adam Smith]

If I recall correctly, that likely has to do with you having periods "." as missing values. 

Adam

<csv_umf_fix_error.R>

<csv_umf_test.csv>

[Rachel Moseley]

Ok, but why would the same csv files produce a y matrix formatted as a character when I run the code, but produce a y matrix formatted as an integer when you run the code? Is it best just to remove those missing values?

Cheers,
Rache

[Adam Smith]

Yes. I recall removing those missing values. Sorry, should have mentioned that.  If detection info is missing, there's no point including in the model unless you're trying to predict occupancy or detection prob for that suite of covariates (using the 'predict' method for an unmarked FitOccu), in which case you'll need a proper NA value instead of a period. 

Those periods gave me SAS nightmares. 😉

Adam

[Rachel Moseley]

Great! Thank you, Adam. All works perfectly now.

Cheers,

Rachel

[Geraldine Klarenberg]

Thank you - has this update been included?

I was having trouble with using csvToUMF for a long format file with the regular installation: it was not recognizing my site variables. It seems like the most recent 'unmarked' version is from May 2017, so I suppose it has not been updated with this yet?

Everything worked great with Adam's github update (note for those on new Macs: you might run into issues with gfortran, see how to fix that here: https://thecoatlessprofessor.com/programming/rcpp-rcpparmadillo-and-os-x-mavericks--lgfortran-and--lquadmath-error/).

Also - note that the csvToUMF documentation says that the order of your data has to be id-observations-sitecovs-obscovs, which didn't work. When I put sitecovs at the end, it did work.

But overall, thanks so much for this update. This was driving me up the wall (my data set is so large, I couldn't figure out how to change it from long to wide myself either).

Geraldine

To unsubscribe from this group and stop receiving emails from it, send an email to unmarked+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.