ENMeval error

[Daniele Da Re]

Hi All, 

I'm running Maxent from R and I am trying to get the ENMeval package to work to evaluate maxent models. 

First I run this model (occurrences of the species in south america), 

eval2.par_lut_sa <- ENMevaluate(occs, envs_sa_angel, bg, method='randomkfold', kfolds = 10, RMvalues=c(0.5,1,2), fc=c('L','LQ','LQP', 'LQPH'), parallel=TRUE)

where

> class(envs_sa_angel)

[1] "RasterStack"

attr(,"package")

[1] "raster"

> names(envs_sa_angel)

[1] "bio1"  "bio4"  "bio5"  "bio6"  "bio12" "bio15" "bio16" "bio17"

And I obtained the results I needed.

Then I tried to run the model with occurrences of the species in europe 

 

eval2.par_lut_eu <- ENMevaluate(occs_eu, envs_eu_angel, bg_lut_eu, method='randomkfold', kfolds = 10, RMvalues=c(0.5,1,2), fc=c('L','LQ','LQP', 'LQPH'), parallel=TRUE)

where

> class(envs_eu_angel)

[1] "RasterStack"

attr(,"package")

[1] "raster"

> names(envs_eu_angel)

[1] "bio1"  "bio4"  "bio5"  "bio6"  "bio12" "bio15" "bio16" "bio17"

But it gives me an error..

Doing random k-fold evaluation groups...

Of 16 total cores using 16

Running in parallel...

Error in file(fn, "rb") : cannot open the connection

In addition: Warning message:

In file(fn, "rb") :   cannot open file 'C:\Users\stirling\AppData\Local\Temp\RtmpKyikai\raster\r_tmp_2017-04-25_170521_19140_55426.gri': No such file or directory

Any suggestions?

Thank you in advance

Cheers

Daniele 

[Jamie M. Kass]

It looks like the raster stack is no longer in memory in your R session. Trying plotting one of the rasters, and if you get the same error, then that is the answer. Read them in again and try ENMevaluate again.

Jamie

[Daniele Da Re]

Hi Jamie, thank you for your answer!

 You are right, but what's happened it's really weird: envs_sa_angel and envs_eu_angel are two RasterStack cropped by a common rasterStack (with a worldwide extent), so I haven' t really understood why envs_sa_angel works, and envs_eu_angel not. I tried to download again all the bioclim tiles, merge and stack together all the tiles and finally cropping to South America and Europe extent. Still, the south america cropped rasterStack works, but the European one no. 

I don't really know how to say...

Cheers

Daniele  

[Daniele Da Re]

Ps. As you can see if I run 

plot(envs_eu_angel[[1]])

I obtain the correct plot

[Jamie M. Kass]

No good ideas except to start a new R session, read the data back in, process the rasterStacks, plot them, and if they plot and you can do other operations on them, they should work with ENMevaluate(). If they still don't, please email me and we'll figure out (perhaps you can send me your data).

Jamie

[Daniele Da Re]

Hi Jamie, 

I restart the new R sessions and find the error: The extent of the study area in Europe was not properly set (the extent was larger than the tiles) . So I solved the problem and ENMevaluate() run correctly. 

I have done some other models without problems with different extent and in different continents, but then when I have tried to do a model for the North America I got the same problem, but this time the extent is properly set. I also tried to save physically on the PC drive the raster stack used for the model, but it still continue to tell me that it is not able to find the temporary directory. If it is ok for you I will send you by email my data because I don't really know how to cope with this issue. 

Thank you in advance for you time and help!

Daniele  

[Sergio Marchant]

Hi Jamie,

I am having the same issues as Daniele. I can plot the raster brick and all the arguments in the command line seems valid to me (see below), but ENMevaluate will only work if parallel option is set to FALSE. What I think is going on is the temp files stored in AppData\Local\Temp are being erased too soon. 

I tried adding the command:

removeTMPFile(h=48)

 to my R script but that is not working either. 

Here is the command line I am using:

enmeval <- ENMevaluate(coord, predictors.sub, backgr, group, bg.grp, RMvalues=seq(0.5,1,2), fc= c("L","LQ","H","LQH","LQHP","LQHPT"), method="user", overlap=FALSE, aggregation.factor= c(2,2), kfolds=part, bin.output=TRUE, clamp=TRUE, parallel=TRUE)

The version of R that I am using is 3.4.0

dismo is 1.1-4

doParallel 1.0.10

ENMeval 0.2.2

foreach 1.4.3

iterators 1.0.8

raster 2.5-8

Rcpp 0.12.10

rgdal 1.2-7

rJava 0.9-8

sp 1.2-4

maxent 3.4.1

I have reinstalled everything from scratch and I have the same problems using the R console or RStudio. The problem is persistent in a windows 10 machine and OS X 10.12.4. I appreciate if you could give me more ideas on what I could do next to make Parallel=TRUE work.

Thanks!! 

[Daniele Da Re]

Hi guys, 

after Sergio's hint, I have tried to run the model with parallel=FALSE, and it run without problem!

[Jamie M. Kass]

Interesting... thanks Sergio for the solution! So there's a problem with the way parallel is working. ENMevaluate() works fine with RasterStacks, so perhaps the problem only occurs for RasterBricks? I'll try to do some investigation into this.

Jamie

[Gonzalo Pinilla-Buitrago]

Hi everyone,

Sergio's solution is the only way that I found to run ENMEvaluate(). I was having problems only for species with a big extend  (e.g. worldwide or two different continents). Jamie, in my case I was using RasterStacks, so maybe is not a problem for using RastersBricks. How Sergio said, perhaps it's a problem handling temp files. If you need to reproduce the error, I don't have the problem sending you my files.

Best,

Gonzalo

[Neander Marcel Heming]

Dear all,

After using ENMeval in parallel (and using raster bricks) for several months without problems, I got the same issue reported by Daniele. I've monitored the 'temp' folder and, as Sergio said, it seems that the files are erased too early.

Jamie, I looked at 'tuning' function and saw that the models, predictive maps, and statistics are gathered (lines 126-134) after the model tuning is performed (in parallel: lines 68-98, single core: lines 100-123). So, when 'stopCluster(c1)' is called (line 98), all connections are closed, the predictive maps in the temporary folder are deleted and will not be available for gathering the necessary data. I don't think that this would be hard to fix. The function could either (1) gather the necessary data before 'stopCluster(c1)' or (2) save the rasters in a location specified by the function and be erased later.

Hope this helps.

Best,

Neander