Re: Issue with MaxEnt in R's dismo package

[David Galbraith]

Is it possible that the parallel processing in the Linux system is getting to a process step prior to an object being fully created and recognized by another process?

On Thu, Feb 7, 2013 at 11:45 AM, Luis Lucifora <luis.l...@gmail.com> wrote:

Hi everyone,

I'm trying to run MaxEnt within the dismo package in R 2.15.2 running in Ubuntu Linux 12.10 (64-bit).

The problem is that the maxent() function gives me an error saying "Error: No especies selected", after several warnings of extra fields in maxent temp files (please see below). When I check the temp files, there is actually a column named "species". I am working with just one species and the occurrence data are in a csv file containing only two columns (lon and lat), exactly as the program asks for. I am new using maxent and cannot figure out what is going on, so any help on this issue will be truly helpful.

I am posting below the whole R code, so that you can inspect the code and the errors by yourselves.

Thanks a lot,

Luis

-- 

Luis O. Lucifora

Instituto de Biología Subtropical - Iguazú

Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)

Universidad Nacional de Misiones

Centro de Investigaciones del Bosque Atlántico (CeIBA)

Casilla de Correo 9

Puerto Iguazú, Misiones, N3370AVQ

Argentina

R code:

> library(maps)

> library(rgdal)

Loading required package: sp

rgdal: version: 0.8-4, (SVN revision 431)

Geospatial Data Abstraction Library extensions to R successfully loaded

Loaded GDAL runtime: GDAL 1.9.0, released 2011/12/29

Path to GDAL shared files: /usr/share/gdal/1.9

Loaded PROJ.4 runtime: Rel. 4.7.1, 23 September 2009, [PJ_VERSION: 470]

Path to PROJ.4 shared files: (autodetected)

> library(maptools)

Loading required package: foreign

Loading required package: grid

Loading required package: lattice

Checking rgeos availability: TRUE

> library(dismo)

Loading required package: raster

raster 2.0-41 (21-December-2012)

> library(raster)

> 

> #occurrence data

> 

> ceto<-read.csv('/home/Work/projects/Cetorhinus/Cetorhinus_occur_dec_actualizado.csv', header=T,dec='.')

> attach(ceto)

> 

> #environment data

> files <- list.files(path= '/home/Work/projects/Cetorhinus/', pattern='asc', full.names=TRUE)

> 

> pred<-stack(files)

> 

> #limit study area

> e <- extent(-70, -40, -60, -10)

> predictors<-crop(pred,e)

> #plot(predictors)

> 

> #separating train and test data

> fold <- kfold(ceto, k=5)

> occtest <- ceto[fold == 1, ]

> occtrain <- ceto[fold != 1, ]

>

> #model

> me <- maxent(predictors, occtrain)

Loading required package: rJava

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 2: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 3: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 4: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 5: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 6: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 7: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 8: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 9: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 10: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 11: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 12: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 13: skipping... 

Warning: Extra fields in /tmp/R_raster_tmp/root/maxent/6106210171031/presence line 14: skipping... 

Error: No species selected

Error en file(con, "r") : no se puede abrir la conexión

Además: Mensajes de aviso perdidos

1: In .local(x, p, ...) :

  2 (13.33%) of the presence points have NA predictor values

2: In raster:::.couldBeLonLat(mask) :

  CRS is NA. Assuming it is longitude/latitude

3: In file(con, "r") :

  no fue posible abrir el archivo '/tmp/R_raster_tmp/root/maxent/6106210171031/species.lambdas': No existe el archivo o el directorio

> 

--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at http://groups.google.com/group/maxent?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[John Baumgartner]

Hi Luis,

Could you please post the output of head(ceto) and predictors? 

Also, it might be worth providing the path argument to maxent(), to prevent the files being saved to the temp directory. i.e., maxent(predictors, occtrain, path='/a/path/to/somewhere')

Cheers,

John

[Luis Lucifora]

Thank David and John for your answers.

David: I don't know, do you have any suggestion to see if that is the case?

John: Please, see below the output of head(ceto) and head(predictors). I added a path to maxent(), as you suggested, but nothing changed except that maxent files are now in my Cetorhinus folder rather than in the temp folder (see below please).

It's probably worth noting that I could make run MaxEnt outside R in this same system. However, I need to make it run from within R. Also, funny as it might sound the example from maxent() help in R runs without any problem in dismo. This makes me think that there is an issue with my data or the path to my data.

Thanks a lot to both of you.

Cheers,

Luis

R output:

> head(ceto)

    longdec    latdec

1 -68,33300 -51,04200

2 -67,09667 -56,46833

3 -65,35500 -54,99200

4 -64,68335 -42,25001

5 -64,63335 -42,68335

6 -58,13334 -51,68335

> head(predictors)

      chlomin chlorange dissox ph salinity sstrange

 [1,]      NA        NA     NA NA       NA       NA

 [2,]      NA        NA     NA NA       NA       NA

 [3,]      NA        NA     NA NA       NA       NA

 [4,]      NA        NA     NA NA       NA       NA

 [5,]      NA        NA     NA NA       NA       NA

 [6,]      NA        NA     NA NA       NA       NA

 [7,]      NA        NA     NA NA       NA       NA

 [8,]      NA        NA     NA NA       NA       NA

 [9,]      NA        NA     NA NA       NA       NA

[10,]      NA        NA     NA NA       NA       NA

[11,]      NA        NA     NA NA       NA       NA

[12,]      NA        NA     NA NA       NA       NA

[13,]      NA        NA     NA NA       NA       NA

[14,]      NA        NA     NA NA       NA       NA

[15,]      NA        NA     NA NA       NA       NA

[16,]      NA        NA     NA NA       NA       NA

[17,]      NA        NA     NA NA       NA       NA

[18,]      NA        NA     NA NA       NA       NA

[19,]      NA        NA     NA NA       NA       NA

[20,]      NA        NA     NA NA       NA       NA

> me <- maxent(predictors, occtrain, path='/home/Work/projects/Cetorhinus/')

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 2: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 3: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 4: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 5: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 6: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 7: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 8: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 9: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 10: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 11: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 12: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 13: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 14: skipping... 

Warning: Extra fields in /home/Work/projects/Cetorhinus/presence line 15: skipping... 

Error: No species selected

Error en file(con, "r") : no se puede abrir la conexión

Además: Mensajes de aviso perdidos

1: In .local(x, p, ...) :

  2 (12,5%) of the presence points have NA predictor values

2: In raster:::.couldBeLonLat(mask) :

  CRS is NA. Assuming it is longitude/latitude

3: In file(con, "r") :

  no fue posible abrir el archivo '/home/Work/projects/Cetorhinus//species.lambdas': No existe el archivo o el directorio

[Luis Lucifora]

I'm sorry John, I think I misunderstood your request. The output of
predictors is below.

Thank you,

Luis


> predictors
class : RasterBrick
dimensions : 600, 360, 216000, 6 (nrow, ncol, ncell, nlayers)
resolution : 0,08333333, 0,08333333 (x, y)
extent : -70, -40,00001, -60, -10 (xmin, xmax, ymin, ymax)
coord. ref. : NA
data source : in memory
names : chlomin, chlorange, dissox, ph, salinity, sstrange
min values : 0,029, 0,000, 4,709, 8,014, 21,594, 2,236
max values : 64,565, 48,538, 7,913, 8,264, 36,940, 17,637


2013/2/8 John Baumgartner <jbb...@gmail.com>:

[John Baumgartner]

Hi Luis. 

I think your problem stems from the fact that for your locale the decimal point is indicated by a comma. The maxent function involves writing out a table of all the predictor values associated with presence points (to the file 'presence') and a similar table for the sampled background (written to the file 'absence'). The function calls write.table with the arg sep=',' specified... this is a problem because it means that commas separate values but they also represent decimal points. These files are then passed to the maxent java program, which has a hard time understanding why there are so many commas.

I'm not sure how to fix this... you might be able to play with setting LC_NUMERIC in Sys.setlocale(), but I have no experience with this so can't really advise. If you want to use R for simplicity (keeping data preparation and maxent runs in the same environment), another option is to prepare data in R, then write it out to .csv files (you could even sample your background manually and save in SWD format), and then use the system() function to call maxent, rather than using dismo. A combination of sprintf and system can be very useful.

That said, I imagine there's an easy solution to this that I'm not aware of... you could try the r-sig-geo mailing list (http://list-archives.org/lists/r-sig-geo.r-project.org/f/2957345505), which Robert seems to keep an eye on. 

Good luck,

John

[John Baumgartner]

I meant to add that this would explain why the example given at ?maxent works for you... there are no decimal points in the presence nor background data.

JB

[Luis Lucifora]

Thanks a lot John!!!! It worked. The problem was that the language of
Ubuntu for dates and numbers was set as Spanish, which uses commas
instead of points for separating integers from decimals. I have noted
this, but R still worked with the commas instead of points, so I
didn´t think much of it. Actually, it was very weird to see decimal
commas in R because I built all my csv occurrence data using decimal
points, but the Spanish-set R transformed decimal points into commas!!
The solution was to set English as the Ubuntu language for numbers and
dates.

Thank you very much!!!

All the best,

Luis




2013/2/10 John Baumgartner <jbb...@gmail.com>:

[alaaeldeen80]

Dear Maxenters,

I'm a new in R and I would like to run maxent through R.
I used this commands
> library(raster)
> library(dismo)
> library(maps)
> library(rgdal)
> library(maps)
> Dorcs<-read.csv("/home/alaaeldin/Downloads/data/Dorcas.csv",sep=',',head=T)
> files <- list.files(path= '/home/alaaeldin/Downloads/data/Layers/all data in goegraphoc coordinate system/ascii/bio_asc/bio ascii', pattern='asc', full.names=TRUE)
> pred<-stack(files)
> fold <- kfold(Dorcas, k=5)
>Dorcastest <- Dorcas[fold == 1, ]
> Dorcastrain <- Dorcas[fold != 1, ]
> mymodel <- maxent(predictors, Dorcastrain)
> e <- extent(25, 40, 22,35)
> predictors<-crop(pred,e)
> me <- maxent(predictors, Dorcastrain)

But usually I got this error message
Error in .getMatrix(p) :
  data should be  a matrix, data.frame, or SpatialPoints* object

Which data meant here, environmental data or presence points?
how I can convert it to matrix or data frame?

This might be so easy for many of you but I'm new in R.
Your help is appreciated

Alaa

[John Baumgartner]

The error is referring to your occurrence data. I noticed that you've initially used "Dorcs" and then subsequently "Dorcas". You've also assigned your predictor stack to "preds" and then referred to it as "predictors" later. I imagine these are just typos in the code you've pasted here and aren't the cause of that particular error. However, the code you've given us looks fine otherwise... 

Does the following example from ?maxent work for you?

fnames <- list.files(path=paste(system.file(package="dismo"), '/ex', sep=''), pattern='grd', full.names=TRUE )

predictors <- stack(fnames)

occurence <- paste(system.file(package="dismo"), '/ex/bradypus.csv', sep='')

occ <- read.table(occurence, header=TRUE, sep=',')[,-1]

# witholding a 20% sample for testing 

fold <- kfold(occ, k=5)

occtest <- occ[fold == 1, ]

occtrain <- occ[fold != 1, ]

# fit model, biome is a categorical variable

me <- maxent(predictors, occtrain, factors='biome')

Visit this group at http://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

[Alaa Eldeen]

Thanks John for your prompt reply.

I have correct the previous mistakes you addressed in my commands but it still gives me same error message "Error in .getMatrix(p) :   data should be  a matrix, data.frame, or SpatialPoints* object"

I have checked my Dorcastrain data and I found it has only one column gathering XY and ofcaurse the same happened Dorcastest. Do you know how I can solve it?

Regarding the commands you sent, it works fine till the last line and then gives me this message
Loading required package: rJava
Error : .onLoad failed in loadNamespace() for 'rJava', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/home/alaaeldin/R/x86_64-pc-linux-gnu-library/3.0/rJava/libs/rJava.so':
  libjvm.so: cannot open shared object file: No such file or directory
Error in .rJava() : rJava cannot be loaded

I have attempted to install "rJava" package but it gives me this error
configure: error: Cannot compile a simple JNI program. See config.log for details.

Make sure you have Java Development Kit installed and correctly registered in R.
If in doubt, re-run "R CMD javareconf" as root.

ERROR: configuration failed for package ‘rJava’
* removing ‘/home/alaaeldin/R/x86_64-pc-linux-gnu-library/3.0/rJava’
* restoring previous ‘/home/alaaeldin/R/x86_64-pc-linux-gnu-library/3.0/rJava’

And I couldn't solve it as well.

Cheers

--

Alaa Eldin Soultan,  B.Sc, MSc.
Wildlife Conservationist,
St Katherine Protectorate,
National parks of Egypt,
Nature Conservation Sector,
Egyptian Environmental Affairs Agency, EEAA
Tel-Fax: + 2069- 3470033
Mobil:+201225773922

[John Baumgartner]

Can you show me what you mean by "gathering XY"? You'll need to separate them into separate columns for x and y. I can show you how if you provide an example.

As the second error suggests, rJava requires that you have JDK installed. If you don't have it already, Google will tell you where to get it (here's one source: http://www.oracle.com/technetwork/java/javase/downloads/index.html?ssSourceSiteId=otnjp).

[Alaa Eldeen]

Hi John,

I have attached the presence data"Sp.csv" and the divided data"occtrain" and "occtest"

Cheers

[John Baumgartner]

I can only view the file on my phone at the moment, and it seems there is no space or anything separating lon and lat. If these are all the records you have, just manually insert commas in each row, then use read.csv to bring it into R.

Otherwise, assuming your data.frame is called sp, and assuming that all latitudes have only 2 digits before the decimal point, you could do:

lon <- sub("(^\\d+\\.\\d+)\\d{2}\\..*", "\\1", so$LongLat)
lat <- sub("^\\d+\\.\\d+(\\d{2}\\..*)", "\\1", so$LongLat)
sp.new <- cbind(lon, lat)

I haven't tested this but I think syntax is correct.

Good luck!

[John Baumgartner]

Sorry, my phone auto-corrected sp to so in a couple of places..

[Alaa Eldeen]

Lots of thanks

I will try it and back to you.

Have a great day