Incorporating an independent testing set in ENMeval

[Fabiola Rodriguez]

Hello everyone,

I wanted to ask if anyone here has ideas or tips of how to use ENMeval to include an independent testing set? In Maxent's GUI you can include it. Since ENMeval has its own testing features:jack-knife, etc, I wonder if it would be valid to run an analysis on each training and testing set separately in ENMeval and then compare them? As others using ENMeval, the reason I would like to do it with this package is the benefits of building multiple models and obtaining AIC values.

I'm interested in hearing, your thoughts.

Best

Fabiola 

[Jamie M. Kass]

Fabiola,

What I’ve done is run ENMeval, then loop over the model objects in the ENMevaluation object and use dismo::evaluate() on your independent dataset. You can then load the results into a vector. Please let me know if you’d like an example.

Also, do you mean AUC or AIC? AIC does not incorporate evaluations on independent data, but AUC can.

Jamie Kass
PhD Candidate
City College of NY

[Dimitris Poursanidis]

Hm, nice approach.

Any R example code is appreciated :)

D.

>->->->->->->->->->->->->->->->-

Dimitris Poursanidis, Ph.D

http://rslab.gr/

http://www.terrasolutions.eu

Google Scholar

ResearchGate

ORDIC

Remote Sensing in Ecology and Conservation - Editorial Board

The latest papers !!! 

1) https://www.frontiersin.org/articles/10.3389/fmars.2017.00347/abstract

2) http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0187143

--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+unsubscribe@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at https://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

[Jamie M. Kass]

Sure, it would look something like this, assuming "e" is your ENMevaluation object, "x" is your independent dataset, "bg" is your background points, and "z" is your RasterStack of predictor variables.

# number of models total
n <- length(e@models)
# empty vector for aucs
aucs <- numeric(n)
# extract the raster values for each dataset
x.vals <- extract(z, x)
bg.vals <- extract(z, bg)
for(i in n) {
  # model at index i
  m <- e@models[[i]]
  # run evaluation (calculate confusion matrix)
  v <- evaluate(x.vals, bg.vals, m)
  # load the resulting AUC into the vector
  aucs[i] <- v@auc
}
 

I haven't tested it, so it may require a little tweaking, but it should work. Also, you can pull out any slot in the object that comes out of evaluate() to record in your vector. Hope this helps.

Jamie Kass
PhD Candidate
City College of NY