Sensitivity and specificity and evaluation of model (if not AUC then what?)

[KamilK]

Hi,

I used maxent in my paper and I used AUC as the main evaluation of model performance. My reviewer had problem with that and suggested to use other measures such as "sensitivity and specificity".

I have no problem with using different measures but I don't know how can I do this... Can somebody help me with that? If he mentioned "sensitivity and specificity" which value from maxent output he means (from file maxent.results)?

Equal training sensitivity and specificity cumulative threshold

Equal training sensitivity and specificity logistic threshold

Equal training sensitivity and specificity area

Equal training sensitivity and specificity training omission

Equal training sensitivity and specificity test omission

Equal training sensitivity and specificity binomial probability

Maximum training sensitivity plus specificity cumulative threshold

Maximum training sensitivity plus specificity logistic threshold

Maximum training sensitivity plus specificity area

Maximum training sensitivity plus specificity training omission

Maximum training sensitivity plus specificity test omission

Maximum training sensitivity plus specificity binomial probability

Equal test sensitivity and specificity cumulative threshold

Equal test sensitivity and specificity logistic threshold

Equal test sensitivity and specificity area

Equal test sensitivity and specificity training omission

Equal test sensitivity and specificity test omission

Equal test sensitivity and specificity binomial probability

Maximum test sensitivity plus specificity cumulative threshold

Maximum test sensitivity plus specificity logistic threshold

Maximum test sensitivity plus specificity area

Maximum test sensitivity plus specificity training omission

Maximum test sensitivity plus specificity test omission

Maximum test sensitivity plus specificity binomial probability

 ?

I'm very confused with all this names.... 

And finally what are the other metrics which I could calculate to say something about performance and trust of the model? 

Are there other values in file mentioned above which could also shed some light on that issue? 

I will be glad for any help!

Best regards,

Kamil

[Fabio Berzaghi]

Hey I am interested in this as well. Did you ever find out more information about this?

 

Thanks

[Camille]

Hi,

From a previous post about the same topic:

Regarding other methods to evaluate your models:

You could also use the null-model, described by Raes & Steege, (2007):

Raes, N., ter Steege, H., 2007. A null-model for significance testing of presence-only species distribution models. Ecography 30, 727–736.

If you have samples <25, you could use the jackknife method, described by Pearson's et al., (2007):

Pearson, R.G., Raxworthy, C.J., Nakamura, M., Townsend Peterson, A., 2007. Predicting species distributions from small numbers of occurrence records: a test case using cryptic geckos in Madagascar. Journal of Biogeography 34, 102–117.

Another method is the Boyce Index, which I find very interesting:

Boyce, M.S., Vernier, P.R., Nielsen, S.E., Schmiegelow, F.K.., 2002. Evaluating resource selection functions. Ecological Modelling 157, 281–300.

Hirzel, A.H., Le Lay, G., Helfer, V., Randin, C., Guisan, A., 2006. Evaluating the ability of habitat suitability models to predict species presences. Ecological Modelling 199, 142–152.

Camille.

[Camille]

The values you are listing are the different thresholds you can use to make binary maps, and the resulting size of the area predicted under these different thresholds.

[Alaa Eldeen]

Dear All,

I would suggest you to go through this paper

Allouche, O., Tsoar, A. & Kadmon, R. (2006) Assessing the accuracy of species distribution models:prevalence, kappa and the true skill statistic (TSS). J. Appl. Ecol, 43, 1223–1232.

Cheers

On Sat, Oct 6, 2012 at 7:52 PM, Christian Bermúdez Rivas <chris...@gmail.com> wrote:

Those are the threshold selection criteria. 

i recommend to you the book: "Ecological niches and geographic distributions", of Peterson et al. Chapter 9. There is an explanation about the specificity and sensitivity for model evaluation!

--
You received this message because you are subscribed to the Google Groups "Maxent" group.

To view this discussion on the web visit https://groups.google.com/d/msg/maxent/-/9KkJAgf128AJ.

To post to this group, send email to max...@googlegroups.com.
To unsubscribe from this group, send email to maxent+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/maxent?hl=en.

--

Alaa Eldin Soultan,  B.Sc, MSc.
Wildlife Conservationist,
St Katherine Protectorate,
National parks of Egypt,
Nature Conservation Sector,
Egyptian Environmental Affairs Agency, EEAA
Tel-Fax: + 2069- 3470033
Mobil:+201225773922