Recommendations for comparing MaxEnt models?

[Cass Kalinski]

Hello,

I am looking for recommendations for best practice methods for comparing MaxEnt models. Ideally, I would like to be able to compare the output rasters of the two models as well as performance metrics on the models themselves. I am finding performance metric references for SDMs but am still trying to sort out what is recommended for MaxEnt modeling specifically. Quite frankly, I am overwhelmed with the variety of methods for both modeling and assessment.

Much of the literature seems focused performance metrics for regression models with presence/absence data. I am not seeing how to compare the spatial characteristics of the prediction rasters coming out of MaxEnt. What I am thinking would be useful, for example, would be differences in extent, differences in the concentration of predictions relative to the presence locations, etc. The raw outputs of MaxEnt preclude many of the standard GIS raster approaches. Or so I assume.  

References to articles that I can research and directional guidance on best practices rather than hand-holding is the expectation. I can figure it out from there. Any links to “how to” articles or blogs with examples in R or ArcGIS would be helpful as well. 

Context:
This is for a master’s thesis in GIS. The comparison is across four models: default versus tuned (using ENMeval) and at two different resolutions for the variables. Study area extent constrained to a 200km by 85km rectangle. One high elevation/treeline species with 74 localities. A second population area with 19 localities is available as a cross check of the trans portability of the models. Tuning was done with ENMeval and models built with dismo. Maxent.jar is being used, not maxnet. (Maxent.jar was an easier target to defend at this time given the relative newness of maxnet.)

Metrics generated so far include AUC, orMTP, or10pct, and AICc. Niche overlap will also be generated, though limited to default versus tuned at each resolution rather than across resolutions. Those were the metrics in scope with the thesis proposal to my committee but after working the data the last month or so, I think a spatial element is lacking. Considering additional performance metrics as well. TSS and kappa seem to be suggested candidates to add, though it is not clear if they are appropriate for MaxEnt and/or presence-only models. 

Short list of articles I am currently trying to absorb:

Allouche et al. 2006. "Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS)."  Journal of Applied Ecology 43 (6):1223-1232. doi: 10.1111/j.1365-2664.2006.01214.x.

Broennimann et al. 2011. "Measuring ecological niche overlap from occurrence and spatial environmental data."  Global Ecology and Biogeography 21 (4):481-497. doi: 10.1111/j.1466-8238.2011.00698.x.

Brunsdon et al. 2002. "Geographically weighted summary statistics — a framework for localised exploratory data analysis."  Computers, Environment and Urban Systems 26 (6):501-524. doi: https://doi.org/10.1016/S0198-9715(01)00009-6. [ck: Saw this one referenced elsewhere on this forum]

Leroy et al. 2018. "Without quality presence–absence data, discrimination metrics such as TSS can be misleading measures of model performance."  Journal of Biogeography 45 (9):1994-2002. doi: 10.1111/jbi.13402.

Liu et al. 2010. "Measuring and comparing the accuracy of species distribution models with presence–absence data."  Ecography 34 (2):232-243. doi: 10.1111/j.1600-0587.2010.06354.x.

Thank you,

Cass Kalinski

Graduate Student

USC Spatial Science Institute 

[Clarinda Wilson]

Hi Cass, this is a paper I've been referencing for my research on northern leopard frogs. The comparison of Maxent models is across environmental variables.

Good luck!

--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at https://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

--

Clarinda Wilson

Master in Environmental Management 

Integrative and Public Land Management Track

Western Colorado University

clarind...@western.edu
480-544-8161

[Cass Kalinski]

Very nice, thank you. There are several pieces of information in the paper that will be useful. The comparison of environmental variables is not pertinent to the focus of my study specifically, but how they evaluated and compared the model performance on the species distributions will be useful. 

Best of luck on your research. Thank you again!
Cass

[muna saba]

--