How to verify spatial projections with known species occurrence records ?

[Syed Amir Manzoor]

Dear All,

I've built a SDM using Maxent in a region 'A' and have projected the model to predict distribution in a region 'B'. 

Since I already have species occurrence records in the region 'B', is there a method of using these records to verify how accurately has Maxent predicted distribution in the region 'B'?

I have a couple of ideas in mind which are：

1. Convert the projected distribution map (projected to region 'B') into binary (suitable/unsuitable habitat) and use Accuracy Assessment in Arcgis to calculate how much of the known occurrence records fall in the two habitat suitability categories (say if 80-90% of the already known species occurrence records fall in the 'suitable habitat' class of the projected distribution map it would mean Maxent has been accurate/efficient in projecting model across the landscape).

2. Alternatively, build two SDM's. First SDM would be built for region 'A' and projected into the region 'B'. Second SDM would be built for region 'B' (with known occurrence records in this region). Then, compare the binary maps (the one for region 'B' with the one projected into 'B' from 'A'.) and in case the two binary maps have difference.

Else, please share if there's any statistical tool that offers a better and verified/published way of serving my purpose.

Thanks,

Amir

[mjb...@york.ac.uk]

Hello Amir,

I would go with 1. Up to you how you generate it, but a confusion matrix and the resulting sensitivity and specificity scores would be a very good way to show how well your model has projected. I have also used true skills statistic to give an overall idea of model accuracy. 

A good reference is FIELDING, A., & BELL, J. (1997). A review of methods for the assessment of prediction errors in conservation presence/absence models. Environmental Conservation, 24(1), 38-49.

cheers

[Jamie M. Kass]

Yes, evaluating the model trained on A based on the points from B, and vice versa, is definitely the way to test how transferable the model is between the areas. To do so, area A points become training data and area B points become testing data for the first round, then for the next round area A points become testing and area B become training. The evaluation statistics you use are up to you, but the commonly used ones are AUC, omission rates based on some threshold, or kappa, TSS, or other confusion matrix-based criteria.

You could implement this easily with ENMeval using method = "user" and defining the groups -- see the vignette for details. Others may have ideas for different ways to approach this.

Jamie Kass
PhD Candidate
City College of NY