Choosing a threshold

[Kevin]

Hi all,

I'm trying to figure out the ideal way to choose a threshold for making binary maps. Two things:

I've seen some recent publications say that using "maximum sensitivity plus specificity" or "max sum of sensitivity plus specificity" (maxSSS) is the best threshold (e.g. Liu et al. 2013 http://onlinelibrary.wiley.com/doi/10.1111/jbi.12058/full and Liu et al 2015 http://onlinelibrary.wiley.com/doi/10.1002/ece3.1878/full ), but they do not specify if this is the training or the testing maxSSS. I'm guessing training? Second,  if I have run Maxent with 10 k-folds but want to choose the threshold for the mean model output, is it ideal to determine maxSSS by taking the mean of all maxSS values for each run?

If not maxSSS, has anyone seen any literature that identifies a better way to choose a threshold? I know some pubs have brought up issues with using these thresholds at all, but I'm unsure of an alternative for what I'm trying to do (compare differences in predicted suitable habitat in 2070 between a model using a full species's range, versus combining models where I split up that species based on phylogenetic lineages)

Thanks for any help!

Kevin

[mjb...@york.ac.uk]

Hello Kevin,

Liu et al is a good source for your threshold value choice. Partially it depends on what you are trying to achieve with your model to which threshold value you choose. For my own research I use maximum test sensitivity plus specificity log threshold. I set maxent to use crossvalidation, over 10 runs - the threshold value for which is in the maxentResults.csv.

I would be strongly inclined not to use training values, but again it depends on what you want from maxent. From your post, I recommend using a suitable test value.

Michael

[Jamie M. Kass]

Although many people use Liu et al for a reference for choosing thresholds, I would also advise you to make an educated decision based on your system and research questions. The study in Liu et al, although it makes suggestions for the field, is hardly comprehensive for all studies, and if max SSS results in an ecologically implausible potential distribution for your species, test other thresholds and simply examine them. Don't make entirely qualitative assessments of thresholds, but don't be afraid to be the expert and make judgment calls.

Jamie Kass
PhD Candidate
City College of NY

[Adam Smith]

I agree with Jamie's note. It's worth noting that the conclusions in Liu et al assume that the presences on which you calculate TSS are randomly (representatively) distributed across the area to which you want to apply the threshold. In other words, if there is sampling bias, then you can't be assured the maxTSS threshold would be the same calculated using background sites or calculated using absences, which is the main reason they advocate using maxTSS. This said, I personally often use maxTSS simply because it visually produces reasonable results.

Also note there's a good argument for not thresholding: it masks differences between, say, sites that are just above the threshold and those that are highly suitable and far above it. So you might think about a middle approach where you apply three (say) thresholds of increasing value to demarcate marginally suitable from suitable from highly suitable.

Adam

Assistant Scientist in Global Change
Missouri Botanical Garden