TSS in R

[yaelle...@gmail.com]

Hi everyone,

Is anyone know of a package in R to calculate TSS? I have looked everywhere but couldn't find anything...

Thanks

[Kevin Carolan]

What's TSS?

--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at http://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

[Michael Treglia]

TSS is simple to calculate, as seen in:
Allouche, O., A. Tsoar, and R. Kadmon. 2006. Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS). Journal of Applied Ecology 43:1223-1232.

It's just "Sensitivity + Specificity - 1"

I haven't used 'dismo' in a while, but from looking at the documentation it looks like you should be able to get Sensitivity and Specificity (true positive rate and true negative rate, respectively) by running the 'evaluate' function and looking at the resulting ModelEvaluation object.  (I'm looking at the documentation here: http://cran.r-project.org/web/packages/dismo/dismo.pdf)

if you're not using dismo, you should be able to get these measures via the packages ROCR and PresenceAbsence.

hope that helps
mike

[Gustavo Bediaga]

Here is some code I´m using that calculates TSS from Maxent output:

I am using replicates, and the Maximum.training.sensitivity.plus.specificity.logistic.threshold threshold rule.

If you are using a different threshold rule, you have to change it in the script.

You will have to change the maxent output directory too.

It will generate a file named __tssgeral.csv, in the Maxent output directory wich has three columns: 1. Name of replica, 2. TSS, 3. Binomial Probability

It will only work if the  "Write background predictions" option is marked in Maxent.

Hope it helps, I have translated some of the code so you can understand the parts that may need changes.

Also, read this https://groups.google.com/forum/#!topic/maxent/eCgJ_0vdOb0

Gustavo

##########################################

#### Avaliação de modelos por TSS

#### a partir do output do Maxent

####

#### A opção "Write background predictions" tem que

#### estar habilitada no Maxent

####

##########################################

library(ROCR)

library(vcd)

library(boot)

##### Output Maxent Directory

setwd("E:/MaxentOutput")

list.files(pattern="_samplePredictions.csv")->listaoutput

sub("_samplePredictions.csv","",listaoutput)->listaoutput

############

## Run it from here

#############

tssgeral <- as.data.frame(matrix(ncol = 3, nrow = 0))

colnames(tssgeral)<-c("Replica","TSS","Bin.Prob")

lapply(listaoutput, function(nome){

  nome->sp

presence <- read.csv(paste(sp,"_samplePredictions.csv",sep=""))

background <- read.csv(paste(sp,"_backgroundPredictions.csv",sep=""))

pp <- presence$Logistic.prediction                # get the column of predictions

testpp <- pp[presence$Test.or.train=="test"]       # select only test points

trainpp <- pp[presence$Test.or.train=="train"]   # select only test points

bb <- background$logistic

combined <- c(testpp, bb)                             # combine into a single vector

###https://groups.google.com/forum/#!topic/maxent/eCgJ_0vdOb0

read.csv("maxentResults.csv")->maxres

### Set the threshold rule here

threshold<-maxres[maxres[,1]==nome,"Maximum.training.sensitivity.plus.specificity.logistic.threshold"]

bin.prob<-maxres[maxres[,1]==nome,"Maximum.training.sensitivity.plus.specificity.binomial.probability"]

#Número de valores maior e menor que threshold no teste

sum(testpp > threshold)->maiortest

sum(testpp < threshold)->menortest

#Número de valores maior e menor que o threshold no background

sum(bb > threshold)->maiorbb

sum(bb < threshold)->menorbb

### Calcula a sensitivity e a specificity

sensit<-(maiortest)/(maiortest+menortest)

specit<-(menorbb)/(maiorbb+menorbb)

tss<-sensit+specit-1

tsssp<-as.data.frame(sp)

tsssp[2]<-tss

tsssp[3]<-bin.prob

colnames(tsssp)<-c("Replica","TSS","Bin.Prob")

rbind(tssgeral,tsssp)->>tssgeral

})

#########

## Escreve o arquivo final

########

write.csv(tssgeral,file="__tssgeral.csv")

####

#  Run it until here

###

[Oscar Martínez]

Hi All!!!! 

I have a big question, which is the threshold you should use? does any one have an idea? 

In the model evaluation-AUC discussion, they say to use the threshold 10 percentile: "If you wanna to identify your threshold value, I would suggest you to use 10 percentile threshold value as it reported to be is the best.In order to to get it, you need to open the html file located in the output directory of the maxnet file you will find a table, see the attached file, and the select the value in the second column (logistic threshold) in the 5th cell, this value is the threshold value that you will use in TSS."

But, the 5th cell says: 10 percentile training presence, "training" not "test". Does anyone have an idea about this? I thought that maybe it was because you are using the training data to evaluate the model in the novel area, but I am not sure about this. 

Thanks! 

Oscar

[kaky]

Dear Gustavo

I apply your code for TSS but i don't got any results, i think there are some code remain to write to give me results or  may be i wrong can you help please.

regards

emad 

[kaky]

dear Gustavo

can you help me to apply your code to do TSS

regards

emad
On Thursday, 30 October 2014 17:27:01 UTC, Gustavo Bediaga wrote:

[Gustavo Bediaga]

Sorry, i didn´t see this post before.

I have answered your other post.

Hope that helps.

Gustavo

[Sony Rk]

Dear Gustavo,

I tried this code. But it is not giving the results other than a csv file with the three colums with its respective names. Could you please help me in this regard.

[Alaa Eldeen]

Hi Sony,

I'm sorry I didn't get your question,what is the problem in this code?

--

You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at http://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

--

Alaaeldin Soultan
PhD researcher
Max Planck Institute for Ornithology
Am Obstberg 1
78315 Radolfzell, Germany
Tel: +49 (0)7732 1501 911
Mob.+4915757474779


Ps:  It takes 24 trees to produce 1 ton of office paper! Think… is it really necessary to print this email?

[Sarah Cantarino]

Hi, I know its been a long time since this was discussed but I still didnt understand how to deal with the replicates.

I calculated the TSS for the replica

​te​

s

​ in R. Each replicate model​

had a different TSS value (Figure below).

 6 of the 10 models add a fair value (<0.4).

So, what should I do:

Choose the one of the 10 models that have the best TSS value or

run more and more models until all of the TSS value are good enough for use the Avg model, as recommended by most of the articles?

C

​ould you recommend me 

 a article about this question (using one of the 10 replicates)?

My validation test with external points its getting very bad values and I thought that maybe it I could use a better model, it would improve the validation outcomes.

Thanks.

Sincerely,

Sarah Cantarino.

​

__________________________________
Sarah de Jesus Cantarino
Graduada em Ciências Biológicas - Licenciatura
Universidade Federal do Espírito Santo - Brasil

Visit this group at https://groups.google.com/group/maxent.

[mjb...@york.ac.uk]

Gustavo, I came across your code after hunting for a simple way to produce TSS from maxent models and I really wanted to thank you for posting your code! It's been a lifesaver as the paper we are working on has had a sudden change of emphasis!

Very much appreciated.

On Thursday, 30 October 2014 17:27:01 UTC, Gustavo Bediaga wrote:

[Dimitris Poursanidis]

After using several times the code for TSS in several models, lastly in a new model i recieve NaN in the TSS columns.

Calculating the TSS with the excel file, i receive Negative values. Is that a reason of NaN in R code ?

[Dimitris Poursanidis]

Hi team

Has anybody face similar issue by using the maxent results from version 3.4.1 ?

D.

[Alaaeldin Soultan]

Hi Dimitris,

which R code you use to calculate the TSS?

--

You received this message because you are subscribed to the Google Groups "Maxent" group.

To unsubscribe from this group and stop receiving emails from it, send an email to maxent+unsubscribe@googlegroups.com.

To post to this group, send email to max...@googlegroups.com.
Visit this group at https://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

--

Alaaeldin Soultan
PhD researcher
MaxPlanck Institute for Ornithology

Am Obstberg 1
78315 Radolfzell, Germany

Tel: +49 (0)7732 1501 35
Mob.+4915757474779

[Dimitris Poursanidis]

Attached the code and attached the data from maxent 3.4.1. for which i receive the error >

 Error in names(x) <- value :  'names' attribute [3] must be the same length as the vector [2] 

To traceball δινει αυτό >

3.`colnames<-`(`*tmp*`, value = c("Replica", "TSS", "Bin.Prob")) 

2.FUN(X[[i]], ...) 

1. lapply(listaoutput, function(nome) {

    sp <- nome

    presence <- read.csv(paste(sp, "_samplePredictions.csv", 

        sep = "")) ... 

i have this error when change between threshold rule and for some rules i have NA in TSS column.

Any help is much appreciated 

Best

D.

>->->->->->->->->->->->->->->->-

Dimitris Poursanidis, Ph.D

http://rslab.gr/

http://www.terrasolutions.eu

Google Scholar

ResearchGate

ORDIC

Remote Sensing in Ecology and Conservation - Editorial Board

The latest papers !!! 

1) https://www.frontiersin.org/articles/10.3389/fmars.2017.00347/abstract

2) http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0187143

[Alaaeldin Soultan]

I found the error.

In line 8 and line 48 you need to use  "bb <- background$Logistic" instead " bb <- background$logistic".

I mean  you have to use capital "L" as it is in your sample background file.

Please change it and run the code again and let me know if it helps

Cheers

[Alaaeldin Soultan]

By the way,

in you case you have also to replace
names(max_res)[95]

with
names(max_res)[127]