TSS calculation

[Champ]

Dear All,

I want to calculate TSS for several species. I ran my maxent model with 10 replicates so i have 10 sample prediction files and 10 background predictions .  I am wondering how to calculate TSS with these 10 files. Previously I ran the model without replicating it and calculated the TSS as given by Alaa Eldeen (maxent google group communication). 

When I use 10 replicates do I have to calculate TSS 10 times and get the average? I do multiple species modeling and that would be difficult. Just want to know whether there is any other way of doing it.

Thanks in advance

Champ

[Kevin Morelle]

Hi,

I can only advise you to the great paper and supplementary material by Veronica Frans and colleagues: https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.12847

There (R code provided in the supp), you'll find all you need to calculate TSS for mutliple runs!

Good luck and thanks Veronica:)

Kevin

--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at https://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

[Husam El ALqamy]

Actually Alaa have also made an R script to calculate TSS for models with replication.

 

	Hossameldin ELALKAMY, MPhill., PhD. GIS Analyst

[Johannes Sörensen]

This is how I calculate the TSS in R. You can do it in excel as well. You have to enable background predictions in Maxent and then use the samples and background csv files for the values. As a threshold it probably makes sense to use one of the MaxSSS thresholds from the maxentResults.csv (Test or Training).

calc_tss<-function( samples, background, threshold ){

  sensitivity <- sum( samples$predict >= threshold ) / nrow( samples )
  specificity <- sum( background$predict < threshold ) / nrow( background )
 
  sensitivity + specificity - 1

[Champ]

Dear All,

I tried the R code given in https://groups.google.com/forum/#!msg/maxent/CUeI5xT9wTI/i3aibLdDOEYJ   to calculate the TSS of the maxent model that I ran with 10 replicates. However I received an error message (attached) at one point. I am not good in R . Grateful if somebody could help me to overcome this issue. 

Many thanks in advance

Champ

[Johannes Sörensen]

I didn't test the whole code but it seems that it is not tested well

Just have a look at these two lines:

tss_general <- as.data.frame(matrix(ncol = 3, nrow = 0))

...

rbind(tssgeral,tsssp)->>tssgeral

If you don't know R you should probably not use lapply because it is really complicated to debug and to understand for a beginner.

Have you tried my code, it's much simpler and easier to understand? Maybe remove the $predict part and only supply the columns from your predictions in samplesPredictions.csv, samplesPredictions.csv. You might have to change nrow to length then.

Begin with one set of files you open manually for just one species for a start and automate more once this is working. Have a look at the csv files and the column names (colnames on the dataframe once you opened it) and the data to really understand what you are doing.

[Veronica Frans]

Hey Champ, Kevin and Johannes,

I never calculated TSS before, so I used this question as a learning opportunity. Thanks, Johannes for sharing your calc_tss() function!

I think I got it to work for multiple runs, adapting from some work I did in the paper that Kevin had mentioned (https://doi.org/10.1111/2041-210X.12847). Thanks for the shout-out, Kevin! ;-)

Anyway, I am attaching here an Rmarkdown PDF file with my run-though on how to calculate TSS for multiple runs. I hope this will help you, Champ!

Johannes, since you've calculated TSS before, do you mind giving this PDF a quick glance? I don't want to misguide forum users in the future if I misused your function. (yikes!)

Cheers,

Veronica Frans

PhD student

Dept. Fisheries and Wildlife

Center for Systems Integration and Sustainability

Michigan State University

Virus-free. www.avast.com

--

[Johannes Sörensen]

Hi,

sorry it took me a while to respnd because I had a thesis to submit. I didn't test the code but it looks good.

I also written this function to determine the MaxSSS threshold. While MaxEnt does this for you, sometimes you may want to use random background data for the TSS while using other to train the model (e.g. for the targeted background approach).

In this case samples and background consists of a vector of the predicted values an not a data frame.

You can extract them from a raster of the projected maps.

calc_tss_threshold<-function( samples, background ){
  predict<-sort( c( samples, background ))
  predict<- unique( predict )
  t<- sapply( predict, FUN= function( x ) calc_tss( samples, background, x ))
  t<- t[,which( t[ 1, ]==max( t[ 1, ]))] # find background where TSS (Sensitivity + Specificity - 1 ) is Max which is equivalent to max( Sensitivity + Specificity ).
  return( as.matrix( t )[ , 1 ]) # Only get the first threshold if there are two with the same TSS
}

[Mohan Joshi]

OOPS. Here are the attachments.

[Mohan Joshi]

Hey Champ,

If you feel comfortable with arcmap. Here is the procedure to calculate TSS, kappa, error rate, accuracy and other parameters. If you have 10 replicates, then take the last sample prediction file and background predictions file. Follow the steps as mentioned in attached document. For your ease, i have also attached an excel sheet to calculate all the parameters. Just put the value of a,b,c,d in the excel sheet.

On Wednesday, December 12, 2018 at 8:03:00 AM UTC+5:45, Champ wrote:

[Champ]

Hi Mohan

I really appreciate for sharing this. Many thanks . 

Champ

[Husam El ALqamy]

Hi Mohan

In the method you suggested in the Arcmap procedure to create the error matrix. In step 3 you mentioned to classify the prediction raster into 1 and 2 for suitable and not suitable but you didn’t mention the threshold used. Also why don’t use a thresholded binary prediction raster from Maxent directly.

 

BCTS GIS	Hossameldin ELALKAMY, MPhill., RPBio GIS Analyst  BC Timber Sales |  Prince George Ministry of Forests, Lands and Natural Resource Operations P. 250.614.7521 C. 778.896.3229 | 2000 Ospika Blvd.  PG, BC., V2N 4W5
	Webpage | GIS Requests | Profile​

Antelope Specialist Group, IUCN

Research Profile

Join me on LinkedIn

--

[Mohan Joshi]

Hey husam,

While reclassifying the raster image, of course you have to use threshold value from maxentresult.csv file. Thanks for the clarification

On Tue, 25 Dec 2018, 12:34 Husam El ALqamy <alq...@gmail.com wrote:

max...@googlegroups.com is not on your Guest List | Approve sender | Approve domain

Hi Mohan

In the method you suggested in the Arcmap procedure to create the error matrix. In step 3 you mentioned to classify the prediction raster into 1 and 2 for suitable and not suitable but you didn’t mention the threshold used. Also why don’t use a thresholded binary prediction raster from Maxent directly.

 

BCTS GIS	Hossameldin ELALKAMY, MPhill., RPBio GIS Analyst  BC Timber Sales |  Prince George Ministry of Forests, Lands and Natural Resource Operations P. 250.614.7521 C. 778.896.3229 | 2000 Ospika Blvd.  PG, BC., V2N 4W5
	Webpage | GIS Requests | Profile​

Antelope Specialist Group, IUCN

Research Profile

Join me on LinkedIn

 

 

From: max...@googlegroups.com <max...@googlegroups.com> On Behalf Of Mohan Joshi
Sent: 17 ديسمبر, 2018 11:42 م
To: Maxent <max...@googlegroups.com>
Subject: [maxent] Re: TSS calculation

 

OOPS. Here are the attachments.

--

You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at https://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

[Taghi Ghassemi-Khademi]

Hi

Dear mohan

I did your procedure to calcalate TSS. But I couldn't acquire suitable results. The table of step 6 was a long table please can you guide me?

[MaxentNoob]

maybe just create two 'empty' vectors

myntss <- c(rep(0, times = n)
# n is number of replicate models
avgtss <- 0

then in a 'for loop' over i in 1:n where n is the number of replicate model predictions (ideally u
you have them in a list for indexing with double square brackets)

{
myntss[n] <- yourtssfunction("extract from myprexictiony[[i]]", "truth")
if (i == n) avgtss <- mean(myntss)
}

that should do it.

Rainer