Troubleshooting for CANAPE analyses (cluster analysis)

132 views
Skip to first unread message

Louise Neo

unread,
May 3, 2019, 4:31:51 AM5/3/19
to Biodiverse Users
Dear all,

I have quite a basic / specific technical question.

I am trying out the CANAPE analysis, following the step-by-step example in the very helpful blog post: https://biodiverse-analysis-software.blogspot.com/2016/04/more-canape-how-to-restrict-your.html (thank you for this!)

I have managed to run the randomization but am getting stuck at the cluster analysis step.

I am using the default settings according to the example shown, and filled in the definition query as given (changing the output and randomization names):

sp_get_spatial_output_list_value (
    output => 'phylo_end', 
    list   => 'CANAPE>>SPATIAL_RESULTS', 
    index  => 'P_PE_WE'
) > 0.95
|| 
sp_get_spatial_output_list_value (
    output => 'phylo_end', 
    list   => 'CANAPE>>SPATIAL_RESULTS', 
    index  => 'P_PHYLO_RPE_NULL2'
) > 0.95 

However, I am not able to proceed as I keep receiving this Processing Error: "Index P_PE_WE does not exist at element 110.25:0.25". 

When I click on "show details", this is what I see: 
at Biodiverse/BaseData.pm line 4788.
at Biodiverse/BaseData.pm line 4788.
at Biodiverse/GUI/Tabs/Clustering.pm line 1289.
at Biodiverse/GUI/Tabs/Clustering.pm line 1289.

I don't understand why I am getting this error. I have checked that there is a value at the cell 110.25:-0.25, Output - P_PE_WE in the CANAPE>>SPATIAL_RESULTS list and that it is not empty.

I have also tried to re-run the randomization and try the cluster analysis on the new randomization results but am still getting the same error.

Any help or advice on this at all would be greatly appreciated! Thank you.

Best,
Louise

Louise Neo

unread,
May 3, 2019, 4:34:11 AM5/3/19
to Biodiverse Users
Sorry, to clarify, I am using Biodiverse version 2.1 in Windows.

Shawn Laffan

unread,
May 4, 2019, 3:17:41 AM5/4/19
to biodiver...@googlegroups.com
Hello Louise,

Thanks for providing all the details.  It makes it much easier to identify the likely source of the issue. 

In this case, the root cause (I think) is that your basedata has labels that are not on the tree.  If you then have one or more groups with no labels on the tree then some of the cells in the spatial output will not have the results for the various phylogenetic indices, e.g. PE_WE.  When the randomisation is run, Biodiverse only generates comparison indices like P_PE_WE when there is something to compare with. 

An issue with the basedata having more labels than the tree is that the randomisation is not guaranteed to assign the same number of tree tips to each cell in each iteration.  Most of the time one wants the same number of tips per cell across iterations, but there might be cases where it is useful to have differing numbers (the probability is perhaps very low, but it is non-zero).

I have opened a new issue to add more details to the error message, and it will be updated for Biodiverse version 3.   https://github.com/shawnlaffan/biodiverse/issues/730


So how to fix or avoid this right now?  There are, as always, several options.  My view is that option 2 is better in most cases, but it is your analysis and so your choice.

1.
Wrap the code in an eval{} block. 

This will trap the errors, and in this case ignore them.  It will also ignore any unrelated errors, but I would not expect any in this case. 

eval {
  sp_get_spatial_output_list_value (

    output => 'phylo_end', 
    list   => 'CANAPE>>SPATIAL_RESULTS', 
    index  => 'P_PE_WE',
    no_error_if_index_not_exists => 1,
  ) > 0.95
  || 
  sp_get_spatial_output_list_value (
    output => 'phylo_end', 
    list   => 'CANAPE>>SPATIAL_RESULTS', 
    index  => 'P_PHYLO_RPE_NULL2',
        no_error_if_index_not_exists => 1,
  ) > 0.95
};



2.
Trim your basedata to match the tree tips. 

This is done via the basedata menu using the "Trim to match tree nodes" option.  The issue with this approach is that you will need to re-run your randomisations, but this is perhaps better in any case to avoid differing numbers of tips across the randomisation iterations. 


Regards,
Shawn.
--
You received this message because you are subscribed to the Google Groups "Biodiverse Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to biodiverse-use...@googlegroups.com.
To post to this group, send email to biodiver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/biodiverse-users/5d357a54-3bb0-4ed7-a2e2-d56339202113%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Assoc Prof Shawn Laffan
  School of Biological, Earth and Environmental Sciences
  UNSW, Sydney 2052, Australia
  Tel +61 2 9385 8093
  http://www.bees.unsw.edu.au/staff/shawn-laffan
  https://shawnlaffan.github.io/biodiverse (free diversity analysis software)
  
  Editor, International Journal of Geographical Information Science
  http://www.tandf.co.uk/journals/ijgis

  UNSW CRICOS Provider Code 00098G

kostas kougioumoutzis

unread,
Jul 7, 2019, 12:29:30 PM7/7/19
to Biodiverse Users
Dear Shawn,

I've been reading your extremely thorough blog-post regarding the CANAPE analysis, experimenting with some data and everything runs smoothly. Yet, the only thing that puzzles me is this: which colour matches the paleo-, neo- and super-endemism cells? It's highly likely that I have overlooked something and I most probaly sound quite silly, so please excuse me if that's the case. 

Thank you in advance for your time and efforts.

Kind regards,

Kostas
To unsubscribe from this group and stop receiving emails from it, send an email to biodive...@googlegroups.com.
To post to this group, send email to biodive...@googlegroups.com.

Shawn Laffan

unread,
Jul 7, 2019, 6:10:26 PM7/7/19
to biodiver...@googlegroups.com, kostas kougioumoutzis
Hello Kostas,

Good to hear it all runs smoothly. 

The initial calculations are done in Biodiverse, but the assignment of cells into CANAPE classes currently needs to be done outside Biodiverse.

What system are you using to work with your spatial data?  (e.g. ArcGIS, QGIS, R).  Once I know I can have a go at developing some instructions and post them. 

And you would have read all the relevant posts, but for other readers on the list they are collated under the CANAPE label: 
https://biodiverse-analysis-software.blogspot.com/search/label/CANAPE


Regards,
Shawn.
To unsubscribe from this group and stop receiving emails from it, send an email to biodiverse-use...@googlegroups.com.
To post to this group, send email to biodiver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/biodiverse-users/feff1a15-5b53-4d28-b5ff-e9c62a19197d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

kostas kougioumoutzis

unread,
Jul 8, 2019, 12:57:45 AM7/8/19
to Biodiverse Users
Hello Shawn,

Thank you for your prompt reply. I am currently doing most of my spatial analyses in R, but please post any instructions in QGIS, ArcGIS or R, as long as it's the most convenient thing for you to do. Thank you once again.

Kind regards,

Kostas

Shawn Laffan

unread,
Jul 8, 2019, 11:29:53 PM7/8/19
to biodiver...@googlegroups.com
Hello Kostas,

If you are familiar with R then you might be able to extract something useful from the Biodiverse pipeline code.  That project needs an overhaul, but for this script you should only need to update the spatial extents of the scale bar. 

https://github.com/NunzioKnerr/biodiverse_pipeline/blob/master/R_release/plot_CANAPE_only_autoextent.R


Regards,
Shawn.
To unsubscribe from this group and stop receiving emails from it, send an email to biodiverse-use...@googlegroups.com.
To post to this group, send email to biodiver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/biodiverse-users/26653192-7b10-4c19-9de3-1d2d15f61c22%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages