Remove highly correlated variables

[Sanjo Jose]

Hello,

I was trying to remove highly correlated variables using SDM toolbox. I given .tif files of 19 bioclimatic variables and set the no data value '-3.40282346639e+038'. Maximum correlation allowed was 0.6; 0.7; 0.8; 0.9. But I'm receiving an error message "with base 10: '-3.40282346639e+038'    failed to execute (removecorrel)". What would be the problem?

Thank you.

[Robert Pace]

Hi Sanjo,

SDM Toolbox has some technical issues with  ArcGIS  version 10.3 and newer.

See link below:

http://sdmtoolbox.org/

[Jamie M. Kass]

In case you're an R user, there are lots of useful "step" functions for removing correlated variables from a predictor dataset based on some threshold of correlation. I can recommend a function I found easy to use: vifstep() and related functions from package usdm. It uses a variance inflation factor and tolerance threshold to remove variables stepwise until none remain with a certain level of correlation.

Jamie M. Kass
PhD Candidate
City College, NYC

[Robert Pace]

ArcGIS can create correlation matrix for your raster data layers, and it can do principle component analysis (PCA) along with output of the eigenvectors and eigenvalues.

To calculate only the correlation matrix in ArcGIS

1.  Load all your rasters.

2.  Open ArcToolbox

3.  Click on the Spatial Analyst folder
4.  Click on Multivariate folder

5.  Click on Band Collection Statistics 

6.  Input all your rasters

7.  Check the Complete Covariance and Correlation Matrices

8.  Designate an Output Statistics File (eg. correlation.txt)

8.  Click OK.

The Output Statistics File will contain the correlation information.

If you want to run a PCA of your data rasters follow the steps 1 through 4 above followed by:

5.  Click Principal Components

6.  Input all your rasters

7.  Designate an output raster to hold the PCA output

8.  Designate output data file (eg. PCA.txt)

9.  Click OK.

The PCA.txt will contain the correlation matrix, co variance coefficients and matrix, along with eigenvalues and eigenvector data.  I usually import this file into Excel and use Text to Columns to make it a bit more readable.

On Monday, December 12, 2016 at 11:11:00 PM UTC-5, Sanjo Jose wrote:

[Sanjo Jose]

Thank you jamie and robert.

--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+unsubscribe@googlegroups.com.
To post to this group, send email to max...@googlegroups.com.
Visit this group at https://groups.google.com/group/maxent.
For more options, visit https://groups.google.com/d/optout.

[Yapeng Chang]

Hi Jamie,

Can you share a guide to the use of these two functions and your code, I am a R beginner.

I'm having a big problem with environmental variables screening. I really hope to get your help.

Thank you.

[Jamie M. Kass]

Yapeng,

Glad to help, but please be specific about the problems you're having. Maybe post the code you're having trouble with. 

Jamie

--
You received this message because you are subscribed to a topic in the Google Groups "Maxent" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/maxent/4v5DyNcsZl8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to maxent+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/maxent/c15eb9e8-24b8-4183-8974-8fa1cab94556%40googlegroups.com.

[Christopher Ilori]

I guess you want to test for multicolinearity (reading your email subject).

This may help:

http://www.sthda.com/english/articles/39-regression-model-diagnostics/160-multicollinearity-essentials-and-vif-in-r/  

--

You received this message because you are subscribed to the Google Groups "Maxent" group.

To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.