collinearity vs. AIC for a better model

[alqamy]

I have a unique situation that I am puzzelled by and I need some help. My understanding is that removing predictor variables that have collenearity among them would yield into a beter performing model. I have a situation where I am using about 27 variables (19 bioclim and 7 derived ones). I investigated colinearity among them using the VIF in linear regression in step wise approach where I was excluding the variable with the highest VIF and repeating the run until I have reached a set of variables with VIF under 10. No problem so far. The problem arises when I test the performance of a variable-refined model with only these shortlisted variables against a global model incorporating the full list using AIC. The global model have much better AIC score (AIC=406) compared to the variable-refined model (AIC=600). My question is " Should I believe in AIC and adopt the global model despite the proved collenearity in the variables or should I adopt the variable-refined model with the collenearity being confirmedly removed and ignore the AIC results ?

Hope that somebody have an answer.

Regards

Husam El Alqamy, B.Sc., M.Phil.
Sr. Biodiversity GIS Analyst ,
Environmental Information Sector, EIS
Environmental Agency Abu Dhabi,UAE 
Antelope Specialist Group, ASG - IUCN

[Dan.L....@gmail.com]

I'm afraid there's not much guidance available on this issue.  The simulation work Stephanie Seifert and I did on this really didn't address it - we only looked at AIC's performance across models that had the same set of variables available to them during model construction.  You can always apply a hybrid approach where you reduce multicolinearity of variables and then use AIC to set beta on that reduced set of variables.

[Husam El Alqamy]

Thank you Dan for your reply, Can you please explain this sentence "we only looked at AIC's performance across models that had the same set of variables available to them during model construction" . If the model has the same set of variables then it is the same model. Do you mean a biger set of model that are being selected from to build different models ( like ommiting one or two variables only at each run)?

Thanks again

Husam El Alqamy, B.Sc., M.Phil.
Sr. Biodiversity GIS Analyst ,
Environmental Information Sector, EIS
Environmental Agency Abu Dhabi,UAE
Antelope Specialist Group, ASG - IUCN

On Wed, Oct 17, 2012 at 1:43 PM, Dan.L....@gmail.com <Dan.L....@gmail.com> wrote:

I'm afraid there's not much guidance available on this issue.  The simulation work Stephanie Seifert and I did on this really didn't address it - we only looked at AIC's performance across models that had the same set of variables available to them during model construction.  You can always apply a hybrid approach where you reduce multicolinearity of variables and then use AIC to set beta on that reduced set of variables.

--
You received this message because you are subscribed to the Google Groups "Maxent" group.

To view this discussion on the web visit https://groups.google.com/d/msg/maxent/-/A-kuJN1Oc70J.

To post to this group, send email to max...@googlegroups.com.
To unsubscribe from this group, send email to maxent+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/maxent?hl=en.

--

[Colin Driscoll]

Different models can be created using the same variables with different samples sets. My understanding is that collinearity doesn't so much impact on the spatial model but can confound the interpretation of the target species relationship with the correlated variables. The better AIC for your larger set of variables suggests to me that there are possible interactions that you have removed when reducing the number.

 

Colin

[Hackett]

I thought what Dan meant was choosing from the models with the same set of variables but different model settings, see their article in 2011 (Warren, D. L. and S. N. Seifert (2011). "Ecological niche modeling in Maxent: the importance of model complexity and the performance of model selection criteria." ECOLOGICAL APPLICATIONS 21 (2): 335-342.)

Hackett

[Megan S]

Hi all,

I have some similar questions. I am running the AICc calculation on models with different sets of variables, chosen by different methods of dealing with collinearity, and have some questions about it.

1. If I run 5 replicates, is it okay to use the lambdas file from one replicate and the ASCII file for the average of all replicates?

2. Does the AICc calculation incorporate any information from your projection (e.g. environmental variables for the last glacial maximum)? I calculated AICc values for 2 different models where all I changed was the climate data in the projection (CCSM vs. MIROC) and got different AICs. 

3. I also get different AIC values for different replicates of the same model. Does anyone know of a good way to deal with this? Maybe average the AICc values for all of the replicates?

4. Is there an accepted standard for how large the difference between AICc values needs to be to determine if one model is better than another?

Any insight on any of these issues is greatly appreciated!

Thanks,

Megan