difference in jacknife and percent contribution
1,790 views
Skip to first unread message
Lourens
Oct 24, 2011, 8:08:23 AM10/24/11
to Maxent
Hi All,
I have produced my final maxent model and i got a confusing (?)
answer. With jacknife test for variable contribution I got a different
‘important’ variable to the percent contribution one. As I understand
this can be due to correlated variables? Percent contribution is
calculated as the model is generated, while jacknife generates models
with different sets of variables. So, you could have a variable with a
high contribution, but during jacknife it would not show to be
important because there is another correlated variable that can
replace it. But my question is that I checked all variables for
correlation (ENMtools) and removed the highly correlated ones (>.75),
so what do I make of this? Which one is a better test for variable
importance % contribution or jacknife? When I use AIC (ENMtools) to
test models run with each variable alone (like the jacknife) I get the
same important variables as the jacknife test. Any help on how to
report and interpret this would be appreciated, or if I have to do
alternative analysis
Thanks in advance
Lourens
I have produced my final maxent model and i got a confusing (?)
answer. With jacknife test for variable contribution I got a different
‘important’ variable to the percent contribution one. As I understand
this can be due to correlated variables? Percent contribution is
calculated as the model is generated, while jacknife generates models
with different sets of variables. So, you could have a variable with a
high contribution, but during jacknife it would not show to be
important because there is another correlated variable that can
replace it. But my question is that I checked all variables for
correlation (ENMtools) and removed the highly correlated ones (>.75),
so what do I make of this? Which one is a better test for variable
importance % contribution or jacknife? When I use AIC (ENMtools) to
test models run with each variable alone (like the jacknife) I get the
same important variables as the jacknife test. Any help on how to
report and interpret this would be appreciated, or if I have to do
alternative analysis
Thanks in advance
Lourens
Colin Driscoll
Oct 25, 2011, 11:04:33 PM10/25/11
to max...@googlegroups.com
Even though you remove the highly correlated variables there can still be interaction between the ones that you use. The jackknife test shows you which variables have the most useful information independent of the others while the heuristic test does not make that distinction. So a variable can have a higher ranking in the heuristic test than in the jackknife test because the model has chosen to assign some of the correlated effect with another variable to that particular variable. Best to consider the biological role of variables for your subject species and not read too much into responses to the variables.
--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To post to this group, send email to max...@googlegroups.com.
To unsubscribe from this group, send email to maxent+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/maxent?hl=en.
David Le Maitre
Oct 26, 2011, 2:24:25 AM10/26/11
to max...@googlegroups.com
Hi Lourens
I agree with Colin, your need to properly understand the relationships between the variables you are using and the ecological/biological factors that are range determinants for the species you are interested in. Then you need to examine their relative importance in the final model and see if that subset and ranking makes sense. You can also experiment with the choices of the features - excluding product features will limit Maxent's ability to take correlations between variables into account. You should also look at the forms of the functional responses that are being fitted to see if those make sense. For example, a bi-/multi-modal response to temperature does not make too much sense. Maxent outputs are driven by the data you put in and a lot of thought should go into the data you put in.
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.
Lourens
Oct 26, 2011, 9:12:59 AM10/26/11
to Maxent
David, Collin,
Thanks for both your valuable replies. We have narrowed the variables
down to 13 based on their importance for the particular species. The
results from both the Jacknife and heuristic test make biological
sense and is what we exspected based on habitat studies. The same top
4 variables were important in both tests, although their order
differs. So the ranking makes sense. We had a look into different
features but the overall results stay pretty much the same. My
question is does this 'correlation' between the 'unknown' variables
matter in the end product? I mean can i report the results as is, or
is the difference between jacknife and heuristic a reason for consern?
Or should i just report one of the two?
Thanks for you time and help
Lourens
On Oct 26, 8:24 am, "David Le Maitre" <DlMai...@csir.co.za> wrote:
> Hi Lourens
>
> I agree with Colin, your need to properly understand the relationships
> between the variables you are using and the ecological/biological
> factors that are range determinants for the species you are interested
> in. Then you need to examine their relative importance in the final
> model and see if that subset and ranking makes sense. You can also
> experiment with the choices of the features - excluding product features
> will limit Maxent's ability to take correlations between variables into
> account. You should also look at the forms of the functional responses
> that are being fitted to see if those make sense. For example, a
> bi-/multi-modal response to temperature does not make too much sense.
> Maxent outputs are driven by the data you put in and a lot of thought
> should go into the data you put in.
>
> David
>
> >>> Colin Driscoll <gisha...@gmail.com> 2011/10/26 05:04 AM >>>
> maxent+un...@googlegroups.com (
> mailto:maxent%2Bunsu...@googlegroups.com ).
> For more options, visit this group athttp://groups.google.com/group/maxent?hl=en.
Thanks for both your valuable replies. We have narrowed the variables
down to 13 based on their importance for the particular species. The
results from both the Jacknife and heuristic test make biological
sense and is what we exspected based on habitat studies. The same top
4 variables were important in both tests, although their order
differs. So the ranking makes sense. We had a look into different
features but the overall results stay pretty much the same. My
question is does this 'correlation' between the 'unknown' variables
matter in the end product? I mean can i report the results as is, or
is the difference between jacknife and heuristic a reason for consern?
Or should i just report one of the two?
Thanks for you time and help
Lourens
On Oct 26, 8:24 am, "David Le Maitre" <DlMai...@csir.co.za> wrote:
> Hi Lourens
>
> I agree with Colin, your need to properly understand the relationships
> between the variables you are using and the ecological/biological
> factors that are range determinants for the species you are interested
> in. Then you need to examine their relative importance in the final
> model and see if that subset and ranking makes sense. You can also
> experiment with the choices of the features - excluding product features
> will limit Maxent's ability to take correlations between variables into
> account. You should also look at the forms of the functional responses
> that are being fitted to see if those make sense. For example, a
> bi-/multi-modal response to temperature does not make too much sense.
> Maxent outputs are driven by the data you put in and a lot of thought
> should go into the data you put in.
>
> David
>
>
> Even though you remove the highly correlated variables there can still
> be interaction between the ones that you use. The jackknife test shows
> you which variables have the most useful information independent of the
> others while the heuristic test does not make that distinction. So a
> variable can have a higher ranking in the heuristic test than in the
> jackknife test because the model has chosen to assign some of the
> correlated effect with another variable to that particular variable.
> Best to consider the biological role of variables for your subject
> species and not read too much into responses to the variables.
>
> On Mon, Oct 24, 2011 at 11:08 PM, Lourens <lourens.leop...@gmail.com>
> Even though you remove the highly correlated variables there can still
> be interaction between the ones that you use. The jackknife test shows
> you which variables have the most useful information independent of the
> others while the heuristic test does not make that distinction. So a
> variable can have a higher ranking in the heuristic test than in the
> jackknife test because the model has chosen to assign some of the
> correlated effect with another variable to that particular variable.
> Best to consider the biological role of variables for your subject
> species and not read too much into responses to the variables.
>
> mailto:maxent%2Bunsu...@googlegroups.com ).
> For more options, visit this group athttp://groups.google.com/group/maxent?hl=en.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Maxent" group.
> To post to this group, send email to max...@googlegroups.com.
> To unsubscribe from this group, send email to
> maxent+un...@googlegroups.com.
> For more options, visit this group athttp://groups.google.com/group/maxent?hl=en.
> --
> You received this message because you are subscribed to the Google
> Groups "Maxent" group.
> To post to this group, send email to max...@googlegroups.com.
> To unsubscribe from this group, send email to
> maxent+un...@googlegroups.com.
>
> --
> This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found athttp://www.csir.co.za/disclaimer.html.
> --
> This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
David Le Maitre
Oct 26, 2011, 10:15:05 AM10/26/11
to Maxent
Hi Lourens and all
A good question. Good practice is that you should report all your results. The model that you focus on is the one that uses all the variables if that is the one you find acceptable - removing variables with correlations >0.75 will still leave some strong correlations.
The tests Maxent does for single variables with single variables are just that. You can report them but make it clear that its just a singe variables effects etc. The key thing is whether in each case it makes ecological sense.
The other test I would apply is whether the shifts are relatively large or small i.e. if the tope 4 all has a similar % contribution than the ranking is not such an issue. If the shifts are large you may should think hard about why.
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
Husam El Alqamy
Oct 26, 2011, 5:57:20 PM10/26/11
to max...@googlegroups.com
Hi Lourens and list
There is another approach to reduce the effect of correlation among predictor variables when using maxent modelling. You can calculate the Variance inflation factor VIF in linear regression in a stepwise approach, where calculation is done using all the predictor variables and then eliminating all the variables that produce VIF greater than 10 then running the calculation again with the new reduced list of predictors and the process continues until you end up with a list of predictable variable all with VIF less than 10.
I tried this approach and it yielded a model with considerable consensus between the jackknifef results and percentage of variable contribution to explaining variability. The method is also used in many master degree research in the ITC institute in Netherlands.
Regards
Husam El Alqamy, B.Sc, M.Phil.
Sr. Biodiversity GIS Analyst,
Enivronmental Information Sector
Environment Agency – Abu Dhabi
Lourens
Nov 2, 2011, 7:08:47 AM11/2/11
to Maxent
Hi All,
Husam, thank for the tip, i did run the VIF and got almost the same
result as before. My list of variables is still the same (VIF less <
10) as before, and the jacknife and heuristic is similar. I am going
wiht the adivice of David to present all results. My % contribution of
top four variables is very similar and they are also the the top four
in jacknife test, althought ranking differs. Will see what supervisors
think about this
Thanks for all the help
Lourens
On Oct 26, 11:57 pm, "Husam El Alqamy" <alq...@gmail.com> wrote:
> Hi Lourens and list
>
> There is another approach to reduce the effect of correlation among
> predictor variables when using maxent modelling. You can calculate the
> Variance inflation factor VIF in linear regression in a stepwise approach,
> where calculation is done using all the predictor variables and then
> eliminating all the variables that produce VIF greater than 10 then running
> the calculation again with the new reduced list of predictors and the
> process continues until you end up with a list of predictable variable all
> with VIF less than 10.
>
> I tried this approach and it yielded a model with considerable consensus
> between the jackknifef results and percentage of variable contribution to
> explaining variability. The method is also used in many master degree
> research in the ITC institute in Netherlands.
>
> Regards
>
> Husam El Alqamy, B.Sc, M.Phil.
>
> Sr. Biodiversity GIS Analyst,
>
> Enivronmental Information Sector
>
> Environment Agency - Abu Dhabi
> <http://www.mailscanner.info/> MailScanner,
Husam, thank for the tip, i did run the VIF and got almost the same
result as before. My list of variables is still the same (VIF less <
10) as before, and the jacknife and heuristic is similar. I am going
wiht the adivice of David to present all results. My % contribution of
top four variables is very similar and they are also the the top four
in jacknife test, althought ranking differs. Will see what supervisors
think about this
Thanks for all the help
Lourens
On Oct 26, 11:57 pm, "Husam El Alqamy" <alq...@gmail.com> wrote:
> Hi Lourens and list
>
> There is another approach to reduce the effect of correlation among
> predictor variables when using maxent modelling. You can calculate the
> Variance inflation factor VIF in linear regression in a stepwise approach,
> where calculation is done using all the predictor variables and then
> eliminating all the variables that produce VIF greater than 10 then running
> the calculation again with the new reduced list of predictors and the
> process continues until you end up with a list of predictable variable all
> with VIF less than 10.
>
> I tried this approach and it yielded a model with considerable consensus
> between the jackknifef results and percentage of variable contribution to
> explaining variability. The method is also used in many master degree
> research in the ITC institute in Netherlands.
>
> Regards
>
> Husam El Alqamy, B.Sc, M.Phil.
>
> Sr. Biodiversity GIS Analyst,
>
> Enivronmental Information Sector
>
>
> From: max...@googlegroups.com [mailto:max...@googlegroups.com] On Behalf Of
> David Le Maitre
> Sent: Wednesday, October 26, 2011 6:15 PM
> To: Maxent
> Subject: Re: difference in jacknife and percent contribution
>
> Hi Lourens and all
>
> A good question. Good practice is that you should report all your results.
> The model that you focus on is the one that uses all the variables if that
> is the one you find acceptable - removing variables with correlations >0.75
> will still leave some strong correlations.
>
> The tests Maxent does for single variables with single variables are just
> that. You can report them but make it clear that its just a singe variables
> effects etc. The key thing is whether in each case it makes ecological
> sense.
>
> The other test I would apply is whether the shifts are relatively large or
> small i.e. if the tope 4 all has a similar % contribution than the ranking
> is not such an issue. If the shifts are large you may should think hard
> about why.
>
> David
>
> >>> Lourens <lourens.leop...@gmail.com> 2011/10/26 03:12 PM >>>
> From: max...@googlegroups.com [mailto:max...@googlegroups.com] On Behalf Of
> David Le Maitre
> Sent: Wednesday, October 26, 2011 6:15 PM
> To: Maxent
> Subject: Re: difference in jacknife and percent contribution
>
> Hi Lourens and all
>
> A good question. Good practice is that you should report all your results.
> The model that you focus on is the one that uses all the variables if that
> is the one you find acceptable - removing variables with correlations >0.75
> will still leave some strong correlations.
>
> The tests Maxent does for single variables with single variables are just
> that. You can report them but make it clear that its just a singe variables
> effects etc. The key thing is whether in each case it makes ecological
> sense.
>
> The other test I would apply is whether the shifts are relatively large or
> small i.e. if the tope 4 all has a similar % contribution than the ranking
> is not such an issue. If the shifts are large you may should think hard
> about why.
>
> David
>
Reply all
Reply to author
Forward
0 new messages