Sensitivity, Tolerance, and Understanding Effect Size

[KVininska]

I have run an NPMR with ~40 env. variables. For predictors, proportion data was arcsine transformed; other quantitative data were log transformed; nominal (ie psuedo-continuous) and categorical data were left alone. No treatment of my species responses (density/count data or biomass data). I get decent xR2 values (0.3-0.7) for the best models. A few questions:

1. Some models differ in the variables chosen when I run NPMR with and without transformation. Transformation in most cases doesn't seem to improve xR2 much, if any. What criticism is there to using the analysis with untransformed data. Again, I haven't tried anything with my response data yet.

2. Now, I'm trying to evaluate effect size. I understand that tolerance is inverse to the "importance" of a variable. What is a large or small tolerance? For the transformed data NPMR runs, tolerances are all <1. For the untransformed data NPMR runs, I get 0.2-40.

3. Sensitivity analysis of selected best models. The largest Sensitivity 1 I'm getting is about 0.1. Most are are the order of 0.001 or 0.01. How do I marry this information with tolerance and my xR2 values? Is the interpretation that a model with "high" xR2 that the response is strongly predicted by the suite of variables, but that the effect of change in those variables is small? How does the tolerance indicate "importance"? I thought this was the smoothing parameter. Confused on this point.

Thanks for any guidance you can provide.

[Bruce McCune]

1. If transformation of your predictors doesn't help their explanatory power, I'd recommend keeping the original scales for interpretability. Why sacrifice that if it doesn't reveal relationships any more clearly?

2.Tolerance is inverse to the importance of a variable only with local mean models. Also, you need to consider that tolerances are reported in terms of the original scale of the variable, so that a predictor with a range of 1000 would probably have a much larger apparent tolerance than a predictor with a range of 0.1.  They can be made equivalent by dividing the tolerance by the range of the variable. For quantitative predictors, sensitivity analysis is universally useful (regardless of model type) for evaluating the explanatory importance of predictors. Sensitivity analysis is, however, questionable for categorical variables. See the Sensitivity Analysis topic in the built-in help for more info.

As for the question, what is large or small? -- there is no definite answer to that -- it depends on the analyst and the situation. It is similar to asking, What is a large or small correlation coefficient?

3. For a local mean model, sensitivities, tolerances, and xR2's will all tend to covary. That is, a model with crummy xR2 will likely have low sensitivities and broad tolerances for the predictors. For a local linear model, sensitivities and xR2's will tend to covary, but tolerances are less informative (again, see the Sensitivity Analysis topic).

You have some predictors that generate 100 times as much response as other predictors, so you have a clear relative scale for what is big and what is small for this.Your predictors with low sensitivities aren't contributing much to the model, so for parsimony you might consider a model without them. Removing them shouldn't have much impact on the xR2.

Hope this helps.

Bruce McCune

--
You received this message because you are subscribed to the Google Groups "HyperNiche and NPMR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hyperniche+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.