Help interpreting results of physignal and physignal.z

[Julio Torres]

Hi folks,

I'm assessing the phylogenetic signal in a genus compose by 6 species using as dataset landmarks taken from skull and mandible in 2D.  I already read the papers that explain the theory behind these two function that assess the signal (Adams 2014, Collyer,Baken& Adams2022.). However, I still don't understand if I am interpreting correctly my results. 

I would really appreciate if some of you can help me with the interpretation or suggest me some reading .

Best regards

Julio

Below my results and commented doubts. I'm using the last version of geomorph 4.0.5

# physignal outputs: physignal(A= data, phy= tree, iter=999);

skull dorsal: K = 0.7004 ; P = 0.39085
skull Lateral: K = 0.7883; P = 0.2
skull Ventral: K= 1.0677; P = 0.026

Mandible: K = 1.2769: P= 0.016

Doubt 1. I interpreted that all my datasets have "strong" phylogenetic signal. But only are significant in ventral and mandible. However, I don't undersand why  K values are above 1 in ventral and mandible datasets.

# physignal.z outputs: physignal.z(A = data, phy = tree,
                              lambda = "front", iter=999)

skull Dorsal: Z= 0.22 ; P =0.4925
skull Lateral:

      The scaling parameter, lambda, was optimized at 0.

       This means that the log-likehood was invariant across permutations and there is no  phylogenetic signal in the data.
skull Ventral: Z= 0.1549; P = 0.4885
Mandibula: Z = -0.0857; P= 0.5045

Doubt 2. Why I don't have Z value in lateral dataset?.

Doubt 3. I chose "front" as an option optimization method. However, even after reading the help documentation I don't understand which option is more appropriate for my small tree (6 species) comparison. 

[Mike Collyer]

Dear Julio,

… phylogenetic signal in a genus compose by 6 species using as dataset landmarks taken from skull and mandible in 2D. ..

Any results should be interpreted cautiously with only 6 observations.  There are only so many random permutations that are possible with such a small data set.

Below my results and commented doubts. I'm using the last version of geomorph 4.0.5

# physignal outputs: physignal(A= data, phy= tree, iter=999);

skull dorsal: K = 0.7004 ; P = 0.39085
skull Lateral: K = 0.7883; P = 0.2
skull Ventral: K= 1.0677; P = 0.026

Mandible: K = 1.2769: P= 0.016

Doubt 1. I interpreted that all my datasets have "strong" phylogenetic signal. But only are significant in ventral and mandible. However, I don't undersand why  K values are above 1 in ventral and mandible datasets.

I would not interpret “strong” based on K.  Effect sizes are more reliable, as we illustrated in the paper you cited.

# physignal.z outputs: physignal.z(A = data, phy = tree,
                              lambda = "front", iter=999)

skull Dorsal: Z= 0.22 ; P =0.4925

For example, this is a small effect size.

skull Lateral:

      The scaling parameter, lambda, was optimized at 0.

       This means that the log-likehood was invariant across permutations and there is no  phylogenetic signal in the data.
skull Ventral: Z= 0.1549; P = 0.4885
Mandibula: Z = -0.0857; P= 0.5045

Doubt 2. Why I don't have Z value in lateral dataset?.

The comment you received explains why.  If lambda is optimized at 0 (star phylogeny), then you have no phylogenetic signal, and the log-likelihood would be constant in every random permutation.  The Z-score is NaN because the standard deviation of the null distribution is 0.  There is no effect size to report if lambda = 0.

Doubt 3. I chose "front" as an option optimization method. However, even after reading the help documentation I don't understand which option is more appropriate for my small tree (6 species) comparison. 

This is maybe a good choice to use if you have more variables than observations, which you probably have.  It concentrates the phylogenetic signal in the fewest dimensions (the first few, i.e., those in front of the others) 

There really is no good advice because research in this area is lacking.  Optimizing a parameter like lambda for multivariate data, combined with using RRPP to find effect sizes is not straightforward.  Until more research is performed, we make these options available.  I would probably use “burn” in most cases that would allow it, because it maximizes the effect size rather than log-likelihood.  Thus it finds the lambda where effect size is greatest.  Although effect size and log-likelihood should be correlated, I would prefer to not assume the effect size is maximized where the log-likelihood is maximized, but find the place on the effect-size landscape that it is obviously valid.

However, I am also not so bold as to suggest that maximizing effect size based on RRPP should supplant maximizing likelihood, a technique that has been a basis of inference in many disciplines.  Again, more research is needed.  It does not hurt to try each method and only worry if you get contrasting results, which with 6 observations, you might.

Best,

Mike

--
You received this message because you are subscribed to the Google Groups "geomorph R package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geomorph-r-pack...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/geomorph-r-package/cda6be9d-cded-4657-8161-34bb8f4624a3n%40googlegroups.com.

[Julio Torres]

Dear Mike

Thanks for your explanation!

Best

Julio

[ani.hri...@gmail.com]

Hi Mike & Geomorph community,

I wanted to piggy-back onto Julio's post in light of Mike's response. I'm also struggling to determine which optimization method is most appropriate for my data and with output interpretation. I have read the 2021 paper and am still experiencing some difficulty wrapping my head around it. For context, my dataset includes 18 mustelid species, and the physignal.z output for all the different optimization methods looks like this: 

My questions are as follows: 

(1) Why would the front, mean, and all optimization methods provide the same results given that they interpret the data in different ways? 

(2) If "burn" optimization should be maximizing effect size, why are all the effect sizes smaller than when using front/mean/all optimization? The story appears to be the same, but the numbers are quite different. 

(3) Looking at average body weight, if lambda is nearly 0 and Kmult=0.16, this suggests no/low phylogenetic signal for this trait and that it largely did not evolve by Brownian motion. What does a significant effect size mean in the context of low phylogenetic signal? Also, Z < 1.96 and still significant - is this acceptable or cause for concern? 

(4) In the 2021 paper, I noticed you performed a comparison of Z values for different traits to determine relative phy signal contributions. What function would be appropriate to run this comparison? Does anyone have sample code available, by chance? 

(5) Would it be deemed appropriate to discuss measures of phylogenetic signal (lambda or Kmult) even if Z is not significant?

Thank you so much; really looking forward to hearing from you!

All the best,

Ani

[Mike Collyer]

Ani,

Quick clarifying question. How many variables are there for relative brain volume and relative brain weight?  These seem like single variables but your analysis suggests these are multivariate?    It would help to know to potentially understand result variability. 

Thanks!

Mike

Sent from my iPhone

On Sep 25, 2023, at 2:49 PM, ani.hri...@gmail.com <ani.hri...@gmail.com> wrote:

Hi Mike & Geomorph community,

I wanted to piggy-back onto Julio's post in light of Mike's response. I'm also struggling to determine which optimization method is most appropriate for my data and with output interpretation. I have read the 2021 paper and am still experiencing some difficulty wrapping my head around it. For context, my dataset includes 18 mustelid species, and the physignal.z output for all the different optimization methods looks like this: 

<Physignal.z Data output.PNG>

My questions are as follows: 

(1) Why would the front, mean, and all optimization methods provide the same results given that they interpret the data in different ways? 

(2) If "burn" optimization should be maximizing effect size, why are all the effect sizes smaller than when using front/mean/all optimization? The story appears to be the same, but the numbers are quite different. 

(3) Looking at average body weight, if lambda is nearly 0 and Kmult=0.16, this suggests no/low phylogenetic signal for this trait and that it largely did not evolve by Brownian motion. What does a significant effect size mean in the context of low phylogenetic signal? Also, Z < 1.96 and still significant - is this acceptable or cause for concern? 

(4) In the 2021 paper, I noticed you performed a comparison of Z values for different traits to determine relative phy signal contributions. What function would be appropriate to run this comparison? Does anyone have sample code available, by chance? 

(5) Would it be deemed appropriate to discuss measures of phylogenetic signal (lambda or Kmult) even if Z is not significant?

Thank you so much; really looking forward to hearing from you!

All the best,

Ani

On Friday, April 28, 2023 at 5:44:04 AM UTC-4 juliomy...@gmail.com wrote:

Dear Mike

Thanks for your explanation!

Best

Julio

--

You received this message because you are subscribed to the Google Groups "geomorph R package" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geomorph-r-pack...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/geomorph-r-package/68579ac2-a6d3-4f98-9091-4238fb31c4c7n%40googlegroups.com.

<Physignal.z Data output.PNG>

[ani.hri...@gmail.com]

Hi Mike -- they are single variables. Thank you for pointing that out; would this analysis still work for them since it is built for multivariate data?

[Mike Collyer]

Actually, on second thought, I think I can see that most of your variables are indeed univariate variables and this should be easy to explain.  There is a reason that you have consistent results for front, mean, and all in these cases.  The front end of one variable is the first and only variable; the mean lambda for one variable is the lambda value you found for that variable; using all variables means using the one variable.  These results have to be the same.

Regarding the burn-in option, it should not be used with univariate data, and thanks to your inadvertent discovery, we will have to add a trap to stop it.  It might also be worth adding a machine learning style algorithm to not seek a solution if Z scores during the burn-in phase are small.  What the burn-in method does is use a smaller number of permutations (I think about 10% of the total requested) to find Z scores in intervals, and then fits a spline to find where Z — not lambda! — is maximized.  Once found, then the full set of permutations is run for a lambda value that corresponds to the preliminary Z maximization, obtaining a Z from that full distribution.

There are two ways the burn-in method of optimization can go haywire.  One is if there is a flat surface for which to fit a spline.  In these cases, Z would probably be small, wherever it is optimized, and whether it was optimized appropriately, since finding a maximum in a surface that is flat is fraught with potential error.  Second is if one is trying to optimize a multivariate lambda (assuming two or more variables) with a single variable, in which case the burn-in method makes no sense.  This is like asking the analysis to ignore the larger number of requested permutations for a straightforward analysis and have it make a biased judgement first based on a distribution with 10% of the permutations.  (A good future argument will be to have the number of burn-in permutations as an option, but as stated in the help file, optimization strategies are still in need of development.)

In practical terms, you should XX-out all burn-in results in your table except for shape.  It is illogical to perform that optimization, a lesson we will have to fix on the next software version to prevent a silly trial.  

Finally, regarding your Avg. Body Weight result, it is good to remember what is happening in the analysis, because although statistical results like this seem odd, they can make sense.  The lambda is practically 0.  The significant P-value indicates that lambda = 0.0000007 — basically a star phylogeny with the tiniest bit of structure near the root of the tree, so small as to be easily ignored — has a result to suggest the trace of the generalized least squares residual variance you find with the covariance matrix used from this result is smaller than expected by chance.  You *could* call that phylogenetic signal but you would be foolish to suggest it resembles what you would expect with Brownian Motion, which tends to be what we use as a standard to think about phylogenetic signal, meaning the K value is rather truthfully small here.  I think we were pretty clear in the paper that one should call it phylogenetic signal when both the effect size is large and lambda suggests the data look like they were sampled from a BM process.  The latter thing is not at all true and former thing is subjectively also not true, even if the P-value is borderline true.

Generally if P = 0.05 one would expect Z = 1.96, so you might also have an issue with strangely distributed residuals in your GLS fits.

I think that covers most bases — hope it made sense.

Best,

Mike

[ani.hri...@gmail.com]

Mike,

Thank you very much for taking the time to draft up that response; I'm really grateful for your explanation and clearing up my questions!

All the best,

Ani