branch-site models, setting one branch vs. several branches in the foreground

[shpar]

Hello,

My input tree consists of reptiles and mammals. I would like to set reptiles in the foreground and determine sites undergoing positive selection in just reptiles. What would be the difference between setting just the branch leading to reptiles in the foreground (and not the branches that follow) vs setting all reptile branches in the foreground? Additionally, if I only set just the branch leading to the reptiles in the foreground, what is the program using to determine omega values since there is no DNA sequence representative there?

I have performed both analyses for several genes and so far results show significance when setting just the branche leading to reptiles in the foreground, but not when setting all reptile branches in the foreground. 

Thanks

[Ziheng]

I suggest that you read the papers on the branch model and branch site model for the motivations for the tests.

If you set one branch ancestral to reptiles as the foreground, the assumption would be that there was adaptation in this gene in the common ancestor, perhaps due to acquisition of new function, but afterwards the gene has been conserved evolving under purifying selection. If you set the whole clade as the foreground, the hypothesis will then be that the gene has been under continual pressure to change or diversify under the same pressure on all branches within the whole reptilian clade. this may be the case if the gene is involved in defense or immunity.
I wonder whether you can tell whether that branch is ancestral to reptiles or ancestral to mammals.
One does not observe the ancestral gene sequences, but the modern sequences provide a lot of information about the ancestors. You can read a textbook on molecular evolution.
Ziheng