What are the differences among columns, patterns and informative sites?

[Hongru Wang]

Hi,

I am running iqtree using SNP data on 5 individuals on a region which has 197 SNPs, the software spit out the message "Alignment has 5 sequences with 197 columns and 56 patterns (26 informative sites)". This makes me quite nervous because my understanding is that the alignment has 5 sequence with 197 columns and 197 patterns, and 197 informative sites.

Best,

Hongru.

[Olga Chernomor]

Hi Hongru,

columns are the columns/sites in the alignment and the number of columns is the length of the alignment. In the alignment there might be the same columns. Different columns are called patterns. While (parsimony) informative sites are patterns that have at least two different characters (nucleotides or amino acids) and each character should occur in at least two species. Essentially, informative sites have information for the grouping of species. These patterns are mainly important in the context of parsimony, where no evolutionary model is used. In maximum likelihood inference all patterns containing different characters are important for the estimation of tree topology and branch lengths, while constant/invariant sites (containing only the same character: only A's or only G's etc) are important for the correct estimation of the branch lengths. Therefore, should not be excluded from the alignment.

Example:

                          123456789

species_1   AACGTACGT

species_2   AACGATCGT

species_3   AACCGTCCT

species_4   AACCTACCT

sites/columns 1 and 2 are identical and contain only A's - invariant site pattern, uninformative

sites/columns 3 and 7 are identical and contain only C's - invariant site pattern, uninformative

sites/columns 4 and 8 are identical and contain 2 G'c and 2 C's - informative site pattern

sites/columns 5, 6 and 9 occur only once, each site is a different pattern

5th - uninformative

6th - informative

9th - invariant, uninformative 

Summing up, the alignment has 9 columns, 

6 patterns 

A C G T A T

A C G A T T

A C C G T T

A C C T A T

(2 informative)

G A 

G T 

C T 

C A 

Cheers,

Olga

[Edward Wallace]

Hi Olga,

This explanation is extremely helpful! A student in my group was just asking me about patterns mean and this is the best explanation I found. Patterns are not really explained elsewhere in the documentation. Could it be added to the main IQ-TREE documentation or the FAQ?

Thanks,

Edward

[Olga Chernomor]

Hi Edward,

I’m glad to hear that you find it useful. I’ll suggest Minh adding this info to IQ-TREE doc/website.

Cheers,

Olga

-- 
You received this message because you are subscribed to the Google Groups "IQ-TREE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iqtree+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iqtree/05083e5e-3b15-4fef-9cbe-3e6b669e4946n%40googlegroups.com.

[Minh Bui]

Thank you Olga! 

I added your answer into the FAQ: http://www.iqtree.org/doc/Frequently-Asked-Questions

Minh

To view this discussion on the web visit https://groups.google.com/d/msgid/iqtree/5548C2C9-152F-427C-BC53-EB99AF957287%40gmail.com.

[e.or...@gmail.com]

Dear Olga,

I am writing an alignment parser to extract these statistics and I have some doubts:

First: I wonder if columns with ambiguities or gaps are treated differently in IQ-TREE?

Second: Do you think that reporting both number of informative sites and number of informative patterns would be useful?

Third: IQ-TREE also report singletons, are these columns (or patterns) that are only different for a single taxon?

Thanks,

Edgardo

[Olga Chernomor]

Dear Edgardo,

First: I wonder if columns with ambiguities or gaps are treated differently in IQ-TREE?

In principle, both “N” and “-“ do not provide any information, so would be treated in the same way. BUT I do not know this for sure. Maybe Minh can comment on this.

Second: Do you think that reporting both number of informative sites and number of informative patterns would be useful?

I think both statistics could be important, but this, of course, depends on the questions one is interested in.

Third: IQ-TREE also report singletons, are these columns (or patterns) that are only different for a single taxon?

Yes. 

Best wishes,

Olga

To view this discussion on the web visit https://groups.google.com/d/msgid/iqtree/6a8a6bec-b4b2-4819-8df4-39ec495c12e3n%40googlegroups.com.

[Heiko Schmidt]

Dear Edgardo,

> On 06.07.2021, at 09:50, Olga Chernomor <o.che...@gmail.com> wrote:
>
> Dear Edgardo,
>
>> First: I wonder if columns with ambiguities or gaps are treated differently in IQ-TREE?
> In principle, both “N” and “-“ do not provide any information, so would be treated in the same way. BUT I do not know this for sure. Maybe Minh can comment on this.

Olga is absolutely right.

In all phylogenetic reconstruction software I know, gaps are treated the same as ambiguous character, that means the same as N (in DNA) or X (in proteins).

This is repeatedly discussed in this forum ;)

Best wishes,
Heiko

> To view this discussion on the web visit https://groups.google.com/d/msgid/iqtree/79062C17-FB45-4D24-BD32-251DD81548FF%40gmail.com.

-----------------------------------------------------------------------------
Heiko Schmidt
Center for Integrative Bioinformatics Vienna (CIBIV)
University of Vienna / Max Perutz Labs
Campus Vienna Biocenter 5 (VBC5)
A-1030 Vienna, Austria
-----------------------------------------------------------------------------
Email: heiko.schmidt(AT)univie.ac.at
Tel. : ++43 +1 / 4277 - 74321
http://www.cibiv.at/
-----------------------------------------------------------------------------

[Edgardo Ortiz Valencia]

Thanks!, my question refers specifically to the reporting of informative sites/patterns, and how the gaps or ambiguities are treated when reporting sites, for example for this alignment:

                          123456789

species_1   AACGTACGT

species_2   ANCGATCGT

species_3   AACCGT-CT

species_4   AACCTACCT

Would this alignment have the same number of uninformative patterns as the previous example given by Olga?

Thanks

Edgardo

On 6. Jul 2021, at 10:30, Minh Bui <m....@anu.edu.au> wrote:

Hi Both,

On 6 Jul 2021, at 5:50 pm, Olga Chernomor <o.che...@gmail.com> wrote:

Dear Edgardo,

First: I wonder if columns with ambiguities or gaps are treated differently in IQ-TREE?

In principle, both “N” and “-“ do not provide any information, so would be treated in the same way. BUT I do not know this for sure. Maybe Minh can comment on this.

Does the information in the FAQ help: http://www.iqtree.org/doc/Frequently-Asked-Questions#how-does-iq-tree-treat-gapmissingambiguous-characters?

If your question is about: how does IQ-TREE determine the type of an alignment site if that site has gaps/ambiguous charaters? then the answer is not short. But please confirm, before I try to write the answer.

Second: Do you think that reporting both number of informative sites and number of informative patterns would be useful?

I think both statistics could be important, but this, of course, depends on the questions one is interested in.

I could be wrong, but I didn’t find the number of informative patterns is useful in maximum likelihood… We never use that statistic in IQ-TREE.

Third: IQ-TREE also report singletons, are these columns (or patterns) that are only different for a single taxon?

Yes. 

To be precise: any site that is not-variant and not-informative is called singleton. So it can show a single (different) character for one or more taxa. E.g.:

species_1   A

species_2   C

species_3   A

species_4   T

is a singleton site.

Cheers,

Minh

[Minh Bui]

Hi Edgardo,

ok, now this question is clearer.

For each site in the alignment, IQ-TREE will ignore all species with gaps/ambiguities, then determine its informativeness on the (reduced) set of species. This is repeated separately for every site, hence the reduced set of species might be different for different site. 

For your provided example, site 1 is determined from all 4 species. Site 2 is determined from three species 1, 3 and 4 (because species 2 shows an ’N’)…. site 7 is determined from 3 species 1, 2 and 4 (as species 3 shows a ‘-‘).

Note that due to this logic, site 2 and 7 are both constant, even though they have N and -. 

Cheers

Minh

To view this discussion on the web visit https://groups.google.com/d/msgid/iqtree/BD4C2483-F722-4248-9F95-4D92AD1E1251%40gmail.com.

[Heiko Schmidt]

Dear Edgardo, (dear Minh - please see below)

To your question whether the number of uninformative sites is the same in your and Olga’s example. No.

In Olga’s example there were 6 patterns of which 2 are informative and 4 uninformative.

In your example several of the uninformative sites are not identical anymore. Hence, there are 8 patterns of which still 2 are informative, but now 6 are uninformative.

However, I also have a question to Minh.

In the following example:

1 2 3 4 5
species_1 A A C C C
species_2 A N - N Y
species_3 N A C C C
species_4 A A C C C

I assume, sites 1 and 2 are regarded different (uninformative) patterns.

In the second block, sites 3 and 4 are identical patterns, because the indel/gap (‘-‘) is treated like an “N”, but I guess that site 5 is regarded a different pattern because it only allows pyrimidines (‘Y’) in species_2.

@Minh: is that correct?

Best wishes,
Heiko

> To view this discussion on the web visit https://groups.google.com/d/msgid/iqtree/319FAFBA-9B9A-470D-937C-08865DBBA3B6%40univie.ac.at.

-----------------------------------------------------------------------------
Heiko Schmidt
Center for Integrative Bioinformatics Vienna (CIBIV)
University of Vienna / Max Perutz Labs

http://www.cibiv.at/
-----------------------------------------------------------------------------

[e.or...@gmail.com]

Dear Heiko,

Thanks for the clarifications, I wrote my parser using your rules (including the assumptions in your last question) and my numbers match with those reported by IQ-TREE. I thought IQ-TREE reported the number of informative patterns (because of how the report is worded and Olga's explanation) but it actually reports number of informative sites. I ran the same alignment with IQ-TREE and my script and I got:

IQ-TREE:

Alignment has 176 sequences with 16126 columns, 10649 distinct patterns
6322 parsimony-informative, 3196 singleton sites, 6608 constant sites

My Script:

Sequences:176

Sites:
{'total': 16126, 'informative': 6322, 'constant': 6608, 'singleton': 3196}

Patterns:
{'total': 10649, 'informative': 6304, 'constant': 1397, 'singleton': 2948}

Thanks to all for the help.

Edgardo

[Minh Bui]

Hi Heiko,

To your question, yes that’s correct, 5th site is a different pattern. 3rd and 4th sites are the same pattern.

However, the definition of invariant sites is more complex in the presence of abiguous characters like this. Y = {C or T}, so site 5 is regarded as an invariant site. But assuming a new site:

species_1 C
species_2 R
species_3 C
species_4 C

This site is NOT invariant, because R = {A or G}, which is not C.

More generally, for each site IQ-TREE will look at the intersection of the state sets across the species. A site is called invariant if this intersection is non-empty, and variant if the intersection is empty. So this definition also works with - and N (which is regarded as {A, C, G, or T}).

Cheers
Minh

> To view this discussion on the web visit https://groups.google.com/d/msgid/iqtree/B59625D9-901D-4870-BC56-6A6E263FD2F5%40univie.ac.at.