Information Theory and Genomes
Devils Advocaat
I would like to run something past you guys and see if I am on the
right track.
So here goes.
If an apparently random message is made up of just four different
symbols (like the nucleotides of a DNA sequence), would the maximum
storage requirement for each symbol be two binary digits?
And if that is correct, would the maximum storage needed to contain
all 3 billion nucleotides of the human genome, be about 750 megabytes?
Garamond Lethe
> I have to admit my grasp of Information Theory is tenuous at best, but
> I would like to run something past you guys and see if I am on the
> right track.
>
> So here goes.
>
> If an apparently random message is made up of just four different
> symbols (like the nucleotides of a DNA sequence), would the maximum
> storage requirement for each symbol be two binary digits?
>
The maximum storage requirement for each symbol is independent of the
number of symbols, but yes, worst case would be two bits to describe
four values.
> And if that is correct, would the maximum storage needed to contain
> all 3 billion nucleotides of the human genome, be about 750 megabytes?
$ (4 * 3000000000 / 8)/(2^20)
[1] 1430.511
At 8 bits per byte and 2^20 bytes per megabyte, I think 1500 megabytes
is a bit closer (but I'm notoriously bad at arithmetic, so caveat
emptor).
John Harshman
> I have to admit my grasp of Information Theory is tenuous at best, but
> I would like to run something past you guys and see if I am on the
> right track.
>
> So here goes.
>
> If an apparently random message is made up of just four different
> the maximum storage requirement for each symbol be
> two binary digits?
Yes.
> And if that is correct, would the maximum storage needed to contain
> all 3 billion nucleotides of the human genome, be about 750
> megabytes?
Let's see: 3 billion times two divided by 8: yep.
> If my understanding is skewed I would be grateful for anyone who can
> put me back on track.
>
Robert Carnegie: Fnord: cc talk-origins@moderators.isc.org
amount of data required to hold the facts that DNA is split into
chromosomes, plus the separate mitochondrial DNA. Also, our DNA is in
the form of "words" of 3 letters, and I don't think all possible
"words" are used, or know whether a "word" and its DNA "mirror image"
are equivalent or distinct - you know that the DNA letters are in
pairs, one strand of the DNA double helix containing "opposites" of
the other strand. There's an explanation in Wikipedia that I don't
understand, or maybe it's a cruel practical joke...
Christopher Denney
> I have to admit my grasp of Information Theory is tenuous at best, but
> I would like to run something past you guys and see if I am on the
> right track.
>
> So here goes.
>
> If an apparently random message is made up of just four different
> symbols (for example the four nucleotides in a DNA sequence), would
> the maximum storage requirement for each symbol be
> two binary digits?
>
> all 3 billion nucleotides of the human genome, be about 750
> megabytes?
>
> put me back on track.
Well using today's computers it would be more accurate to say that you
could store 4 symbols, or two pairs, in a given eight bit byte, and
given 1024 bytes in a kbyte, and 1024 kbytes in a meg. then closer to
715 megs. Or double that for both sides of a sequence that is 3x10^9
"letters" long.
Robert Carnegie: Fnord: cc talk-origins@moderators.isc.org
Charles Brenner
> I have to admit my grasp of Information Theory is tenuous at best, but
> I would like to run something past you guys and see if I am on the
> right track.
>
> So here goes.
>
> If an apparently random message is made up of just four different
> symbols (like the nucleotides of a DNA sequence), would the maximum
> storage requirement for each symbol be two binary digits?
Yes.
> And if that is correct, would the maximum storage needed to contain
> all 3 billion nucleotides of the human genome, be about 750 megabytes?
3 billion base pairs, so technically 6 billion nucleotides. But
assuming the pairing rule, 750 megabytes is definitely sufficient.
Vend
Correct, considering "mega" as 10^6.
marks...@yahoo.com
You would need 2 bits to represent 4 possible codes 00,01,10,11
for each character you would get 4 elements of a dna sequence.
3,000,000,000 /4 = 750,000,000
Desertphile
<mank...@yahoo.co.uk> wrote:
> I have to admit my grasp of Information Theory is tenuous at best, but
> I would like to run something past you guys and see if I am on the
> right track.
>
> So here goes.
>
> If an apparently random message is made up of just four different
> symbols (for example the four nucleotides in a DNA sequence), would
> the maximum storage requirement for each symbol be
> two binary digits?
>
> And if that is correct, would the maximum storage needed to contain
> all 3 billion nucleotides of the human genome, be about 750
> megabytes?
Yes. One can record the basic human genome on one CD-RW, and then
record many millions more on a second CD-RW by noting the
differences from the complete (first CD) version.
That doesn't have to do with "information" however, as far as I
have read on the subject.
> If my understanding is skewed I would be grateful for anyone who can
> put me back on track.
In the past I have read a bloody hell of a lot of web pages on
what "information" is and is not in reference to a specified
genome: various methods have been proposed to provide a metric for
what "information" means but it still appears to be a muddled heap
o' crap to me (but then what the bloody fuck do I know?!).
Sequencing every base pair in the human genome would not be the
same as examining the "information" within it: it would provide
information about the human genome, which is a totally different
thing.
In the past I (and others) have claimed that genomes contain zero
information because (other than generically engineered genes) no
intelligence was applied to create those genomes. A hell of a lot
of people disagreed with me, so perhaps I am wrong.
Information occurs inside brains, not in DNA or RNA, and/or are
collections of products created by those brains. For information
to exist, there must be a brain (human or non-human) to understand
what is being observed.
--
http://desertphile.org
Desertphile's Desert Soliloquy. WARNING: view with plenty of water
"Why aren't resurrections from the dead noteworthy?" -- Jim Rutz
David Hare-Scott
> I have to admit my grasp of Information Theory is tenuous at best, but
> I would like to run something past you guys and see if I am on the
> right track.
>
I don't know much about that either but I programmed for some time.
> So here goes.
>
> If an apparently random message is made up of just four different
> symbols (for example the four nucleotides in a DNA sequence), would
> the maximum storage requirement for each symbol be
> two binary digits?
>
Two bits can encode up to 4 different values.
> And if that is correct, would the maximum storage needed to contain
> all 3 billion nucleotides of the human genome, be about 750
> megabytes?
>
Yes at most. Depending on the redundancy of the string it could be less
with an efficient non-loss encoding scheme.
David
r norman
>I have to admit my grasp of Information Theory is tenuous at best, but
>I would like to run something past you guys and see if I am on the
>right track.
>
>So here goes.
>
>If an apparently random message is made up of just four different
>symbols (for example the four nucleotides in a DNA sequence), would
>the maximum storage requirement for each symbol be
>two binary digits?
>
>And if that is correct, would the maximum storage needed to contain
>all 3 billion nucleotides of the human genome, be about 750
>megabytes?
>
>If my understanding is skewed I would be grateful for anyone who can
>put me back on track.
A binary computer file of 750 MBytes, coded properly, will contain the
complete record of the 3 billion nucleotide human genome. You can do
it in much less using compression techniques that take advantage of
redundancies or non-randomness in the sequence.
The "information content" of the genome is an entirely different
story. It seems that there is a lot of "redundancy" or "junk" or
"extraneous" material in those 3 billion nucleotides in the sense that
there are a very large number of alternative sequences that, as far as
we know, should produce exactly the same biological result.
Calculating an actual number for the information content is rather
problematical, to say the least. And there are those who argue that
the notion of information in the classical communication theory or
mathematical sense should not even be applied to the genome.
Ernest Major
<e5a34d5e-b2c5-4e84...@k19g2000yqc.googlegroups.com>,
"Robert Carnegie: Fnord: cc talk-o...@moderators.isc.org"
<rja.ca...@excite.com> writes
protein-coding DNA is only a small proportion of the total DNA. Also, it
turns out to be the case that, although the genetic code is redundant,
all "words" are used in the human genome. (It's possible that some
bacteria don't use all 64 codon values.)
--
alias Ernest Major
John Harshman
Codons only happen in protein-coding sequences, which make up around 1%
of the genome, maybe less. Within that 1%, all 64 possible words are
used, though some are rare. (The three stop codons, for example, can
only happen at the very end of a sequence.) A codon and its mirror image
mean something quite different, but only one strand of DNA generally
gets translated into protein, and only in one direction. (There are
exceptions, in which genes on opposite strands overlap, and can be read
in both directions. But rare.)
Christopher Denney
wrote:
Of course that's just writing a list out in memory or to disk.
In order to use the data you would have to hang it in a data structure
which could be as simple as some pointers to preceding and next
symbols in the string. The pointers are going to be on the order of
1-4 bytes each, so the data structure becomes very important in
determining how much space information takes up.
Mark Isaak
> Devils Advocaat wrote:
>> I have to admit my grasp of Information Theory is tenuous at best, but I
>> would like to run something past you guys and see if I am on the right
>> track.
>>
>> So here goes.
>>
>> If an apparently random message is made up of just four different
>> symbols (for example the four nucleotides in a DNA sequence), would the
>> maximum storage requirement for each symbol be two binary digits?
>
> Yes.
>
>> And if that is correct, would the maximum storage needed to contain all
>> 3 billion nucleotides of the human genome, be about 750 megabytes?
>
> Let's see: 3 billion times two divided by 8: yep.
Also, there is a great deal of repetition in the genome, so the file
storing it would be quite a bit smaller when zipped.
--
Mark Isaak eciton (at) earthlink (dot) net
"It is certain, from experience, that the smallest grain of natural
honesty and benevolence has more effect on men's conduct, than the most
pompous views suggested by theological theories and systems." - D. Hume
John Harshman
Which is how the genetic code is able to change in some organisms with
small genomes.
Kalkidas
9c6f-5e1...@j5g2000yqm.googlegroups.com:
Less than that with compression. Run-length encoding, etc...
Sapient Fridge
<18312390-8773-4c92...@j5g2000yqm.googlegroups.com>,
Devils Advocaat <mank...@yahoo.co.uk> writes
Sounds right to me - ignoring mitochondrial DNA, any kind of compression
and the fact that a megabyte is 1024 * 1024 bytes not 1,000,000.
Only one side of the DNA has to be encoded because the other side always
has the complimentary base pair.
Every person has 2 genomes though (23 *pairs* of chromosomes) so if you
wanted to encode all the genetic material in a single human then you
would need double that storage
--
sapient_...@spamsights.org ICQ #17887309 * Save the net *
Grok: http://spam.abuse.net http://www.cauce.org * nuke a spammer *
Find: http://www.samspade.org http://www.netdemon.net * today *
Kill: http://mail-abuse.com http://au.sorbs.net http://spamhaus.org
richardal...@googlemail.com
My personal view is that the whole information theory argument is no
more than a red herring. DNA behaves like DNA. It codes proteins. If
you want to treat it as "information", you need to formulate a model
of "information" based on the way DNA behaves.
The way the argument is used by creationists is to assert that DNA is
information, and then argue that because in some peculiar model of
information theory they have devised or misapplied, "new information"
cannot be created, and that therefore evolution can't happen beyond
the variation present in the DNA of an existing population.
It's a silly and dishonest argument.
RF
Steven L.
wrote in message news:brv4k5d5hbosdl012...@4ax.com:
There's no one right *definition* of "information."
But mathematical information theory, which I think the OP is alluding
to, doesn't require human intent. Information is defined as the
logarithm of the number of possible alternatives.
--
--
Steven L.
sdli...@earthlinkNOSPAM.net
Remove the "NOSPAM" before sending to this email address.
hersheyh
> Robert Carnegie: Fnord: cc talk-orig...@moderators.isc.org wrote:> Trying again in case -
The above, about the amount of DNA actually involved in encoding
proteins, is quite true and points out one of the problem with the
common use of the word "information" and the 'information theory' use
of that word (which often, interestingly enough for some definitions,
tells us that completely random and totally useless sequences have the
maximum amount information).
In common colloquial usage of the term "information", a large random
sequence is not considered "information." Sequences with
"information" (colloquial usage) are usually considered to be only
*useful* sequences (meaning useful to the organism with it or some
observer, and that is a problem in itself).
That colloquial meaning is certainly the way that creationists try to
use the term (misleading semantics is the only tool they have) and is
the only way in which their "information theory" numerology makes
sense. *If* every nucleotide were absolutely required to be that
nucleotide and only that nucleotide, then and only then would the
amount of *useful* information would be calculable by determining the
size of the genome.
That said, if all you are interested in is the amount of storage space
needed to record a single human genome, with every nucleotide in the
right place, the calculation is about right. It would be possible to
somewhat reduce the amount of storage space required by allowing
compression by name of repetitive sequences. Or one could, if coding
information and known regulatory sequences were all that you were
interested in, one could significantly reduce required storage space
by basically calling a nucleotide N whenever, to the best of current
knowledge, the exact nucleotide at that position doesn't matter.
Interestingly enough a significant number of nucleotides *within*
coding sequences can be N or close to it (especially in the 3rd letter
of codons). Of course, actually testing whether *all* possible
*combinations* of N will produce a particularly useful protein is not
an easy task and would undoubtedly produce exceptions.
But storage space is relatively cheap, so I don't see storage space as
much of a problem wrt recording the exact sequence of entire genomes.
Just don't confuse *that* entire sequence information with *useful*
information.
Steven L.
news:jqe5k5lsra01o4m6g...@4ax.com:
> On Sun, 3 Jan 2010 23:41:18 -0800 (PST), Devils Advocaat
> <mank...@yahoo.co.uk> wrote:
>
> >I have to admit my grasp of Information Theory is tenuous at best, but
> >I would like to run something past you guys and see if I am on the
> >right track.
> >
> >So here goes.
> >
> >If an apparently random message is made up of just four different
> >symbols (for example the four nucleotides in a DNA sequence), would
> >the maximum storage requirement for each symbol be
> >two binary digits?
> >
> >And if that is correct, would the maximum storage needed to contain
> >all 3 billion nucleotides of the human genome, be about 750
> >megabytes?
> >
> >If my understanding is skewed I would be grateful for anyone who can
> >put me back on track.
>
> A binary computer file of 750 MBytes, coded properly, will contain the
> complete record of the 3 billion nucleotide human genome. You can do
> it in much less using compression techniques that take advantage of
> redundancies or non-randomness in the sequence.
Given that in practice, the human genome only codes for some 22 amino
acids by codons that are 3 nucleotides long, the total information
content is even less. Since each nucleotide is one out of four
alternatives, each codon could theoretically code for 64 different
things. But the other codings are not used (except for some
"punctuation" like "stop codons").
Kleuskes & Moos
Good point.
May i humbly submit that by Shannon random sequences actually have a
larger information content than non-random equences... Usually, that
is.
Devils Advocaat
> "r norman" <r_s_nor...@comcast.net> wrote in message
As I recall with respect to the human genome:
One particular codon represents the amino acid Methionine, except for
when it represents the beginning of a protein coding sequence.
And the three "stop" codons don't represent anything other than stop.
Three of the 20 amino acids are represented by 6 different codons
each.
Five are represented by 4 codons each.
Only one is represented by 3 codons.
And the remaining nine are represented by 2 codons each.
Oddly enough not one amino acid is represented by five codons.
Its a mystery that :)
>
> --
> --
> Steven L.
> sdlit...@earthlinkNOSPAM.net
Devils Advocaat
> "desertph...@invalid-address.net" <desertph...@invalid-address.net>
> wrote in messagenews:brv4k5d5hbosdl012...@4ax.com:
Indeed I was in my own fumbling way alluding to the concept of
information as addressed in Shannon's Information Theory.
>
> --
> --
> Steven L.
> sdlit...@earthlinkNOSPAM.net