The thermodynamics of computation
Evgenii Rudnyi
> There's an interesting paper by Bennett that I ran across, which
> discusses the relation of Shannon entropy, thermodynamic entropy, and
> algorithmic entropy in the context of DNA and RNA replication:
>
>
http://qi.ethz.ch/edu/qisemFS10/papers/81_Bennett_Thermodynamics_of_computation.pdf
>
>
>
> Brent
I have browsed the paper. It is nice indeed. A couple of comments.
1) Reversible computation
The author seems not to reject the idea of reversible computation. This,
in my view, shows that the first statement from the paper
"Computers may be thought of as engines for transforming free energy
into waste heat and mathematical work."
just does not work literally. If reversible computation is possible,
then we do not have any thermodynamic limits in this respect. What is
left is just a thermal noise in form of kT.
2) Maxwell demon
I have never understood a problem with the Maxwell's demon. Why it is
not enough to say that it does not exist? Why for example Maxwell's
demon touches the imagination of physicists and engineers and the idea
of the God not?
3) Reversible chemical reactions and reversible thermodynamic processes
I think that the author misuses the term reversible in a sense that the
word has completely different meaning in thermodynamics and in
chemistry. In thermodynamics, the reversible process implies that the
entropy of the system and surrounding does not change (the entropy of
the Universe remains constant). In chemistry, a term reversible reaction
means we have two reactions (forward and backward) running in parallel.
Thereafter, by playing with conditions we could transform A to B and
then B back to A. However, when a reversible chemical reaction takes
place it is impossible to implement it as a reversible thermodynamic
process. Hence a reversible chemical reaction is not thermodynamically
reversible.
4) Algorithmic entropy
I have missed the point on the connection between the algorithmic
entropy and thermodynamic entropy. Here would be good to be back to the
Jason's example from about his work on secure pseudo-random number
generators
http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
What a thermodynamic system should be considered at all here?
In my view, the algorithm is independent of implementation details. It
seems that this is one of the points at this list when people claim that
it could be possible to make a conscious robot. Yet, how then the
thermodynamic entropy could be connected with the algorithmic entropy?
5) DNA, RNA and information
I have recently read
Barbieri, M. (2007). Is the cell a semiotic system? In: Introduction to
Biosemiotics: The New Biological Synthesis. Eds.: M. Barbieri, Springer:
179-208.
and below there is a quote. It is quite a different viewpoint on the
processes in the cell.
"At this point, we can summarize all the above concepts by saying that
in protein synthesis:
(1) Organic information is the sequence used by a copymaker during a
copying process.
(2) An organic sign is the sequence used by a codemaker during a coding
process.
(3) An organic meaning is the sequence produced by a codemaker during a
coding process.
(4) Organic information, organic signs and organic meaning are neither
quantities nor qualities. They are a new kind of natural entities which
are referred to as nominable entities.
(5) Organic information, organic signs and organic meaning have the same
scientific status as physical quantities because they are objective and
reproducible entities which can be defined by operative procedures.
(6) Organic information, organic signs and organic meaning have the same
scientific status as fundamental physical quantities because they cannot
be reduced to, or derived from, simpler entities."
Evgenii
--
http://blog.rudnyi.ru
Russell Standish
> On 09.02.2012 07:49 meekerdb said the following:
>
> > There's an interesting paper by Bennett that I ran across, which
> > discusses the relation of Shannon entropy, thermodynamic entropy, and
> > algorithmic entropy in the context of DNA and RNA replication:
> >
> > http://qi.ethz.ch/edu/qisemFS10/papers/81_Bennett_Thermodynamics_of_computation.pdf
> >
> >
> >
> > Brent
>
> I have browsed the paper. It is nice indeed. A couple of comments.
>
> 1) Reversible computation
>
> The author seems not to reject the idea of reversible computation.
> This, in my view, shows that the first statement from the paper
>
> "Computers may be thought of as engines for transforming free energy
> into waste heat and mathematical work."
>
> just does not work literally. If reversible computation is possible,
> then we do not have any thermodynamic limits in this respect. What
> is left is just a thermal noise in form of kT.
>
My understanding is that is possible to perform a reversible
computation with arbitrarily small amounts of energy provided you do
the computation slowly enough. The only way to do it for zero energy
expenditure is to not do it at all.
> 2) Maxwell demon
>
> I have never understood a problem with the Maxwell's demon. Why it
> is not enough to say that it does not exist? Why for example
> Maxwell's demon touches the imagination of physicists and engineers
> and the idea of the God not?
Its a thought experiment. Its quite well-defined (formalisable) whereas
the notion of God is not.
>
> 3) Reversible chemical reactions and reversible thermodynamic processes
>
> I think that the author misuses the term reversible in a sense that
> the word has completely different meaning in thermodynamics and in
> chemistry. In thermodynamics, the reversible process implies that
> the entropy of the system and surrounding does not change (the
> entropy of the Universe remains constant). In chemistry, a term
> reversible reaction means we have two reactions (forward and
> backward) running in parallel. Thereafter, by playing with
> conditions we could transform A to B and then B back to A. However,
> when a reversible chemical reaction takes place it is impossible to
> implement it as a reversible thermodynamic process. Hence a
> reversible chemical reaction is not thermodynamically reversible.
>
A reversible computation has the same meaning of reversible as in
thermodynamics. Change of entropy is zero. Information is
conserved. Reversible computations can never erase memory locations,
for instance, or implement assignment.
> 4) Algorithmic entropy
>
> I have missed the point on the connection between the algorithmic
> entropy and thermodynamic entropy. Here would be good to be back to
> the Jason's example from about his work on secure pseudo-random
> number generators
>
> http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
>
> What a thermodynamic system should be considered at all here?
>
> In my view, the algorithm is independent of implementation details.
> It seems that this is one of the points at this list when people
> claim that it could be possible to make a conscious robot. Yet, how
> then the thermodynamic entropy could be connected with the
> algorithmic entropy?
That seems like a non-sequitur. Could you expand on your thinking please?
>
> 5) DNA, RNA and information
>
> I have recently read
>
> Barbieri, M. (2007). Is the cell a semiotic system? In: Introduction
> to Biosemiotics: The New Biological Synthesis. Eds.: M. Barbieri,
> Springer: 179-208.
>
I'm afraid semiotics leaves me cold. I've never seen one useful
conjecture come out of it. Apologies to all those Pearceans out there.
--
----------------------------------------------------------------------------
Prof Russell Standish Phone 0425 253119 (mobile)
Principal, High Performance Coders
Visiting Professor of Mathematics hpc...@hpcoders.com.au
University of New South Wales http://www.hpcoders.com.au
----------------------------------------------------------------------------
Evgenii Rudnyi
If one well defines a thought experiment with the Maxwell's demon, then
it is quite clear that such thing does not exist. Why then to spend on
it so much time?
>>
>> 3) Reversible chemical reactions and reversible thermodynamic
>> processes
>>
>> I think that the author misuses the term reversible in a sense
>> that the word has completely different meaning in thermodynamics
>> and in chemistry. In thermodynamics, the reversible process implies
>> that the entropy of the system and surrounding does not change
>> (the entropy of the Universe remains constant). In chemistry, a
>> term reversible reaction means we have two reactions (forward and
>> backward) running in parallel. Thereafter, by playing with
>> conditions we could transform A to B and then B back to A.
>> However, when a reversible chemical reaction takes place it is
>> impossible to implement it as a reversible thermodynamic process.
>> Hence a reversible chemical reaction is not thermodynamically
>> reversible.
>>
>
> A reversible computation has the same meaning of reversible as in
> thermodynamics. Change of entropy is zero. Information is conserved.
> Reversible computations can never erase memory locations, for
> instance, or implement assignment.
It is hard to say for sure what the author meant. Let me first quote him.
p. 912(8) "It is well known that all chemical reaction are in principle
reversible: the same Brownian motion that accomplishes the forward
reaction also sometimes brings product molecules together, pushes them
backward through the transition state, and lets them emerge as reactant
molecules."
p. 934(30) "As indicated before, the synthesis of RNA by RNA polymerase
is a logically reversible copying operations, and under appropriate
(nonphysiological) conditions, it could be carried out at an energy cost
of less than kT per nucleotide."
My understanding was that in the first quote reversible has the meaning
from chemistry. Let us consider for example a reaction
A = B
with the forward reaction rate of 1000 and the backward reaction rate of
1. Then we can imagine two different initial states
1) C(A) = 1, C(B) = 0
2) C(A) = 0, C(B) = 1
The equilibrium state will be the same, but we reach it from different
sides. In both cases however the process will be thermodynamically
irreversible.
My point was that one word has different meanings and it would be good
to understand what has been meant.
>> 4) Algorithmic entropy
>>
>> I have missed the point on the connection between the algorithmic
>> entropy and thermodynamic entropy. Here would be good to be back
>> to the Jason's example from about his work on secure pseudo-random
>> number generators
>>
>> http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
>>
>> What a thermodynamic system should be considered at all here?
>>
>> In my view, the algorithm is independent of implementation
>> details. It seems that this is one of the points at this list when
>> people claim that it could be possible to make a conscious robot.
>> Yet, how then the thermodynamic entropy could be connected with
>> the algorithmic entropy?
>
> That seems like a non-sequitur. Could you expand on your thinking
> please?
I have read once more the section 6 "Algorithmic entropy and
thermodynamics" (p. 936 (30)) from the paper. I should confess that I do
not know exactly what the author meant with the algorithmic entropy. My
reading was
algorithmic entropy == entropy of an algorithm
and I have considered and will stick to this meaning.
In my understanding, when we consider an algorithm, this is a pure IT
construct, that does not depend whether I will implement it with an
abacus or some Turing machine, with Intel or PowerPC processor. From
this follows that the algorithm and hence its entropy does not depend on
temperature or pressure of a physical system that does the computation.
In my view it makes sense.
Let us consider consciousness now. Our brains produces it and our brain
has some thermodynamic entropy. If we assume that the same effect could
be achieved with some robot, does it mean that the thermodynamic entropy
of the robot must be the same as that of the brain?
>>
>> 5) DNA, RNA and information
>>
>> I have recently read
>>
>> Barbieri, M. (2007). Is the cell a semiotic system? In:
>> Introduction to Biosemiotics: The New Biological Synthesis. Eds.:
>> M. Barbieri, Springer: 179-208.
>>
>
> I'm afraid semiotics leaves me cold. I've never seen one useful
> conjecture come out of it. Apologies to all those Pearceans out
> there.
Everything is in comparison. Recently I got interested in Artificial
Life and people have recommended me Christoph Adami �Introduction to
Artificial Life�. He for example claims
p. 5 "An even more general approach is the thermodynamic one, which
attempts to define living systems in terms of their ability to maintain
low levels of entropy, or disorder, only".
One could say something like this but then the question what is the
entropy. And this is what Adami writes about the entropy
p. 94 �Entropy is a measure of the disorder present in a system, or
alternatively, a measure of our lack of knowledge about this system.�
p. 96 �If an observer gains knowledge about the system and thus
determines that a number of states that were previously deemed probable
are in fact unlikely, the entropy of the system (which now has turned
into a conditional entropy), is lowered, simply because the number of
different possible states in the lower. (Note that such a change in
uncertainty is usually due to a measurement).
p. 97 �Clearly, the entropy can also depend on what we consider
�different�. For example, one may count states as different that differ
by, at most, del_x in some observable x (for example, the color of a
ball drawn from an ensemble of differently shaded balls in an urn). Such
entropies are then called fine-grained (if del_x is small), or
course-grained (if del_x is large) entropies.�
I am a thermodynamicist and frankly speaking I was just shocked after
reading it. For me it was clear that Adami does not know what the
experimental thermodynamics is (and presumably he is unaware of the
experimental thermodynamics at all).
I understand now that Adami's viewpoint is quite common among physicists
but I do not think that this brings us "useful conjecture come out of
it". I had discussion about this on the biotaconv list, see summary at
http://blog.rudnyi.ru/2010/12/entropy-and-artificial-life.html
but no one there could explain me what is the differences in
consequences in artificial life research between the two statements
1) The thermodynamic and information entropies are equivalent.
2) The thermodynamic and information entropies are completely different.
It seems that either 1) or 2) does not influence artificial life
research at all.
In this sense, I like the Barbieri's paper much more. At least I could
follow his logic.
Evgenii
John Clark
> If one well defines a thought experiment with the Maxwell's demon, then it is quite clear that such thing does not exist. Why then to spend on it so much time?
Maxwell's demon is possible in classical physics and it was not clear that quantum mechanics made it impossible until 1929 when Leo Szilard proved that to be the case. And understanding just why it can not exist aids in understanding the relationship between energy information entropy and reversibility. Maxwell's demon was the starting point for Rolf Landauer's discovery in 1960 that erasing information always requires energy and increases entropy because it's thermodynamically irreversible.
The bottom line is that Maxwell's demon would work if it had information on when to open and close its shutter, if it had that information it could decrease the pressure on one side of a tank filled with gas and increase it on the other without expending appreciable energy and use that pressure difference to do work. However the stumbling block is the information, it would take more energy to obtain that information than you'd get from that work. Actually by pure chance Maxwell's demon can work, but only very very rarely and only for a very very short time.
John K Clark
Everything is in comparison. Recently I got interested in Artificial Life and people have recommended me Christoph Adami “Introduction to Artificial Life“. He for example claims
p. 5 "An even more general approach is the thermodynamic one, which attempts to define living systems in terms of their ability to maintain low levels of entropy, or disorder, only".
One could say something like this but then the question what is the entropy. And this is what Adami writes about the entropy
p. 94 “Entropy is a measure of the disorder present in a system, or alternatively, a measure of our lack of knowledge about this system.”
p. 96 “If an observer gains knowledge about the system and thus determines that a number of states that were previously deemed probable are in fact unlikely, the entropy of the system (which now has turned into a conditional entropy), is lowered, simply because the number of different possible states in the lower. (Note that such a change in uncertainty is usually due to a measurement).
p. 97 “Clearly, the entropy can also depend on what we consider “different”. For example, one may count states as different that differ by, at most, del_x in some observable x (for example, the color of a ball drawn from an ensemble of differently shaded balls in an urn). Such entropies are then called fine-grained (if del_x is small), or course-grained (if del_x is large) entropies.”
I am a thermodynamicist and frankly speaking I was just shocked after reading it. For me it was clear that Adami does not know what the experimental thermodynamics is (and presumably he is unaware of the experimental thermodynamics at all).
I understand now that Adami's viewpoint is quite common among physicists but I do not think that this brings us "useful conjecture come out of it". I had discussion about this on the biotaconv list, see summary at
http://blog.rudnyi.ru/2010/12/entropy-and-artificial-life.html
but no one there could explain me what is the differences in consequences in artificial life research between the two statements
1) The thermodynamic and information entropies are equivalent.
2) The thermodynamic and information entropies are completely different.
It seems that either 1) or 2) does not influence artificial life research at all.
In this sense, I like the Barbieri's paper much more. At least I could follow his logic.
Evgenii
--
You received this message because you are subscribed to the Google Groups "Everything List" group.
To post to this group, send email to everything-list@googlegroups.com.
To unsubscribe from this group, send email to everything-list+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/everything-list?hl=en.
Evgenii Rudnyi
> On Sun, Feb 19, 2012 at 5:32 AM, Evgenii Rudnyi<use...@rudnyi.ru>
> wrote:
>
>>
>>> If one well defines a thought experiment with the Maxwell's
>>> demon, then
>> it is quite clear that such thing does not exist. Why then to spend
>> on it so much time?
>
>
> Maxwell's demon is possible in classical physics and it was not clear
I am not sure I understand what do you mean. How Maxwell's demon is
possible in classical physics? I personally would say that Maxwell's
demon could work just within a brain of some crazy scientist.
Evgenii
> that quantum mechanics made it impossible until 1929 when Leo Szilard
> proved that to be the case. And understanding just why it can not
> exist aids in understanding the relationship between energy
> information entropy and reversibility. Maxwell's demon was the
> starting point for Rolf Landauer's discovery in 1960 that erasing
> information always requires energy and increases entropy because it's
> thermodynamically irreversible.
>
> The bottom line is that Maxwell's demon would work if it had
> information on when to open and close its shutter, if it had that
> information it could decrease the pressure on one side of a tank
> filled with gas and increase it on the other without expending
> appreciable energy and use that pressure difference to do work.
> However the stumbling block is the information, it would take more
> energy to obtain that information than you'd get from that work.
> Actually by pure chance Maxwell's demon can work, but only very very
> rarely and only for a very very short time.
>
> John K Clark **
Russell Standish
> On Sun, Feb 19, 2012 at 5:32 AM, Evgenii Rudnyi <use...@rudnyi.ru> wrote:
>
> >
> > > If one well defines a thought experiment with the Maxwell's demon, then
> > it is quite clear that such thing does not exist. Why then to spend on it
> > so much time?
>
>
> Maxwell's demon is possible in classical physics and it was not clear that
> quantum mechanics made it impossible until 1929 when Leo Szilard proved
> that to be the case. And understanding just why it can not exist aids in
> understanding the relationship between energy information entropy and
> reversibility. Maxwell's demon was the starting point for Rolf Landauer's
> discovery in 1960 that erasing information always requires energy and
> increases entropy because it's thermodynamically irreversible.
>
Good answer John. Does anyone want to pick on Evgeni's comments about
Chris Adami's book?
It weird, because Chris's book gives some of the bext examples of the
application of statistical physics to artificial life. In particular,
his observation that mutation should play an analogous role to
temperature in an evolutionary process, and that several evolutionary
regimes exist as mutation is varied, corresponding to phase
transitions in materials.
This phenomena I have observed in my own evolutionary
experiments. Plus, it appears to be correlated to Mark Bedau's
evolutionary classes.
This is the paper I usually refer to, although his ideas have evolved
somewhat since 1998:
M. A. Bedau, E. Snyder, N. H. Packard. 1998. A Classification of
Long-Term Evolutionary Dynamics. In C. Adami, R. Belew, H. Kitano, and
C. Taylor, eds., Artificial Life VI, pp. 228-237. Cambridge: MIT
Press. Also published as Working Paper No.98-03-025, Santa Fe
Institute, Santa Fe, NM.
http://people.reed.edu/~mab/publications/papers/alife6.pdf
Evgenii Rudnyi
> On Sun, Feb 19, 2012 at 11:21:01AM -0500, John Clark wrote:
>> On Sun, Feb 19, 2012 at 5:32 AM, Evgenii Rudnyi<use...@rudnyi.ru>
>> wrote:
>>
>>>
>>>> If one well defines a thought experiment with the Maxwell's
>>>> demon, then
>>> it is quite clear that such thing does not exist. Why then to
>>> spend on it so much time?
>>
>>
>> Maxwell's demon is possible in classical physics and it was not
>> clear that quantum mechanics made it impossible until 1929 when Leo
>> Szilard proved that to be the case. And understanding just why it
>> can not exist aids in understanding the relationship between energy
>> information entropy and reversibility. Maxwell's demon was the
>> starting point for Rolf Landauer's discovery in 1960 that erasing
>> information always requires energy and increases entropy because
>> it's thermodynamically irreversible.
>>
>
> Good answer John. Does anyone want to pick on Evgeni's comments
> about Chris Adami's book?
>
> It weird, because Chris's book gives some of the bext examples of
> the application of statistical physics to artificial life. In
> particular, his observation that mutation should play an analogous
> role to temperature in an evolutionary process, and that several
> evolutionary regimes exist as mutation is varied, corresponding to
> phase transitions in materials.
I have nothing against Adami's book as such. His description of his
software avida and his experiments with it are okay. My point was about
his claim that his work has something to do with thermodynamics. It is
definitely not. The thermodynamic entropy is not there. The quotes from
the book displays this pretty clear.
You have written about "an analogous role". I would not object if you
say that there is an analogy between the thermodynamic entropy and
information. Yet, I am against the statement that the thermodynamic
entropy is information and I believe that I have given many examples
that show this. Thermodynamic entropy is not subjective and not context
dependent*, so my claim is that Adami does not understand what the
thermodynamic entropy is. He has never taken a class in experimental
thermodynamics, this is the problem.
* I would accept the notation that the entropy is context dependent in a
sense that its definition depends on the thermodynamics theory. If we
change the theory, then the entropy could have some other meaning. But
it seems not what you have meant.
Evgenii
meekerdb
What you are overlooking is that information is *about* things. So entropy in
thermodynamics is information about the system's location in phase space. That's what
connects "information" and "work" and "temperature". Entropy in communication theory is
about the location of a message in message space. It's a different application of the
same concept. The two overlap when considering the minimum free energy requirements of a
physical realization of a computation - but existing computers operate far above those
minimums so the overlap is only of theoretical interest.
> Thermodynamic entropy is not subjective and not context dependent*, so my claim is that
> Adami does not understand what the thermodynamic entropy is. He has never taken a class
> in experimental thermodynamics, this is the problem.
I'm beginning to think you have never taken a class in statistical mechanics. There's a
good online course here:
http://farside.ph.utexas.edu/teaching/sm1/lectures/lectures.html
Those particularly relevant to this thread start at
http://farside.ph.utexas.edu/teaching/sm1/lectures/node61.html
and go through the next six or seven.
Brent
Evgenii Rudnyi
What is left is to apply your concept to examples in practice. Then it
would be more clear what you mean. Let me repeat just one question that
you have not answered yet (but I believe that I have given much more
examples and they have not been worked out).
The only example of the entropy used by engineers in informatics has
been given by Jason and I will quote him below. Could you please tell
me, the thermodynamic entropy of what is discussed in his example?
I am ready to learn the meaning of information in thermodynamics. Please
just explain it by means of practical examples. I personally do not see
thermodynamics in the Jason's work. Please just explain what I am missing.
On 03.02.2012 00:14 Jason Resch said the following:
�
> Evgenii,
>
> Sure, I could give a few examples as this somewhat intersects with my
> line of work.
>
> The NIST 800-90 recommendation (
> http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf )
> for random number generators is a document for engineers implementing
> secure pseudo-random number generators. An example of where it is
> important is when considering entropy sources for seeding a random
> number generator. If you use something completely random, like a
> fair coin toss, each toss provides 1 bit of entropy. The formula is
> -log2(predictability). With a coin flip, you have at best a .5
> chance of correctly guessing it, and -log2(.5) = 1. If you used a
> die roll, then each die roll would provide -log2(1/6) = 2.58 bits of
> entropy. The ability to measure unpredictability is necessary to
> ensure, for example, that a cryptographic key is at least as
> difficult to predict the random inputs that went into generating it
> as it would be to brute force the key.
>
> In addition to security, entropy is also an important concept in the
> field of data compression. The amount of entropy in a given bit
> string represents the theoretical minimum number of bits it takes to
> represent the information. If 100 bits contain 100 bits of entropy,
> then there is no compression algorithm that can represent those 100
> bits with fewer than 100 bits. However, if a 100 bit string contains
> only 50 bits of entropy, you could compress it to 50 bits. For
> example, let�s say you had 100 coin flips from an unfair coin. This
> unfair coin comes up heads 90% of the time. Each flip represents
> -log2(.9) = 0.152 bits of entropy. Thus, a sequence of 100 coin
> flips with this biased coin could be represent with 16 bits. There
> is only 15.2 bits of information / entropy contained in that 100 bit
> long sequence.
>> Thermodynamic entropy is not subjective and not context dependent*,
>> so my claim is that Adami does not understand what the
>> thermodynamic entropy is. He has never taken a class in
>> experimental thermodynamics, this is the problem.
>
> I'm beginning to think you have never taken a class in statistical
> mechanics. There's a good online course here:
I have done a class in statistical thermodynamics. Actually it was a
pretty good class where different approaches of Boltzmann, Gibbs and
other have been considered in detail.
The difference is that I do not believe that a similar equation in
different areas imply that the different things are the same.
If you would like to show that information is very useful in
thermodynamics, please apply it to simple thermodynamic problems to show
how the concept of information has simplified for example the
computation of the phase diagram (or equilibrium composition between N2,
H2 and NH3). Should I repeat my examples?
Evgenii
meekerdb
The link I sent below works out the entropy of an ideal gas using information. You keep
asking for "practical examples" but that's like asking for practical examples of
calculating molecular reaction free energy from quantum mechanics. It is very difficult
because it depends on the electron energy levels. It has been done in a few simple (not
necessarily practical) cases as a proof of principle. But it is not the way engineering
or chemistry is done because it is both easier and more reliable to measure them. But
that doesn't mean that they don't have energy or that the concept of energy doesn't
apply. No one calculates the strength of steel from carbon and iron atomic bonds and
crystal structure either. But that doesn't mean the strength of steel is a separate,
independent property.
Other, smarter people have already done that.
> please apply it to simple thermodynamic problems to show how the concept of information
> has simplified for example the computation of the phase diagram (or equilibrium
> composition between N2, H2 and NH3). Should I repeat my examples?
No, you should consider why chemists don't just calculate all reactions and structure from
atomic theory and QM.
Brent
Russell Standish
>
> I have nothing against Adami's book as such. His description of his
> software avida and his experiments with it are okay. My point was
> about his claim that his work has something to do with
> thermodynamics. It is definitely not. The thermodynamic entropy is
> not there. The quotes from the book displays this pretty clear.
>
> You have written about "an analogous role". I would not object if
Chris uses the word analogy to connect mutation and temperature. But
not between information and entropy.
> you say that there is an analogy between the thermodynamic entropy
> and information. Yet, I am against the statement that the
> thermodynamic entropy is information and I believe that I have given
> many examples that show this. Thermodynamic entropy is not
> subjective and not context dependent*, so my claim is that Adami
> does not understand what the thermodynamic entropy is. He has never
> taken a class in experimental thermodynamics, this is the problem.
>
I can't speak for Chris, but somehow I doubt that very much.
> * I would accept the notation that the entropy is context dependent
> in a sense that its definition depends on the thermodynamics theory.
> If we change the theory, then the entropy could have some other
> meaning. But it seems not what you have meant.
>
It is true that in thermodynamics, there is usually little argument
about what the macroscopic variables are. As a consequence, entropy is
essentially an objective quantity, and the context fades into the
background.
But even between (micro-/grand-) canonical ensembles, there are subtle
differences between what macroscopic variables are significant, hence
difference between the entropies, which vanish in the thermodynamic
limit.
Evgenii Rudnyi
> On 2/20/2012 12:02 PM, Evgenii Rudnyi wrote:
...
>> I am ready to learn the meaning of information in thermodynamics.
>> Please just explain it by means of practical examples.
>
>
> The link I sent below works out the entropy of an ideal gas using
> information. You keep asking for "practical examples" but that's like
> asking for practical examples of calculating molecular reaction free
> energy from quantum mechanics. It is very difficult because it
> depends on the electron energy levels. It has been done in a few
> simple (not necessarily practical) cases as a proof of principle. But
> it is not the way engineering or chemistry is done because it is both
> easier and more reliable to measure them. But that doesn't mean that
> they don't have energy or that the concept of energy doesn't apply.
> No one calculates the strength of steel from carbon and iron atomic
> bonds and crystal structure either. But that doesn't mean the
> strength of steel is a separate, independent property.
I do not get your point. Chemists use molecular simulation extensively
and you will find the works where even phase diagram are computed from
the first principle. Please run
phase diagram from the first principles
on the Google Scholar. Yet, information is not there. Hence I am lost.
...
>> If you would like to show that information is very useful in
>> thermodynamics,
>
> Other, smarter people have already done that.
Could you please give an example? Then it would be easier to understand
your position.
>> please apply it to simple thermodynamic problems to show how the
>> concept of information has simplified for example the computation
>> of the phase diagram (or equilibrium composition between N2, H2 and
>> NH3). Should I repeat my examples?
>
> No, you should consider why chemists don't just calculate all
> reactions and structure from atomic theory and QM.
You underestimate chemists. As I have mentioned they use molecular
simulation extensively. You can find some examples in my old lectures
(they are a bit outdated though as they are about eight years old)
http://evgenii.rudnyi.ru/teaching.html#md
But Shannon's information is not there.
Evgenii
> Brent
Evgenii Rudnyi
> On Mon, Feb 20, 2012 at 07:33:13PM +0100, Evgenii Rudnyi wrote:
>>
>> I have nothing against Adami's book as such. His description of
>> his software avida and his experiments with it are okay. My point
>> was about his claim that his work has something to do with
>> thermodynamics. It is definitely not. The thermodynamic entropy is
>> not there. The quotes from the book displays this pretty clear.
>>
>> You have written about "an analogous role". I would not object if
>
> Chris uses the word analogy to connect mutation and temperature. But
> not between information and entropy.
This what I have meant and this is the point where I disagree. Adami
comes to the conclusion that the thermodynamic entropy is subjective.
Let me quote him again
p. 96 �If an observer gains knowledge about the system and thus
determines that a number of states that were previously deemed probable
are in fact unlikely, the entropy of the system (which now has turned
into a conditional entropy), is lowered, simply because the number of
different possible states in the lower. (Note that such a change in
uncertainty is usually due to a measurement).
p. 97 �Clearly, the entropy can also depend on what we consider
�different�. For example, one may count states as different that differ
by, at most, del_x in some observable x (for example, the color of a
ball drawn from an ensemble of differently shaded balls in an urn). Such
entropies are then called fine-grained (if del_x is small), or
course-grained (if del_x is large) entropies.�
The entropy he is talking about in these quotes has nothing to do with
the thermodynamic entropy. You can close or open your eyes, the
entropies as determined in the JANAF Tables do not change.
>
>> you say that there is an analogy between the thermodynamic entropy
>> and information. Yet, I am against the statement that the
>> thermodynamic entropy is information and I believe that I have
>> given many examples that show this. Thermodynamic entropy is not
>> subjective and not context dependent*, so my claim is that Adami
>> does not understand what the thermodynamic entropy is. He has
>> never taken a class in experimental thermodynamics, this is the
>> problem.
>>
>
> I can't speak for Chris, but somehow I doubt that very much.
>
>> * I would accept the notation that the entropy is context
>> dependent in a sense that its definition depends on the
>> thermodynamics theory. If we change the theory, then the entropy
>> could have some other meaning. But it seems not what you have
>> meant.
>>
>
>
> It is true that in thermodynamics, there is usually little argument
> about what the macroscopic variables are. As a consequence, entropy
> is essentially an objective quantity, and the context fades into the
> background.
>
> But even between (micro-/grand-) canonical ensembles, there are
> subtle differences between what macroscopic variables are
> significant, hence difference between the entropies, which vanish in
> the thermodynamic limit.
>
What does it mean for the application I have mentioned and for
information in the IT? I still do not understand this, as the numerical
values of information in IT and as derived from the thermodynamic
entropy are quite different. Hence it is completely unclear how to use
this in practical applications. Then what does it bring?
You have written about semiotics
"I've never seen one useful conjecture come out of it."
What are useful conjecture from saying that because the equations for
information and the entropy are the same, they must be the same thing?
Evgenii
Evgenii Rudnyi
...
> I'm beginning to think you have never taken a class in statistical
> mechanics. There's a good online course here:
>
> http://farside.ph.utexas.edu/teaching/sm1/lectures/lectures.html
>
> Those particularly relevant to this thread start at
>
> http://farside.ph.utexas.edu/teaching/sm1/lectures/node61.html
>
> and go through the next six or seven.
>
I wanted to study this text to understand the relationship between the
entropy and information better. However, I cannot find information in
there, say I guess
"How can we obtain some information about the statistical properties of
the molecules which make up air?"
you do not mean the term information in this sentence.
It seems that this is a normal course on statistical thermodynamics as I
get used to where there is no notion of information in thermodynamics.
Have I missed something? How this link helps us in our discussion from
your viewpoint?
Evgenii