Chatentropy
3 views
Skip to first unread message
Alexandre Wang
May 14, 2009, 8:05:30 AM5/14/09
to chate...@googlegroups.com
All the postings to date
--------------------------------------------------------------------------------
De : jpba...@numericable.com [mailto:jpba...@numericable.com]
Envoyé : lundi 11 mai 2009 11:14
À :
Dear colleagues,
I am pleased to be accepted in this group discussion. Thanks Alexandre to organize this group.
I find the comments by S. Abe very clear and, to my opinion they represent a good starting point for the discussion between complex systems and thermodynamics.
In the middle of the last century, just before world war two, a big problem was : can you describe the critical behavior -i.e. essentially the existence of power laws - starting from the standard thermodynamics? Onsager gave an answer by solving exactly the Ising model. The answer is "Yes we can !" ( a very famous sentence today) I do not know if this is related to the discussion but due to the Kubo's theory -which is a linear response theory- it is possible to calculate some quantities - the diffusion coeffiecient, for instance, with the help of time correlation functions (the velocity-velocity in the case of diffusion). Of course it was first tempting to assume that such functions depend exponentially on time. Later more precise calculations and simulations based on molecular dynamics have shown that the decreasing of the correlation function was not exponential but exhibits for long time a dependence like time power -d/2 where d is the dimension of space. This is another example where the usual statistical thermodynamics may generate naturally some power laws. Another example concerns the ionic solutions for which it is possible to show that standard calculations lead to quantities that are not analytical functions of the ionic density. This is a consequence of coulombic potential which is long ranged.
Finally, if we start from usual statiscal mechanics we can recover the usual thermodynamics in the so called thermodynamic limit. In some cases this limit is obtained using the Stirling formula restricted to its first term. But if the system is not infinitely large we have to keep the next term in the Stirling formula. Immediatly this leads to non traditional results but issue from traditional ones. I am not a specialist but this seems in the spirit of Tsallis.
These few examples show that usual statistical mechanics can lead to the existence of power laws.
Now, for me the question that appears is how to define a complex system? If we work with systems having nothing to do with thermodynamics why entropy should be an efficient concept to start a totally new approach ?
Friendly.
J.P. Badiali
-----Message d'origine-----
De : su...@sf6.so-net.ne.jp [mailto:su...@sf6.so-net.ne.jp] Envoyé : jeudi 7 mai 2009 19:14 À :
Dear Ladies and Gentlemen,
First of all, I am pleased very much to get involved with your discussion forum.
Actually, I have already learned some things from your discussions.
It seems to me that the main issue at the moment is concerned with generalized entropies in connection with non-exponential distributions.
The situation may actually be more complicated than what we naively imagine.
It is my opinion that the point is if we really wish to connect these discussions with thermodynamics. Here, "thermodynamics" means not only equilibrium one but also stationary nonequilibrium. A question is if there exists stationary statistical mechanical theory that can describe systems in nonequilibrium stationary states.
In this respect, we can go back to Paul and Tatiana Ehrenfest, who have asked the following question already in 1912: Is there some measure like the entropy, which achieves its extremal value in a nonequilibrium stationary state?
In general, so-called complex systems in nonequilibrium stationary states stay in such states for a very long time, much longer than their typical microscopic times scales. This leads one to imagine that such states may maximize a certain measure such as the entropy.
I should accept that a naive approach based on generalized entropies is profoundly inconsistent with the thermodynamic principles, actually. Therefore, the current attempts such as Tsallis' q-statistics are still very unclear.
However, at the same time, still I can imagine that there will a way to go beyond the exact equilibrium theory in order to describe complex systems with broken ergodicity/mixing and nontrivial phase-space structures.
Yours sincerely,
Sumiyoshi Abe
-----Message d'origine-----
De : Alexandre Wang
Envoyé : jeudi 7 mai 2009 19:11
À : '
Objet : group discussion of generic entropy
Dear All,
First let me welcome warmly Sonia, Stefan, Rudolf and Lu Yi who join us in the group discussion.
Dr. Sonia R. Bentes is from the Instituto Superior de Contabilidade e Administração de Lisboa
Dr. Stefan Thurner and Dr. Rudolf Habel are from the Complex Systems Research Group Medical University of Vienna
Dr. Lu Yi is from the Normal University of Baotou, Inner Mongolia (my country)
Several people asked me about the origin of this discussion and some precision on the discussed questions.
The origin of the discussion is a personel note I made about several generalized entropies. You can see a part of it in the attached file. Obviously, the underlying idea in these tentatives of generalization surpasses the previous idea of generalizing the Shannon form with such and such given functional forms (Renyi, Tsallis or others).
The question is why we do that, especially when the probability is already known (my friend Eduard harassed me with this question :-<). In the discussion, Eduard also put forward other questions and insertions as you can see in the discussion to date.
Please read the note which is attached and, if you wish, send your remarks to everybody in the mailing list.
Bon weekend
Alexandre
-----Message d'origine-----
De : Alexandre Wang
Envoyé : mardi 5 mai 2009 15:48
À :
Dear all,
First let's welcome Antonio to our group. Dr. Antonio Scarfone is from Politechnico di Torino, Italy.
Thanks to Eduard for the last remarks. For people who don't have access to PRE, the paper of Eduard mentioned in his message can be downloaded via arXiv:cond-mat/0604573
Other papers related directly to this furum are the following (among others which can be found in the references of these papers)
-S. Abe, J. Physics A: Math. Gen. {\bf 36}(2003)8733 (arXiv:cond-mat/0211437)
-S. Abe, G. Kaniadakis and A. M. Scarfone. J. Phys. A: Math. Gen. {\bf 37}(2004)10513 (arXiv:cond-mat/0401290)
-S. Thurner and Rudolf Hanel, Entropies for complex systems: generalized-generalized entropies, AIP conference proceedings, {\bf 956}(2007)68-75 (arXiv:0709.1685)
-Q.A. Wang, Probability distribution and entropy as a measure of uncertainty, J. Phys. A:
Math. Theor., {\bf 41}(2008)065004 (arXiv:cond-mat/0612076)
- Stefan Thurner and Rudolf Hanel, Maximum entropy approach to central limit distributions of correlated variables, arXiv:0804.3477
You can add references into this list.
I am thinking about establishing a website with restricted access which would be more agreeable to write, read and easier to keep for future project (we can think about a book in which each one summarizes his contribution). But I don't know yet what is the best approach to the creation of this site. If someone has experience or know how to do that, please give me an instruction by sending a message to me. Thanks.
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : mardi 28 avril 2009 14:50 À : Alexandre Wang Cc : Alain Le Mehauté; Aziz El Kaabouchi; François Tsobnang; Laurent Nivanen; jpba...@numericable.com; jcchen; Congjie Ou 欧聪杰; gzsu; Li Wei; Jian Jiang; xc...@mail.ccnu.edu.cn; paul lescot; Giorgio Kaniadakis; su...@sf6.so-net.ne.jp; Dmitrii Tayurskii; YuraLysogorskiy; Zvezdov Denis; Raoul; wr0124; jiulin du; yilu...@163.com; QUARATI PIERO; Tongling Lin Objet : Re: TR:
Dear all,
I think it is a good idea to launch a collective discussion.
I am ready to participate.
Now, on the latest portion of the discussion
> -----Message d'origine-----
> De : Alexandre Wang
> Envoyé : vendredi 24 avril 2009 18:13
> À : 'Eduard VAKARIN'
> Objet : RE:
>
> Dear Eduard,
> I have just come back from China. I saw your message in China. But I
> didn't have time there for the discussion.
>
> I see that you are very attached to the referential information
> theory based on Shannon-Khinchin hypotheses. I understand.
I am not very attached. Just I belive the logics behind and do not see why I have to "invent" an alternative entropy measure which (in addition) contains unknown parameters, or is even non-additive for independent systems.
If one cannot obtain a non-exponential distribution from the Shannon form with a linear constraint, then this means that the physics behind is more complex.
I mean we have to find an appropriate constraint leading to the result of interest. I have tried to do that in a recent paper (Phys. Rev. E 74, 036120 (2006)).
> My intention is not to mixe the entropy of thermodynamics and the
> entropy in information theory. I don't know whether or not there is
> relationship between these two entropies, although there are already
> some hints such as Landauer's erasure principle.
Exactly. This principle really hints that there is a link between the two entropies, but they are not identical.
> So what I have tried to do is to look at the thermodynamic entropy,
> to do something similar to what is done in information theory, that
> is to search for the functional form of thermodynamic entropy, to
> prove the maxent from other physics principles and to find the way
> to include non exponential probability in this framework. The first
> part of this work is for equilibrium system. This is the reason why
> I talked about thermodynamic laws.
Thermodynamic entropy is a state function S(N,V,T), it is not a functional.
The thermodynamic laws tell us what what we can do with it. So I do not think that we are free "to do something similar...". Otherwise we are out of the thermodynamics. That will be something else.
> I am not so sure but it seems to me that for this purpose one is
> obliged to overcome the Shannon formula if it is used as functionals
> of physics entropy, since with non exponential probability there is
> contradiction between Shannon formula and thermodynamic laws :
> dS=\beta*(d<E>-<dE>).
Again, thermodynamic entropy is not a functional. The Shannon (information) entropy is not (in general) a thermodynamic object, so I do not see any contradiction with thermodynamic laws. The probability distributions (exponential or non-exponential) have no direct relation with the thermodynamics. As I already told, it is possible to get a non-exponential distribution from the Shannon form, but the constraint should be non-linear.
In addition, this illuminates the fhysics behind: a non-exponential distribution (in energy, for instance) appears as a result of a restricted phase space (the thermodynamic entropy is constrained). The latter points out towards an non-Gibbsian physics (either finite systems, or stationary non-equilibrium states).
>
> As you have noticed, this is a contrary approach to inference
> theory. Each time when I talk about entropy, it is the physics
> entropy but not the informational entropy for inference. I think I
> have proved the maxent property for this physics entropy from other
> principles. The maxent of inference theory is not concerned in my
> idea.
If it is a "fhysical entropy" in the thermodynamic meaning, then one has to clarify how the probability distributions enter there.
I would understand a formulation of the "varentropy" in a probabilistic meaning. That is as an attempt to find some general features without resorting to a specific functional form for the information entropy.
> The word "correct measure" is also related to physics entropy. I
> cannot yet give you convincing reason for this notion. What I can
> say id that when I think about this word, the question I asked
> myself is the following: Suppose a given prob distribution
> (exponential for example) of a physical system whose uncertainty is
> calculated by using two functionals (with the same unity): Shannon
> formula and, say, the formula derived for stretched expontial
> distribution (Abe, Plastino). You will obtain two values for the
> uncertainty. You may say that the two values are both correct. If
> the state or the distribution of the system does not change, this
> conclusion would be Ok.
Yes, I agree. In order to measure the uncertainty one may use any convenient estimator.
> Now the exponential distribution, after an evolution to another
> equilibrium state, becomes streched exponential one. You may be
> interested in the (physical!) entropy variation.
First of all how is it possible to drive (in equilibrium) a system between two EQUILIBRIUM states with different distributions? If such a situation takes place, then this immediately suggests that something had happend with your system.
> The two formula will give two different variations. But I know that
> the variation of the physical entropy (of second law) can be uniquely
> determined for this given process. So what formula should be used if
> we want to calculate the variation through the entropy functional? I
> think that the correct measure (functional) should give the uniquely
> determined correct variation.
Again, an entropy calculated for a given distribution (whatever the distribution and the information entropy forms) and the one entering the second law are two different objects. It is not evident that starting from a given information functional we will be able to built up a scheme similar to the standard thermodynamics.
With best wishes
Eduard
______________________________________________
De : Alexandre Wang
Envoyé : lundi 27 avril 2009 18:30
À : Alain Le Mehauté; Aziz El Kaabouchi; François Tsobnang; Laurent Nivanen; 'jpba...@numericable.com'; 'jcchen'; 'Congjie Ou 欧聪杰'; 'gzsu'; 'Li Wei'; Jian Jiang; 'xc...@mail.ccnu.edu.cn'; 'paul lescot'; 'Giorgio Kaniadakis'; 'su...@sf6.so-net.ne.jp'; 'Dmitrii Tayurskii'; 'Yura Lysogorskiy'; 'Zvezdov Denis'; 'Raoul'; 'wr0124'; 'jiulin du'; 'yilu...@163.com'; 'QUARATI PIERO'; 'Tongling Lin'
Cc : 'Eduard VAKARIN'
Objet : TR:
Dear all, I would like to inform you of this discussion started recently with a good physicist and also a good friend, Dr. E. Vakarin, who asked me some crucial (and deep) questions about a recent entropic approach to complex system (proposed by Abe, Kaniadakis et al, and ismans scientists a bit later). I would like to invite everybody, if you are interested, to participate in this discussion for the simple reason that group discussion is much more enriching than two person discussing secretly and mysteriously :-), and that I do not know every answer to the present questions and (I hope) the coming ones
If you want to discuss, please click "Reply to all" (Repondre a tous). Please just let me know if someone does not want to receive discussions from the group.
For the PhD and graduate students in this group: if you have specific technical questions about some basic notions or needs of references, please do not send message by "reply to all". Just send a message to me, then I can ask the right person to answer your questions if I cannot do it. Thanks
Welcome to Eduard to our small (and complex) network.
Alexandre
-----Message d'origine-----
De : Alexandre Wang
Envoyé : vendredi 24 avril 2009 18:13
À : 'Eduard VAKARIN'
Objet : RE:
Dear Eduard,
I have just come back from China. I saw your message in China. But I didn't have time there for the discussion.
I see that you are very attached to the referential information theory based on Shannon-Khinchin hypotheses. I understand.
My intention is not to mixe the entropy of thermodynamics and the entropy in information theory. I don't know whether or not there is relationship between these two entropies, although there are already some hints such as Landauer's erasure principle.
So what I have tried to do is to look at the thermodynamic entropy, to do something similar to what is done in information theory, that is to search for the functional form of thermodynamic entropy, to prove the maxent from other physics principles and to find the way to include non exponential probability in this framework. The first part of this work is for equilibrium system. This is the reason why I talked about thermodynamic laws.
I am not so sure but it seems to me that for this purpose one is obliged to overcome the Shannon formula if it is used as functionals of physics entropy, since with non exponential probability there is contradiction between Shannon formula and thermodynamic laws : dS=\beta*(d<E>-<dE>).
As you have noticed, this is a contrary approach to inference theory. Each time when I talk about entropy, it is the physics entropy but not the informational entropy for inference. I think I have proved the maxent property for this physics entropy from other principles. The maxent of inference theory is not concerned in my idea.
The word "correct measure" is also related to physics entropy. I cannot yet give you convincing reason for this notion. What I can say id that when I think about this word, the question I asked myself is the following: Suppose a given prob distribution (exponential for example) of a physical system whose uncertainty is calculated by using two functionals (with the same unity): Shannon formula and, say, the formula derived for stretched expontial distribution (Abe, Plastino). You will obtain two values for the uncertainty. You may say that the two values are both correct. If the state or the distribution of the system does not change, this conclusion would be Ok.
Now the exponential distribution, after an evolution to another equilibrium state, becomes streched exponential one. You may be interested in the (physical!) entropy variation. The two formula will give two different variations. But I know that the variation of the physical entropy (of second law) can be uniquely determined for this given process. So what formula should be used if we want to calculate the variation through the entropy functional? I think that the correct measure (functional) should give the uniquely determined correct variation.
Have I answered your questions?
Have a nice weekend
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : jeudi 16 avril 2009 12:48 À : Alexandre Wang Objet : RE:
Dear Alexandre,
I hope you are safely back from China.
Let us continue our discussion.
First of all, what do you mean under a "correct" and the best measure?
As we have already discussed, if it is simply a "spread" measure, then we may use whatever "convenient" estimation. On the other hand if we look at the entropy as the inference tool, then we are not free to play with the forms appearing in the current literature. There are some LOGICAL things behind.
There is a set of axioms by Khinchin (widely accepted).
One of the most important is the test of independent systems. Another point, as I have already shown, there should be no matter if two independent systems are treated (inferred) separately or jointly. All that selects the Shannon form as a unique measure.
Again, we are mixing several things here. An entropy of a distribution has nothing to do with the laws of thermodynamics. Therefore, I do not understand why you are linking the Shannon (information) entropy and the laws of thermodynamics. For this reason I cannot see why to look for an entropy measure that leads to a given distribution with a given constraint.
Before going further let us clarify this point.
With best regards
Eduard
-----Message d'origine-----
De : Alexandre Wang
Envoyé : mercredi 1 avril 2009 18:02
À : 'Eduard VAKARIN'
Objet : RE:
Dear Eduard,
I am very happy to be able to discuss with you all these aspects.
I agree with you to say that all the known functionals can be used to mesure the "spread" of distribution. But you know, there are other things in the uncertainty of prob than the "spread". I have the conviction that for a given distribution, there should be amoung the possible forms a "correct" and best one which in addition maximizes itself only for this prob distribution. Apart from this maxent criterion, I am searching for other proofs for saying that all the known forms are not equivalent for measuring probabilistic uncertainty. We try to do this with some special distribution of complex systems. We can discuss that one day.
Suppose a small system in equilibrium whose prob distribution of energy is measured for different temperatures. Now I want to know its entropy (& its dependence on temperature, pressure etc) satidfying 1er & 2nd law. Shannon entropy is false for this case, because it cannot give us correct thermodynamic relations with non exponential distributions. The safest way to do this is to use the 1st&2nd laws, i.e., dS=(dE-dW)/T=(1/T)*sum_iE_idp_i (varentropy). A reason for calculating entropy when the prob is known.
I will leave for China next week for about two weeks. I will continue to think about your question why we search for a measure functional when the prob is already known. This is a very good question.
More later
Amicalement
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : lundi 30 mars 2009 18:29 À : Alexandre Wang Objet : RE:
Bonsoir Alexandre,
Sorry for the late response. I was thinking on the things you explained.
I have still some problems in following your logics. Then main point is why should we serach for an entropy measure when the distributions is already given?
As I have briefly shown you, your variational formulation can be deduced from the MaxEnt procedure even without specifying the entropy measure. I agree, this issue could be interesting.
In order to proceed further we should find a common point of view on the problem formulation. Again, if the distribution is known, why to search for the entropy functional that gives this distribution after maximization?
> The idea of the variational definition of informational entropy
> originated from two questions. The first is how to measure correctly
> the uncertainty of a given probability distribution of some random
> variables. The widely accepted measure is as you know the Shannon
> formula which is already questionable as a universal measure of
> probabilistic uncertainty. The second is whether or not the
> "correct" uncertainty measure can be maximized for the corresponding
> distribution.
What do you mean "correct"? Imagine you have an object. Depending on how you are going to use it, you may measure it differently: weight, color, price,...
This is a bit similar to what we have discussed. If we consider the entropy as a measure of the distribution "spread", then we are free to use practically any of the known forms.
Another story if we are dealing with inference. In this case there is also some logics behind the entropy choice.
But this is another story. As I told you, I cannot understand why should be search for the entropy functional when the distribution is already known?
With my best wishes.
Eduard
-----Message d'origine-----
De : Alexandre Wang
Envoyé : mardi 17 mars 2009 14:55
À : 'Eduard VAKARIN'
Objet : RE:
Cher Eduard,
I am very glad to hear from you.
The idea of the variational definition of informational entropy originated from two questions. The first is how to measure correctly the uncertainty of a given probability distribution of some random variables. The widely accepted measure is as you know the Shannon formula which is already questionable as a universal measure of probabilistic uncertainty. The second is whether or not the "correct" uncertainty measure can be maximized for the corresponding distribution.
The above two questions can be made into one: to find a measure which always gives the maximal value of uncertainty for any distribution. In this sense, this measure is an optimal one to make sure that nothing is omitted in using it (words of Jaynes supporting maxent). It is not the case with Shannon formula if the probability is not exponential.
Indeed, along this line, the maxent as a tool of assigning probability does not have any meaning, since, as you wrote, the probability is already knwon. Maxent is only a criterion here. As did Abe in his paper about the entropy which maximizes any distribution as he said.
The variational definition is finally the same thing as that of Abe. My question is that, without posing maxent as a condition of definition, why the S defined by dS=\eta(d<x>-<dx>) is maximizable. There must be something behind the difference (d<x>-<dx>). We discussed that since the proposition of varentropy. But we don't see the why. If I don't make mistake, the answer to this question is also an answer to the long standing question why the second law entropy is maximized at equilibrium. Can you think about this and give me your opinion?
One of the utility of varentropy dS=\eta(d<x>-<dx>) is that it can be used for nonequilibrium system in which the random variable x is for example temperature, presure or mass gradient. In this case, S is not but can be related to the second law entropy if a process takes place between two equilibrium states. We wrote papers about that as you know.
Another achivement with varentropy is that the maximum of S can be justified from mecahnics principles such as virtual work and least action principles, in the sense we discussed last time with Jean-Pierre.
A disadvantage of the variational definition is that for many systems in non stable evolution during which we only know the temperary probability over a limited state points, S cannot be calculated. A group of students of ISMANS wanted to calculate S for Arnold cat mapping. But they failed before the system reaches equilibrium.
Another question is the form of S for constant probability distribution. With Aziz, we found a solution. As a matter of fact, that solution implies that the Boltzmann formula lnW is the best and simplest entropy for p=1/W.
More later
Have a good day
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : vendredi 13 mars 2009 16:25 À : Alexandre Wang Objet : RE:
Bonjour Alexandre,
J'ai lu tes notes. Il y a des questions intéressantes. Je vais passer en anglais par ce que ça peut être utile pour les discussions sur un texte commun.
So, the main problem that troubles me is why we are looking for a functional form of the entropy when the distribution is already given?
Obviously we can overcome the technical points and to merge your form of variational entropy with what we have discussed before my departure. But I think we should "justify" the problem.
If I know the distribution then I can do whatever I want: taking averages...
Why should I search for the entropy functional that gives me this distribution?
Cordialement
Eduard
--------------------------------------------------------------------------------
De : jpba...@numericable.com [mailto:jpba...@numericable.com]
Envoyé : lundi 11 mai 2009 11:14
À :
Dear colleagues,
I am pleased to be accepted in this group discussion. Thanks Alexandre to organize this group.
I find the comments by S. Abe very clear and, to my opinion they represent a good starting point for the discussion between complex systems and thermodynamics.
In the middle of the last century, just before world war two, a big problem was : can you describe the critical behavior -i.e. essentially the existence of power laws - starting from the standard thermodynamics? Onsager gave an answer by solving exactly the Ising model. The answer is "Yes we can !" ( a very famous sentence today) I do not know if this is related to the discussion but due to the Kubo's theory -which is a linear response theory- it is possible to calculate some quantities - the diffusion coeffiecient, for instance, with the help of time correlation functions (the velocity-velocity in the case of diffusion). Of course it was first tempting to assume that such functions depend exponentially on time. Later more precise calculations and simulations based on molecular dynamics have shown that the decreasing of the correlation function was not exponential but exhibits for long time a dependence like time power -d/2 where d is the dimension of space. This is another example where the usual statistical thermodynamics may generate naturally some power laws. Another example concerns the ionic solutions for which it is possible to show that standard calculations lead to quantities that are not analytical functions of the ionic density. This is a consequence of coulombic potential which is long ranged.
Finally, if we start from usual statiscal mechanics we can recover the usual thermodynamics in the so called thermodynamic limit. In some cases this limit is obtained using the Stirling formula restricted to its first term. But if the system is not infinitely large we have to keep the next term in the Stirling formula. Immediatly this leads to non traditional results but issue from traditional ones. I am not a specialist but this seems in the spirit of Tsallis.
These few examples show that usual statistical mechanics can lead to the existence of power laws.
Now, for me the question that appears is how to define a complex system? If we work with systems having nothing to do with thermodynamics why entropy should be an efficient concept to start a totally new approach ?
Friendly.
J.P. Badiali
-----Message d'origine-----
De : su...@sf6.so-net.ne.jp [mailto:su...@sf6.so-net.ne.jp] Envoyé : jeudi 7 mai 2009 19:14 À :
Dear Ladies and Gentlemen,
First of all, I am pleased very much to get involved with your discussion forum.
Actually, I have already learned some things from your discussions.
It seems to me that the main issue at the moment is concerned with generalized entropies in connection with non-exponential distributions.
The situation may actually be more complicated than what we naively imagine.
It is my opinion that the point is if we really wish to connect these discussions with thermodynamics. Here, "thermodynamics" means not only equilibrium one but also stationary nonequilibrium. A question is if there exists stationary statistical mechanical theory that can describe systems in nonequilibrium stationary states.
In this respect, we can go back to Paul and Tatiana Ehrenfest, who have asked the following question already in 1912: Is there some measure like the entropy, which achieves its extremal value in a nonequilibrium stationary state?
In general, so-called complex systems in nonequilibrium stationary states stay in such states for a very long time, much longer than their typical microscopic times scales. This leads one to imagine that such states may maximize a certain measure such as the entropy.
I should accept that a naive approach based on generalized entropies is profoundly inconsistent with the thermodynamic principles, actually. Therefore, the current attempts such as Tsallis' q-statistics are still very unclear.
However, at the same time, still I can imagine that there will a way to go beyond the exact equilibrium theory in order to describe complex systems with broken ergodicity/mixing and nontrivial phase-space structures.
Yours sincerely,
Sumiyoshi Abe
-----Message d'origine-----
De : Alexandre Wang
Envoyé : jeudi 7 mai 2009 19:11
À : '
Objet : group discussion of generic entropy
Dear All,
First let me welcome warmly Sonia, Stefan, Rudolf and Lu Yi who join us in the group discussion.
Dr. Sonia R. Bentes is from the Instituto Superior de Contabilidade e Administração de Lisboa
Dr. Stefan Thurner and Dr. Rudolf Habel are from the Complex Systems Research Group Medical University of Vienna
Dr. Lu Yi is from the Normal University of Baotou, Inner Mongolia (my country)
Several people asked me about the origin of this discussion and some precision on the discussed questions.
The origin of the discussion is a personel note I made about several generalized entropies. You can see a part of it in the attached file. Obviously, the underlying idea in these tentatives of generalization surpasses the previous idea of generalizing the Shannon form with such and such given functional forms (Renyi, Tsallis or others).
The question is why we do that, especially when the probability is already known (my friend Eduard harassed me with this question :-<). In the discussion, Eduard also put forward other questions and insertions as you can see in the discussion to date.
Please read the note which is attached and, if you wish, send your remarks to everybody in the mailing list.
Bon weekend
Alexandre
-----Message d'origine-----
De : Alexandre Wang
Envoyé : mardi 5 mai 2009 15:48
À :
Dear all,
First let's welcome Antonio to our group. Dr. Antonio Scarfone is from Politechnico di Torino, Italy.
Thanks to Eduard for the last remarks. For people who don't have access to PRE, the paper of Eduard mentioned in his message can be downloaded via arXiv:cond-mat/0604573
Other papers related directly to this furum are the following (among others which can be found in the references of these papers)
-S. Abe, J. Physics A: Math. Gen. {\bf 36}(2003)8733 (arXiv:cond-mat/0211437)
-S. Abe, G. Kaniadakis and A. M. Scarfone. J. Phys. A: Math. Gen. {\bf 37}(2004)10513 (arXiv:cond-mat/0401290)
-S. Thurner and Rudolf Hanel, Entropies for complex systems: generalized-generalized entropies, AIP conference proceedings, {\bf 956}(2007)68-75 (arXiv:0709.1685)
-Q.A. Wang, Probability distribution and entropy as a measure of uncertainty, J. Phys. A:
Math. Theor., {\bf 41}(2008)065004 (arXiv:cond-mat/0612076)
- Stefan Thurner and Rudolf Hanel, Maximum entropy approach to central limit distributions of correlated variables, arXiv:0804.3477
You can add references into this list.
I am thinking about establishing a website with restricted access which would be more agreeable to write, read and easier to keep for future project (we can think about a book in which each one summarizes his contribution). But I don't know yet what is the best approach to the creation of this site. If someone has experience or know how to do that, please give me an instruction by sending a message to me. Thanks.
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : mardi 28 avril 2009 14:50 À : Alexandre Wang Cc : Alain Le Mehauté; Aziz El Kaabouchi; François Tsobnang; Laurent Nivanen; jpba...@numericable.com; jcchen; Congjie Ou 欧聪杰; gzsu; Li Wei; Jian Jiang; xc...@mail.ccnu.edu.cn; paul lescot; Giorgio Kaniadakis; su...@sf6.so-net.ne.jp; Dmitrii Tayurskii; YuraLysogorskiy; Zvezdov Denis; Raoul; wr0124; jiulin du; yilu...@163.com; QUARATI PIERO; Tongling Lin Objet : Re: TR:
Dear all,
I think it is a good idea to launch a collective discussion.
I am ready to participate.
Now, on the latest portion of the discussion
> -----Message d'origine-----
> De : Alexandre Wang
> Envoyé : vendredi 24 avril 2009 18:13
> À : 'Eduard VAKARIN'
> Objet : RE:
>
> Dear Eduard,
> I have just come back from China. I saw your message in China. But I
> didn't have time there for the discussion.
>
> I see that you are very attached to the referential information
> theory based on Shannon-Khinchin hypotheses. I understand.
I am not very attached. Just I belive the logics behind and do not see why I have to "invent" an alternative entropy measure which (in addition) contains unknown parameters, or is even non-additive for independent systems.
If one cannot obtain a non-exponential distribution from the Shannon form with a linear constraint, then this means that the physics behind is more complex.
I mean we have to find an appropriate constraint leading to the result of interest. I have tried to do that in a recent paper (Phys. Rev. E 74, 036120 (2006)).
> My intention is not to mixe the entropy of thermodynamics and the
> entropy in information theory. I don't know whether or not there is
> relationship between these two entropies, although there are already
> some hints such as Landauer's erasure principle.
Exactly. This principle really hints that there is a link between the two entropies, but they are not identical.
> So what I have tried to do is to look at the thermodynamic entropy,
> to do something similar to what is done in information theory, that
> is to search for the functional form of thermodynamic entropy, to
> prove the maxent from other physics principles and to find the way
> to include non exponential probability in this framework. The first
> part of this work is for equilibrium system. This is the reason why
> I talked about thermodynamic laws.
Thermodynamic entropy is a state function S(N,V,T), it is not a functional.
The thermodynamic laws tell us what what we can do with it. So I do not think that we are free "to do something similar...". Otherwise we are out of the thermodynamics. That will be something else.
> I am not so sure but it seems to me that for this purpose one is
> obliged to overcome the Shannon formula if it is used as functionals
> of physics entropy, since with non exponential probability there is
> contradiction between Shannon formula and thermodynamic laws :
> dS=\beta*(d<E>-<dE>).
Again, thermodynamic entropy is not a functional. The Shannon (information) entropy is not (in general) a thermodynamic object, so I do not see any contradiction with thermodynamic laws. The probability distributions (exponential or non-exponential) have no direct relation with the thermodynamics. As I already told, it is possible to get a non-exponential distribution from the Shannon form, but the constraint should be non-linear.
In addition, this illuminates the fhysics behind: a non-exponential distribution (in energy, for instance) appears as a result of a restricted phase space (the thermodynamic entropy is constrained). The latter points out towards an non-Gibbsian physics (either finite systems, or stationary non-equilibrium states).
>
> As you have noticed, this is a contrary approach to inference
> theory. Each time when I talk about entropy, it is the physics
> entropy but not the informational entropy for inference. I think I
> have proved the maxent property for this physics entropy from other
> principles. The maxent of inference theory is not concerned in my
> idea.
If it is a "fhysical entropy" in the thermodynamic meaning, then one has to clarify how the probability distributions enter there.
I would understand a formulation of the "varentropy" in a probabilistic meaning. That is as an attempt to find some general features without resorting to a specific functional form for the information entropy.
> The word "correct measure" is also related to physics entropy. I
> cannot yet give you convincing reason for this notion. What I can
> say id that when I think about this word, the question I asked
> myself is the following: Suppose a given prob distribution
> (exponential for example) of a physical system whose uncertainty is
> calculated by using two functionals (with the same unity): Shannon
> formula and, say, the formula derived for stretched expontial
> distribution (Abe, Plastino). You will obtain two values for the
> uncertainty. You may say that the two values are both correct. If
> the state or the distribution of the system does not change, this
> conclusion would be Ok.
Yes, I agree. In order to measure the uncertainty one may use any convenient estimator.
> Now the exponential distribution, after an evolution to another
> equilibrium state, becomes streched exponential one. You may be
> interested in the (physical!) entropy variation.
First of all how is it possible to drive (in equilibrium) a system between two EQUILIBRIUM states with different distributions? If such a situation takes place, then this immediately suggests that something had happend with your system.
> The two formula will give two different variations. But I know that
> the variation of the physical entropy (of second law) can be uniquely
> determined for this given process. So what formula should be used if
> we want to calculate the variation through the entropy functional? I
> think that the correct measure (functional) should give the uniquely
> determined correct variation.
Again, an entropy calculated for a given distribution (whatever the distribution and the information entropy forms) and the one entering the second law are two different objects. It is not evident that starting from a given information functional we will be able to built up a scheme similar to the standard thermodynamics.
With best wishes
Eduard
______________________________________________
De : Alexandre Wang
Envoyé : lundi 27 avril 2009 18:30
À : Alain Le Mehauté; Aziz El Kaabouchi; François Tsobnang; Laurent Nivanen; 'jpba...@numericable.com'; 'jcchen'; 'Congjie Ou 欧聪杰'; 'gzsu'; 'Li Wei'; Jian Jiang; 'xc...@mail.ccnu.edu.cn'; 'paul lescot'; 'Giorgio Kaniadakis'; 'su...@sf6.so-net.ne.jp'; 'Dmitrii Tayurskii'; 'Yura Lysogorskiy'; 'Zvezdov Denis'; 'Raoul'; 'wr0124'; 'jiulin du'; 'yilu...@163.com'; 'QUARATI PIERO'; 'Tongling Lin'
Cc : 'Eduard VAKARIN'
Objet : TR:
Dear all, I would like to inform you of this discussion started recently with a good physicist and also a good friend, Dr. E. Vakarin, who asked me some crucial (and deep) questions about a recent entropic approach to complex system (proposed by Abe, Kaniadakis et al, and ismans scientists a bit later). I would like to invite everybody, if you are interested, to participate in this discussion for the simple reason that group discussion is much more enriching than two person discussing secretly and mysteriously :-), and that I do not know every answer to the present questions and (I hope) the coming ones
If you want to discuss, please click "Reply to all" (Repondre a tous). Please just let me know if someone does not want to receive discussions from the group.
For the PhD and graduate students in this group: if you have specific technical questions about some basic notions or needs of references, please do not send message by "reply to all". Just send a message to me, then I can ask the right person to answer your questions if I cannot do it. Thanks
Welcome to Eduard to our small (and complex) network.
Alexandre
-----Message d'origine-----
De : Alexandre Wang
Envoyé : vendredi 24 avril 2009 18:13
À : 'Eduard VAKARIN'
Objet : RE:
Dear Eduard,
I have just come back from China. I saw your message in China. But I didn't have time there for the discussion.
I see that you are very attached to the referential information theory based on Shannon-Khinchin hypotheses. I understand.
My intention is not to mixe the entropy of thermodynamics and the entropy in information theory. I don't know whether or not there is relationship between these two entropies, although there are already some hints such as Landauer's erasure principle.
So what I have tried to do is to look at the thermodynamic entropy, to do something similar to what is done in information theory, that is to search for the functional form of thermodynamic entropy, to prove the maxent from other physics principles and to find the way to include non exponential probability in this framework. The first part of this work is for equilibrium system. This is the reason why I talked about thermodynamic laws.
I am not so sure but it seems to me that for this purpose one is obliged to overcome the Shannon formula if it is used as functionals of physics entropy, since with non exponential probability there is contradiction between Shannon formula and thermodynamic laws : dS=\beta*(d<E>-<dE>).
As you have noticed, this is a contrary approach to inference theory. Each time when I talk about entropy, it is the physics entropy but not the informational entropy for inference. I think I have proved the maxent property for this physics entropy from other principles. The maxent of inference theory is not concerned in my idea.
The word "correct measure" is also related to physics entropy. I cannot yet give you convincing reason for this notion. What I can say id that when I think about this word, the question I asked myself is the following: Suppose a given prob distribution (exponential for example) of a physical system whose uncertainty is calculated by using two functionals (with the same unity): Shannon formula and, say, the formula derived for stretched expontial distribution (Abe, Plastino). You will obtain two values for the uncertainty. You may say that the two values are both correct. If the state or the distribution of the system does not change, this conclusion would be Ok.
Now the exponential distribution, after an evolution to another equilibrium state, becomes streched exponential one. You may be interested in the (physical!) entropy variation. The two formula will give two different variations. But I know that the variation of the physical entropy (of second law) can be uniquely determined for this given process. So what formula should be used if we want to calculate the variation through the entropy functional? I think that the correct measure (functional) should give the uniquely determined correct variation.
Have I answered your questions?
Have a nice weekend
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : jeudi 16 avril 2009 12:48 À : Alexandre Wang Objet : RE:
Dear Alexandre,
I hope you are safely back from China.
Let us continue our discussion.
First of all, what do you mean under a "correct" and the best measure?
As we have already discussed, if it is simply a "spread" measure, then we may use whatever "convenient" estimation. On the other hand if we look at the entropy as the inference tool, then we are not free to play with the forms appearing in the current literature. There are some LOGICAL things behind.
There is a set of axioms by Khinchin (widely accepted).
One of the most important is the test of independent systems. Another point, as I have already shown, there should be no matter if two independent systems are treated (inferred) separately or jointly. All that selects the Shannon form as a unique measure.
Again, we are mixing several things here. An entropy of a distribution has nothing to do with the laws of thermodynamics. Therefore, I do not understand why you are linking the Shannon (information) entropy and the laws of thermodynamics. For this reason I cannot see why to look for an entropy measure that leads to a given distribution with a given constraint.
Before going further let us clarify this point.
With best regards
Eduard
-----Message d'origine-----
De : Alexandre Wang
Envoyé : mercredi 1 avril 2009 18:02
À : 'Eduard VAKARIN'
Objet : RE:
Dear Eduard,
I am very happy to be able to discuss with you all these aspects.
I agree with you to say that all the known functionals can be used to mesure the "spread" of distribution. But you know, there are other things in the uncertainty of prob than the "spread". I have the conviction that for a given distribution, there should be amoung the possible forms a "correct" and best one which in addition maximizes itself only for this prob distribution. Apart from this maxent criterion, I am searching for other proofs for saying that all the known forms are not equivalent for measuring probabilistic uncertainty. We try to do this with some special distribution of complex systems. We can discuss that one day.
Suppose a small system in equilibrium whose prob distribution of energy is measured for different temperatures. Now I want to know its entropy (& its dependence on temperature, pressure etc) satidfying 1er & 2nd law. Shannon entropy is false for this case, because it cannot give us correct thermodynamic relations with non exponential distributions. The safest way to do this is to use the 1st&2nd laws, i.e., dS=(dE-dW)/T=(1/T)*sum_iE_idp_i (varentropy). A reason for calculating entropy when the prob is known.
I will leave for China next week for about two weeks. I will continue to think about your question why we search for a measure functional when the prob is already known. This is a very good question.
More later
Amicalement
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : lundi 30 mars 2009 18:29 À : Alexandre Wang Objet : RE:
Bonsoir Alexandre,
Sorry for the late response. I was thinking on the things you explained.
I have still some problems in following your logics. Then main point is why should we serach for an entropy measure when the distributions is already given?
As I have briefly shown you, your variational formulation can be deduced from the MaxEnt procedure even without specifying the entropy measure. I agree, this issue could be interesting.
In order to proceed further we should find a common point of view on the problem formulation. Again, if the distribution is known, why to search for the entropy functional that gives this distribution after maximization?
> The idea of the variational definition of informational entropy
> originated from two questions. The first is how to measure correctly
> the uncertainty of a given probability distribution of some random
> variables. The widely accepted measure is as you know the Shannon
> formula which is already questionable as a universal measure of
> probabilistic uncertainty. The second is whether or not the
> "correct" uncertainty measure can be maximized for the corresponding
> distribution.
What do you mean "correct"? Imagine you have an object. Depending on how you are going to use it, you may measure it differently: weight, color, price,...
This is a bit similar to what we have discussed. If we consider the entropy as a measure of the distribution "spread", then we are free to use practically any of the known forms.
Another story if we are dealing with inference. In this case there is also some logics behind the entropy choice.
But this is another story. As I told you, I cannot understand why should be search for the entropy functional when the distribution is already known?
With my best wishes.
Eduard
-----Message d'origine-----
De : Alexandre Wang
Envoyé : mardi 17 mars 2009 14:55
À : 'Eduard VAKARIN'
Objet : RE:
Cher Eduard,
I am very glad to hear from you.
The idea of the variational definition of informational entropy originated from two questions. The first is how to measure correctly the uncertainty of a given probability distribution of some random variables. The widely accepted measure is as you know the Shannon formula which is already questionable as a universal measure of probabilistic uncertainty. The second is whether or not the "correct" uncertainty measure can be maximized for the corresponding distribution.
The above two questions can be made into one: to find a measure which always gives the maximal value of uncertainty for any distribution. In this sense, this measure is an optimal one to make sure that nothing is omitted in using it (words of Jaynes supporting maxent). It is not the case with Shannon formula if the probability is not exponential.
Indeed, along this line, the maxent as a tool of assigning probability does not have any meaning, since, as you wrote, the probability is already knwon. Maxent is only a criterion here. As did Abe in his paper about the entropy which maximizes any distribution as he said.
The variational definition is finally the same thing as that of Abe. My question is that, without posing maxent as a condition of definition, why the S defined by dS=\eta(d<x>-<dx>) is maximizable. There must be something behind the difference (d<x>-<dx>). We discussed that since the proposition of varentropy. But we don't see the why. If I don't make mistake, the answer to this question is also an answer to the long standing question why the second law entropy is maximized at equilibrium. Can you think about this and give me your opinion?
One of the utility of varentropy dS=\eta(d<x>-<dx>) is that it can be used for nonequilibrium system in which the random variable x is for example temperature, presure or mass gradient. In this case, S is not but can be related to the second law entropy if a process takes place between two equilibrium states. We wrote papers about that as you know.
Another achivement with varentropy is that the maximum of S can be justified from mecahnics principles such as virtual work and least action principles, in the sense we discussed last time with Jean-Pierre.
A disadvantage of the variational definition is that for many systems in non stable evolution during which we only know the temperary probability over a limited state points, S cannot be calculated. A group of students of ISMANS wanted to calculate S for Arnold cat mapping. But they failed before the system reaches equilibrium.
Another question is the form of S for constant probability distribution. With Aziz, we found a solution. As a matter of fact, that solution implies that the Boltzmann formula lnW is the best and simplest entropy for p=1/W.
More later
Have a good day
Alexandre
-----Message d'origine-----
De : Eduard VAKARIN [mailto:eduard....@upmc.fr] Envoyé : vendredi 13 mars 2009 16:25 À : Alexandre Wang Objet : RE:
Bonjour Alexandre,
J'ai lu tes notes. Il y a des questions intéressantes. Je vais passer en anglais par ce que ça peut être utile pour les discussions sur un texte commun.
So, the main problem that troubles me is why we are looking for a functional form of the entropy when the distribution is already given?
Obviously we can overcome the technical points and to merge your form of variational entropy with what we have discussed before my departure. But I think we should "justify" the problem.
If I know the distribution then I can do whatever I want: taking averages...
Why should I search for the entropy functional that gives me this distribution?
Cordialement
Eduard
Reply all
Reply to author
Forward
0 new messages