entropy
Eduard VAKARIN
I agree with what Professor Abe says. Namely, until now there is no
clear relation
between the system dynamics and its statistics (one has to clarify
what kind of statistics is meant) in the case of chaotic, non-ergodic,
non-hamiltonian systems.
But I think there are some misleading things behind. On my opinion
whatever the system is considered all the details known (chaotic,
non-ergodic, non-hamiltonian, equilibrium, nonequilibrium, etc) come
as constraints for the
inference procedure. Therefore, I cannot see why to construct
different forms of statistical entropy for different distributions
(moreover if the distribution is given).
With best regards
Eduard Vakarin
Alexandre Wang
If I don't make mistake, Eduard (Vakarin) has repeated that question 5
times: why to generalize entropy! The "generalizists" to try to answer
it.
Several participants have made interesting remarks on the links between
dynamical and statistical mechanics, and between complexity and
generalization of statistics theory. Many points in the remarks merit
further discussion.
Let us come back to entropy. Eduard repeated a question that the
"generalizists" never mention seriously in their publication to my
knowledge. This gaves the impression that the "generalizism" was
something of obvious and granted for everybody, and that the only
question was how to do it.
On the contrary, the "anti-generalizists" do not answer the
"anti-question" neither: why not to generalize entropy? They advocate
for the generality of either Boltzmann entropy (like Professor Dieter
Gross, see for example arXiv:nucl-th/0603028) or Shannon formula by
specific applications or derivation of nonexponential probability
distributions with additional hypotheses (the first physicist doing that
to my best knowledge was Myron Tribus in his book Rational Descriptions,
Decisions and Designs. Pergamon Press Inc.(1969) where he derived for
example lognormal distribution using Shannon entropy and specific
constraints.
I will continue this message a bit later.
Q. A. Wang
Raoul
But I think you can invite to this discussion Prof. Zaripov R.G.
Recently he published his books "New Meausures and Methods in Information Theory", "Self-organization and Irreversibility in Nonextensive Systems"
He considered the properties of all available entropies: Shannon-Wiener, Renyi-Harvda, Charvat-Darocsy and etc. Unfortunately, his books were published in Russian. But he has publications written in English related to this subject that you are in the state of discussion. Probably his comments will be very useful for you.
I am giving his E-mail for contacts with him: zar...@mail.knc.ru
With regards, Prof. Raoul Nigmatullin.
Alexandre Wang пишет:
Eduard VAKARIN
I think it is good that Alexandre (Wang) tries to direct the dicussion
in a constructive way. I agree with his suggestions. More specifically:
Quoting Alexandre Wang <aw...@ismans.fr>:
> If I don't make mistake, Eduard (Vakarin) has repeated that question 5
> times: why to generalize entropy! The "generalizists" to try to answer
> it.
It is indeed interesting and important (at least for me). I think that
answering this question in a simple way we can understand a lot of
things and avoid many misleadings.
> Several participants have made interesting remarks on the links between
> dynamical and statistical mechanics, and between complexity and
> generalization of statistics theory. Many points in the remarks merit
> further discussion.
I agree. A link between the dynamics and statistics is an important issue.
There problems and interesting questions, but on my opinion it is not
directly related to the entropy problem we are discussing.
> Let us come back to entropy. Eduard repeated a question that the
> "generalizists" never mention seriously in their publication to my
> knowledge. This gaves the impression that the "generalizism" was
> something of obvious and granted for everybody, and that the only
> question was how to do it.
Agree again. But to answer the question "how to do it" it is necessary
to have in mind an idea on "why we are doing this". Of course there
are some arguments behind the generalization trials. But for me it
looks like attempts to "deform" the standard scheme and... maybe
something will come out.
> On the contrary, the "anti-generalizists" do not answer the
> "anti-question" neither: why not to generalize entropy?
For me the simplest (maybe stupid) answer is the Occam's razor principle.
On should not change his mind unless some "objective" things force him
to do that. Imagine we have a "zoo" of entropy measures. What can we
learn from that?
Being applied to a system two entropy functionals will give different results.
Which one should we believe?
> They advocate
> for the generality of either Boltzmann entropy (like Professor Dieter
> Gross, see for example arXiv:nucl-th/0603028) or Shannon formula by
> specific applications or derivation of nonexponential probability
> distributions with additional hypotheses (the first physicist doing that
> to my best knowledge was Myron Tribus in his book Rational Descriptions,
> Decisions and Designs. Pergamon Press Inc.(1969) where he derived for
> example lognormal distribution using Shannon entropy and specific
> constraints.
On my opinion there is no even reason to advocate Shannon. As I
already told, it comes from a set of clear and widely accepted axioms
(by Khinchin for instance) related to the concept of a probability
distribution. Thus there two ways:
1) The Shannon form is not a unique solution, satisfying these axioms.
2) We do not believe or want to modify this well-established axiomatics.
In both cases we can try to find a more general solution. But again,
this should be a constructive way. For the moment (it is my opinion)
we are trying to test several possible "generalizations" with the
"couriosity" to see what comes out.
It would be very interesting form me to find the books by R.G. Zaripov
(if they exist in some electronic format). Reading in russian is not a
problem for me
(I mean to say I am a native speaker)
With best wishes
Eduard Vakarin
Alexandre Wang
Dear all,
The silence of our group for the last month implies that each one is reflecting seriously I think. I know that Eduard is enjoying the native book about generalized statistics. I would like to continue my postings I started on May 29th.
I wrote last time that the "generalizists" and the "anti-generalizists" of entropy have to answer the pros and cons of the generalization (to make something different for either entropy functional or maxent constraints). I have the impression that hese two modifications, if proposed only aximatically as a priori truth without justification from facts or physical laws, are logically parallel ones - no one superior than another.
The modification of maxent constraints, as a axiom, needs justification. For example, using <c*ln x> to maximize Shannon entropy to get power law as Tribus did in his 1969 book seems to me something droping from the sky (by the way, I have a question, does anyone among you knows some facts or laws disqualifying the arithmetic mean?).
Keeping the arithmetic mean for maxent and replacing, axiomatically, Shannon entropy with other functionals, we can get nonexponential probability. But this is, no more no less, a parallel approach to the constraint modification.
Keeping the arithmetic mean for maxent, one can also get nonexponential prob through other hypothetical approach such as superstatistics. The actual superstatistics idea is an (interesting) theoretical hypothesis. The approach is surely intriguing and promising. But philosophically, this is a construction using hypothesis over hypothesis. It would be more powerful and convincing if the hypotheses are motivated by facts and laws.
I actually do not see convincing arguments solving this dilemma (entropy or arithmetic mean). But I am rather inclined to the modification of entropy or of information and prefer using arithmetic mean generally for any statistical calculus.
I have another reason for that.
I tried in the last several years to keep the fundamental principles of the Newtonian mechanics and to extend them to random dynamics, in order to see whether or not this is a path to thermodynamics from mechanics. A character of random dynamics is that the uniqueness of path between two states is lost. So the virtual work principle can no more be used for each path since this would lead to the same Newtonian equation for different paths. I was obliged to use the average vitrtual work dW for a given moment of time.
This work turns out to dW=dS-\gamma*dE where gamma is constant, E is energy (of the system), dS=sum_(over i, all states of that moment)E_i*dp_i, and p_i is the probability distribution of the states.
Hence dS=d<E> - <dE>. This is the origine of the idea that S might be a measure of the randomness of that dynamics. If we boldly say that this "uncertain mechanics" is useful to thermodynamic system, S can be proven to be the thermodynamic entropy if the system is in thermal equilibrium at the moment of this analysis.
This is perhaps not a solid reason for not using Shannon entropy. I only see here a hint that other definitions may be more conveniently useful (Shannon formula is a special case here).
I am looking forward to the opinion of every body.
Best wishes
Q A Wang
su...@sf6.so-net.ne.jp
Prof. Wang's writing makes me feel AGAIN that actually we do not know
what constraints we should impose when we maximize entropy (Boltzmann-
Gibbs-Shannon, Tsallis, or others). My explicit question is why we impose
the constraint on the average energy, not e.g. the logarithm of the energy.
As far as I know, even Jaynes did not answer this question. Without clarifying
this issue, the maximum entropy method remains highly arbitrary. This point
naturally leads to the following question: which costs more, changing entropy
or changing constraints?
It is my opinion that this kind of seemingly ridiculous questions could remain
as long as we blindly follow and generalize the maximum entropy method WITHOUT
physical basis.
These days, I was reading a book entitled "Entropy and the Time Evolution of Macroscopic
Systems" by W. T. Grandy, Jr., who is generally supposed to be a great follower
of E. T. Jaynes. Unfortunately, he too does not reflect enough the problem of constraints.
Qiuping, you should do it, maybe!?!?
All the best,
Sumiyoshi Abe
Eduard VAKARIN
I think that Alexandre (Wang) really tries to put the discussion in a
"constructive" way. I will try to make my comments.
Quoting Alexandre Wang <aw...@ismans.fr>:
> Dear all,
>
> The silence of our group for the last month implies that each one is
> reflecting seriously I think. I know that Eduard is enjoying the native
> book about generalized statistics.
That is really a good review of what has been done, including possible
generalizations of the entropy measure. Unfortunately, the question on
why should we do all that is not answered.
> I wrote last time that the "generalizists" and the "anti-generalizists"
> of entropy have to answer the pros and cons of the generalization (to
> make something different for either entropy functional or maxent
> constraints). I have the impression that hese two modifications, if
> proposed only aximatically as a priori truth without justification from
> facts or physical laws, are logically parallel ones - no one superior
> than another.
One can imagine a lot of possible generalizations. So if we have a
"zoo" of entropy functionals, what can we learn from that? Which form
should we use in a given problem? Why one of the generalizations is
better then others and what is the criterium for their comparison.
That is simply a general (logical) problem on the path of
generalization (or generalization over generalization).
On my opinion, if we are trying to develop an inference tool (like
MAxEnt), then we should come to an "universal" definition of the
entropy functional. Something like we have a Hamiltonian or a
Lagrangian in mechanics (or statistical mechanics). Then different
problems come with different constraints.
> The modification of maxent constraints, as a axiom, needs justification.
> For example, using <c*ln x> to maximize Shannon entropy to get power law
> as Tribus did in his 1969 book seems to me something droping from the
> sky (by the way, I have a question, does anyone among you knows some
> facts or laws disqualifying the arithmetic mean?).
I am not trying to advocate the Tribus trick. I cannot understand what
does it mean "The modification of maxent constraints". We are not free
to "play" with the constraints. Different constraints correspond to
different physical situations and we cannot blindly replace the
constraint in order to get a desired distribution.
> Keeping the arithmetic mean for maxent and replacing, axiomatically,
> Shannon entropy with other functionals, we can get nonexponential
> probability. But this is, no more no less, a parallel approach to the
> constraint modification.
That is not a question of the mode of averaging. The quiestion is:
which quantity is averaged? I do not think that replacing the
functional is a parallel approach. If with a "right, universal"
entropy functional we cannot obtain a distribution we want (with a
given constraint), then this means that with this physics (encoded in
the constraint) this is impossible. The we have to change the
constraint to see which processes lead to a given (non-exponential)
distribution. If we change the entropy functional, we cannot learn the
physics from this approach. One can imagine that there is always a
functional which will give a desired distribution. But what can we
learn from that?
> Keeping the arithmetic mean for maxent, one can also get nonexponential
> prob through other hypothetical approach such as superstatistics. The
> actual superstatistics idea is an (interesting) theoretical hypothesis.
> The approach is surely intriguing and promising. But philosophically,
> this is a construction using hypothesis over hypothesis. It would be
> more powerful and convincing if the hypotheses are motivated by facts
> and laws.
I agree, there are some assumptions. But they are different from those
accepted in the course of the entropy generalization.
> I actually do not see convincing arguments solving this dilemma (entropy
> or arithmetic mean). But I am rather inclined to the modification of
> entropy or of information and prefer using arithmetic mean generally for
> any statistical calculus.
Again, changing the constrain we switch to a different physical
situation. That is more-less clear. But modifying the entropy
functional, we do not know what we are doing. Then a given
distribution can be obtained rather straightforwardly, but we cannot
tell what is the physics behind.
Briefly, this is my opinion.
Alain Le Mehauté
Find enclosed a new publications from ISMANS just for your information. Thank again for your contribution for advanced idea
Looking forward
ALM
-----Message d'origine-----
De : chate...@googlegroups.com [mailto:chate...@googlegroups.com] De la part de Eduard VAKARIN
Envoyé : vendredi 3 juillet 2009 13:10
À : chate...@googlegroups.com
Objet : [Chatentropy] Re: entropy