Factoring and entropy
Brad
In a dream I had this morning someone told me “Good programming is
about minimizing H – entropy”. So I looked it up and the entropy of a
random variable is called H(X). It seems like a good (maybe profound)
way to view the subject.
Forth programmers (especially those aligned with Chuck) are especially
aggressive about factoring, which is IMHO a way of reducing entropy.
There are other ways. Information hiding reduces I(X) for example.
A brief overview of H(X) of a random variable can be found here:
http://en.wikipedia.org/wiki/Entropy_(information_theory)
Is there any way that the entropy of a program can be measured?
-Brad
Jonah Thomas
Brad <hwf...@gmail.com> wrote:
> Hi All,
>
> In a dream I had this morning someone told me “Good programming is
> about minimizing H – entropy”. So I looked it up and the entropy of a
> random variable is called H(X). It seems like a good (maybe profound)
> way to view the subject.
>
> Forth programmers (especially those aligned with Chuck) are especially
> aggressive about factoring, which is IMHO a way of reducing entropy.
> There are other ways. Information hiding reduces I(X) for example.
I think there's more to it than that. Forth is about not only reducing
the entropy, but also about reducing the information content of each
program to the minimum necessary to get the desired results.
Coos Haak
> Hi All,
>
> In a dream I had this morning someone told me 嚙踝蕭Good programming is
> about minimizing H 嚙碾 entropy嚙踝蕭. So I looked it up and the entropy of a
> random variable is called H(X). It seems like a good (maybe profound)
> way to view the subject.
>
> Forth programmers (especially those aligned with Chuck) are especially
> aggressive about factoring, which is IMHO a way of reducing entropy.
> There are other ways. Information hiding reduces I(X) for example.
>
> A brief overview of H(X) of a random variable can be found here:
> http://en.wikipedia.org/wiki/Entropy_(information_theory)
>
> Is there any way that the entropy of a program can be measured?
>
Sorry, my chemical background shines through, but H is enthalpy and S
is entropy ;-)
I (information) is the negative of the increase of the entropy S. The
information of a complex program is greater than one written by the
next chimp. Total chaos has the lowest information and the highest
entropy.
--
Coos
CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html
Andreas
>> about minimizing H � entropy�. So I looked it up and the entropy of a
>> random variable is called H(X). It seems like a good (maybe profound)
>> way to view the subject.
>>
>> Forth programmers (especially those aligned with Chuck) are especially
>> aggressive about factoring, which is IMHO a way of reducing entropy.
>> There are other ways. Information hiding reduces I(X) for example.
>
> I think there's more to it than that. Forth is about not only reducing
> the entropy, but also about reducing the information content of each
> program to the minimum necessary to get the desired results.
Incomprehensibility was always an inherent feature of a good Forth program.
ironically :o)
Bernd Paysan
> Incomprehensibility was always an inherent feature of a good Forth
> program.
According to Shannon, a signal with minimal entropy is indistinguishable
from noise - yet it is the most compact way to express this particular
information. No wonder incomprehensibility comes with compactness.
--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/
humptydumpty
Hi!
A interesting and enjoying paper is R.J. Brown's 'Viva la spaghetti!':
http://www.elilabs.com/~rj/spagetti/index.html
Have a nice reading!
humptydumpty
P.M.Lawrence
Have a look at http://en.wikipedia.org/wiki/Chaitin%27s_constant.
There's a good discussion of the thinking behind it in the book "Five
Golden Rules", if you can find it. Chaitin likes to use Lisp for his
worked examples and research in the area, for reasons related to your
musings about Forth here. It made me think of some work I did myself
(at http://users.beagle.com.au/peterl/furphy.html) because the less
user friendly, ultra-simple variant of that has no syntax errors for
any possible source - everything compiles and runs as something, not
that having syntax errors is a fundamental problem. P.M.Lawrence.
MarkWills
It's an interesting theory Brad, but I wonder if we can reduce the
'entropy' in your original post, and 'factor it down' by saying:
"To reduce entropy, reduce complexity." <grin>
;-)
Mark
Krishna Myneni
One can estimate the Shannon entropy of a bit stream, although this
entropy is only precisely defined when the bitstream is infinitely
long. For a Forth program which does this, see
ftp://ccreweb.org/software/kforth/examples/shannon.4th
However, I think what you are getting at is to compare two programs
with identical functionality to see which is the better program. I
don't think the Shannon entropy will help you with that --- an
experienced Forth programmer would be a better estimator.
Nevertheless, you might first try to precisely define the term,
"entropy of a program". In physics and information theory, entropy has
a precise definition.
Krishna
jacko
using E() from stats as expected E(X) = mean. The entropy in things
such as a program relate to frequency of word occurrence, but this may
be phrased as stats from many thousands of programs or just within the
context of the one program.
I do know quite a bit about entropy, and am quite happy to publish the
vhdl code for the bit fridge. Some will get the reference to
thermodynamics, and total local system entropy. The general idea is to
assign a length of sample opportunity to a bit of a certain value.
This has the effect of increasing the presence of the zero bit state,
while reducing the presence of the one bit state. This biases or
offsets the sample statistics to reduce the entropy of a bit stream.
Given a low enough entropy of a biased random bit stream, a modulation
coding exists which works as a recoverable error correction coding.
The error being the minority bit state, and the information stream
being all of the majority bit state, but at a lower symbol rate.
cheers jacko.
-- (C)2008 K Ring Technologies Semiconductor
-- BSD http://nibz.googlecode.com (no support)
-- Comercial licencing available
-- Tel: +44 796 797 3001 (a real space odessy)
-- Maintained by Simon Jackson, BEng.
-- E-mail: jackokr...@gmail.com
-- Hardware support kodek for nibz
library ieee;
-- library altera;
use ieee.std_logic_1164.all;
use ieee.std_logic_arith.all;
entity k is
port (
do : buffer std_Logic;
di : in std_logic;
busy : buffer std_logic;
comde : std_logic;
clk : in std_logic;
-- a rising edge signal (run)
run : in std_logic;
-- for setting the prbs
-- may be of use as a general prbs ...
load : in std_logic_vector(15 downto 0);
save : buffer std_logic_vector(15 downto 0);
rw : in std_logic;
sel : in std_logic
);
end entity;
architecture rtl of k is
signal asymproc : std_logic;
signal which : std_logic;
type s is (action,motion);
signal state : s;
-- intermediates
signal wind : std_logic_vector(4 downto 0);
signal procp : std_logic_vector(1 downto 0);
signal tapx : std_logic;
signal ends : std_logic;
-- process control flags
signal procdo : std_logic;
signal winddo : std_logic;
signal modo : std_logic;
signal runlast : std_logic;
signal whenok : std_logic;
signal tmp : std_logic;
constant tap : natural := 3;
signal pmux : std_logic_vector(15 downto 0);
begin
process(clk)
-- prbs
begin
end process;
procp <= pmux(1 downto 0);
procdo <= procp(1) and procp(0);
-- set motion velocities via probability switching
modo <= procdo xor which xor do xor '1';
-- set sampling window
wind <= pmux(7 downto 3);
winddo <= wind(4) and wind(3) and wind(2) and
wind(1) and wind(0);
process(clk)
-- k state machine
begin
if(rising_edge(clk)) then
runlast <= run;
end if;
if(rising_edge(clk) and busy = '1') then
case state is
when motion =>
asymproc <= asymproc xor modo;
state <= action;
when action =>
if(asymproc = '0') then
if(which = '0') then
-- sample
if(winddo =
'1') then
busy
<= '0';
--
sample do
end if;
else
-- random
do <= do xor
procp(0);
end if;
else
which <= which xor
procdo;
end if;
state <= motion;
end case;
-- NB. if 2nd rising edge run before busy zero
-- then misses a radix slot
elsif(rising_edge(clk) and whenok = '1' and busy =
'0') then
-- set busy, cleared by kodek finish bit
busy <= '1';
whenok <= '0';
do <= tmp;
end if;
if(rising_edge(clk) and runlast = '0' and run = '1')
then
whenok <= '1';
tmp <= di;
end if;
-- prbs
if(comde = '1') then
ends <= save(15);
else
ends <= save(0);
end if;
tapx <= save(tap) xor ends;
if(rising_edge(clk) and busy = '1') then
if(comde = '1') then
save <= save(14 downto 0)&tapx;
else
save <= tapx&save(15 downto 1);
end if;
end if;
if(comde = '1') then
pmux <= save;
else
pmux <= tapx&save(15 downto 1);
end if;
-- load
if(rising_edge(clk) and sel = '1' and rw = '0') then
-- NB. a load while busy can be a random
number gen
-- but not much use in general operation
-- do a kind of reset
asymproc <= '0'; -- rand/samp side
which <= '0'; -- samp side
state <= motion; -- perform motion methods
first
-- (begin and end on them)
-- assign essentials
save <= load;
end if;
end process;
end rtl;
Krishna Myneni
Actually, white noise has maximal entropy. It also has the greatest
information content, from an information theory viewpoint. Consider a
bit sequence of arbitrary size consisting of all 1's or all 0's. The
Shannon entropy of such a sequence is zero. Similarly, a bit pattern
of any length which repeats ad infinitum has a Shannon entropy of
zero. There is virtually no information content in such a pattern,
because one symbol can be used to transmit an arbitrarily large
sequence of data.
A good way to assess the information content of a file is to use a
data compression program and determine the compression ratio. A file
with a large compression ratio has low information content, while a
file that is incompressible has maximum information content.
Krishna
Bernd Paysan
> A good way to assess the information content of a file is to use a
> data compression program and determine the compression ratio. A file
> with a large compression ratio has low information content, while a
> file that is incompressible has maximum information content.
Well, the other possibility of being incompressible is that it is only
noise, and no actual content. You can use data compression to evaluate a
random number generator - if lzma doesn't find anything to squeeze out, the
random number generator can't be that bad. You can turn data with low
information content into uncompressible noise by encrypting it.
Example:
> ll wurstkessel.*.lzma
-rw-r--r-- 1 berndp asic 7615 4. Dez 10:24 wurstkessel.fs.lzma
-rw-r--r-- 1 berndp asic 28104 4. Dez 10:25 wurstkessel.wurst.lzma
The first file is my wurstkessel cryptographic algorithm, compressed with
lzma. The second one is the first encrypted with a 512 bit key, and then
compressed - it is slightly longer than the compressed file itself, since it
contains the lzma overhead. The amount of real information inside is
basically the same plus the 512 bit key, (plus algorithm, and data used
(which is another 2kB of irreducible randomness) - but the algorithm and
data is already in the file ;-).
This is also the reason why audio or image data is so difficult to compress
"lossless" - there's way too much noise in it, and unlike a lossy encoder,
you just can't compress noise into "noise floor between 500 and 550Hz at
-50dB".
Albert van der Horst
Krishna Myneni <krishna...@ccreweb.org> wrote:
<SNIP>
>
>A good way to assess the information content of a file is to use a
>data compression program and determine the compression ratio. A file
>with a large compression ratio has low information content, while a
>file that is incompressible has maximum information content.
A conversion from highly repetitive code to well factored code
may be quite similar to this compression process.
>
>Krishna
Groetjes Albert
--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
Krishna Myneni
> Krishna Myneni wrote:
> > A good way to assess the information content of a file is to use a
> > data compression program and determine the compression ratio. A file
> > with a large compression ratio has low information content, while a
> > file that is incompressible has maximum information content.
>
> Well, the other possibility of being incompressible is that it is only
Assume I had a theoretically perfect data compression program, and
that I used such a program to compress a large file, which to me has
"actual content". I send you the compressed file, and if I gave you no
other information whatsoever, such as the compression algorithm, you
might easily conclude that the file contained only noise bits. That
doesn't mean there is no information content in the compressed file. A
large block of text, for example, which was maximally compressed would
appear as nothing more than a noise pattern. The distinction between
actual content and "only noise" is artificial from the viewpoint of
information content.
If we are making a sampling measurement such as an audio signal, the
noise pattern contains some information about the environment and
fidelity of the data acquisition system. For example, the noise
pattern may consist of discrete frequencies, such as line noise and
harmonics, in which case it may be easily filtered. Data compression
should remove the redundant information. The incompressible portion
contains the extra information associated with other noise sources.
> You can use data compression to evaluate a
> random number generator - if lzma doesn't find anything to squeeze out, the
> random number generator can't be that bad. You can turn data with low
> information content into uncompressible noise by encrypting it.
>
> Example:
>
> > ll wurstkessel.*.lzma
>
> -rw-r--r-- 1 berndp asic 7615 4. Dez 10:24 wurstkessel.fs.lzma
> -rw-r--r-- 1 berndp asic 28104 4. Dez 10:25 wurstkessel.wurst.lzma
>
> The first file is my wurstkessel cryptographic algorithm, compressed with
> lzma. The second one is the first encrypted with a 512 bit key, and then
> compressed - it is slightly longer than the compressed file itself, since it
> contains the lzma overhead. The amount of real information inside is
> basically the same plus the 512 bit key, (plus algorithm, and data used
> (which is another 2kB of irreducible randomness) - but the algorithm and
> data is already in the file ;-).
>
> This is also the reason why audio or image data is so difficult to compress
> "lossless" - there's way too much noise in it, and unlike a lossy encoder,
> you just can't compress noise into "noise floor between 500 and 550Hz at
> -50dB".
>
> --
> Bernd Paysan
> "If you want it done right, you have to do it yourself"http://www.jwdt.com/~paysan/
Krishna
Krishna Myneni
wrote:
> In article <b168e907-8f61-4cc0-af00-f6543d83b...@n35g2000yqm.googlegroups.com>,
> Krishna Myneni <krishna.myn...@ccreweb.org> wrote:
> <SNIP>
>
>
>
> >A good way to assess the information content of a file is to use a
> >data compression program and determine the compression ratio. A file
> >with a large compression ratio has low information content, while a
> >file that is incompressible has maximum information content.
>
> A conversion from highly repetitive code to well factored code
> may be quite similar to this compression process.
>
>
>
> >Krishna
>
> Groetjes Albert
>
> --
> --
> Albert van der Horst, UTRECHT,THE NETHERLANDS
> Economic growth -- being exponential -- ultimately falters.
> albert@spe&ar&c.xs4all.nl &=nhttp://home.hccnet.nl/a.w.m.van.der.horst
I think so. Factoring repetitive code sequences should reduce the
compressibility of the program, thereby increasing its relative
information content. That should mean the program is easier to
understand by the reader, but whether that's always true or not isn't
clear to me.
Krishna
Marcel Hendrix
> On Dec 4, 3:32 am, Bernd Paysan <bernd.pay...@gmx.de> wrote:
>> Krishna Myneni wrote:
>> > A good way to assess the information content of a file is to use a
>> > data compression program and determine the compression ratio. A file
>> > with a large compression ratio has low information content, while a
>> > file that is incompressible has maximum information content.
>> Well, the other possibility of being incompressible is that it is only
>> noise, and no actual content. ...
> Assume I had a theoretically perfect data compression program, and
> that I used such a program to compress a large file, which to me has
> "actual content". I send you the compressed file, and if I gave you no
> other information whatsoever, such as the compression algorithm, you
> might easily conclude that the file contained only noise bits. That
> doesn't mean there is no information content in the compressed file. A
> large block of text, for example, which was maximally compressed would
> appear as nothing more than a noise pattern. The distinction between
> actual content and "only noise" is artificial from the viewpoint of
> information content.
Right! In slightly other words:
This discussion about "noise" and "information content" is useless without
exact definitions. Take this thought experiment: Using the best algorithm
you can find, encrypt the clf article you're reading now to a bitstream b.
1. Did the information content vanish? (Intuitively) no.
2. Would it be possible to find bitstream b by observing the output
of a good random number generator long enough? (Intuitively) yes.
3. Does the experiment's outcome change when I use a pleasing bit of
noise from an old record instead of the clf article? (Intuitively)
no.
Apparently the fact if something is "just noise" or "valid information"
can not be identified on sight.
There is however a third dimension: the minimum number of bits to describe
anything I want to encode. When I encode the noise generator just at the time
it is generating a perfect sine of 1 Volt at 1 kHz for 1 second, I need only
a few bits to do it. A clf article containing 1000 times the character "a"
would likewise compress to a few bits. You can not say there's no information
in this when the bitpattern occurs at CERN in a LHC experiment or the article
on the front page of the Times of London.
Yet another angle: if we photocopy this clf article on a bad copier and encode
it with 32 bits per pixel at 1600 dpi, does the information content change?
Does " 0 !" in a Forth article contain as much information as "0 !" in a Cobol
program?
The technically more interesting concept seems to be how to quantify
"lossless," and what the cost of lossless transmission is.
-marcel
Albert van der Horst
Krishna Myneni <krishna...@ccreweb.org> wrote:
>On Dec 4, 11:57=A0am, Albert van der Horst <alb...@spenarnc.xs4all.nl>
>wrote:
>> In article <b168e907-8f61-4cc0-af00-f6543d83b...@n35g2000yqm.googlegroups=
>.com>,
>> Krishna Myneni =A0<krishna.myn...@ccreweb.org> wrote:
>> <SNIP>
>>
>>
>>
>> >A good way to assess the information content of a file is to use a
>> >data compression program and determine the compression ratio. A file
>> >with a large compression ratio has low information content, while a
>> >file that is incompressible has maximum information content.
>>
>> A conversion from highly repetitive code to well factored code
>> may be quite similar to this compression process.
>>
>>
>>
>> >Krishna
>>
>> Groetjes Albert
>>
>> --
>> --
>> Albert van der Horst, UTRECHT,THE NETHERLANDS
>> Economic growth -- being exponential -- ultimately falters.
>
>
>I think so. Factoring repetitive code sequences should reduce the
>compressibility of the program, thereby increasing its relative
>information content. That should mean the program is easier to
>understand by the reader, but whether that's always true or not isn't
>clear to me.
Probably not. However the number of errors seems to be one in 5
lines, irrespective of language. This would mean a big win
for compact languages, and compaction. (Mythical Man Month, IIRC).
If a piece of code is used N times, the chance of having a bug
removed by testing is multiplied by approximately N.
Therefore there are few errors in e.g. machine instructions
that are used all the time.
OTOH : scripts that ramble on and on are expected to contain
corners that are never tested.
Jonah Thomas
OK. As a thought experiment, consider all the valid inputs to your
program, and pair them against the valid outputs. Find a way to include
the "don't care" conditions.
That bag of pairs is the information content of the specification for
your program. The order you put the pairs in is presumably not
important.?
And you want a program that wil reproduce all those pairs, that will
give you the desired output for each valid input. That's what it takes
to have a correct program. And one possible ideal for the program beyond
correctness is that out of all the possible correct programs, this
program should have the minimum information content.
A program can have the same shannon information content as a clf article
or RNG output, but so what? The things we want to compare it to are
other correct programs.
Now, suppose that a correct Forth program has a smaller information
content than a correct COBOL program. But suppose that after both
programs are compressed with zip, the COBOL program then has a smaller
total information content and smaller than the original Forth program
too. (I'm going to say that zip only reduces the number of identical
sequences of bits, if that isn't true of zip then replace it with a
compression algorithm that only does that.) Then that says the COBOL
program had more "stupid redundancy", more identical copies of strings
of bits in it. It's larger because of that. But for the total of other
sources of information, it must do better than the Forth program.
Because zip has overhead, and even including the overhead the result is
smaller than the Forth program. So the Forth program must have useless
information that is not just repetition. For example, it might use THEN
half the time and ENDIF half the time. It might use SWAP OVER half the
time and TUCK half the time. That kind of thing. A smarter
Forth-oriented compression algorithm could remove the simpler redundancy
just like a good Forth optimising compiler would.
Should we measure the information content of the source code or should
it be the executable code? Clearly smaller executable code is a good
thing, though not the only thing. Smaller source code is good but there
are other criteria too like readability, which is hard to measure. What
about the information content of the whole system? If you have a
processor which can truly do more, then your executable can be smaller.
You can tell it "do X" instead of having to write the code to do X. But
the actual program is not really simpler, just more of the complexity
has been poshed off somewhere that's easy to overlook. So when we look
at information content, shouldn't we also inclde the information content
of the processor and the compiler?
Jerry Avins
I don't know that it's a good measure. Consider a subroutine that adds
two arbitrary 8-bit numbers. (Subroutine so that we needn't consider I/O
issues.) If we distinguish between [a b] and [b a] there are 2^16 - 2^8
input pairs and 2^9 outputs. That' a lot of "information", far more than
any code that would compute the result. How does the information in a
look-up table compare to that in an algorithm?
Above I wrote "2^16 - 2^8". I think that's more informative than 65,280.
65,280 is more compact (even with the comma) but I dislike magic
numbers. Does the information about where the number came from count?
Jerry
--
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������
Jonah Thomas
> Jonah Thomas wrote:
> > As a thought experiment, consider all the valid inputs to your
> > program, and pair them against the valid outputs. Find a way to
> > include the "don't care" conditions.
> >
> > That bag of pairs is the information content of the specification
> > for your program. The order you put the pairs in is presumably not
> > important.?
> >
> > And you want a program that wil reproduce all those pairs, that will
> > give you the desired output for each valid input. That's what it
> > takes to have a correct program. And one possible ideal for the
> > program beyond correctness is that out of all the possible correct
> > programs, this program should have the minimum information content.
> >
> > A program can have the same shannon information content as a clf
> > article or RNG output, but so what? The things we want to compare it
> > to are other correct programs.
> I don't know that it's a good measure. Consider a subroutine that adds
>
> two arbitrary 8-bit numbers. (Subroutine so that we needn't consider
> I/O issues.) If we distinguish between [a b] and [b a] there are 2^16
> - 2^8 input pairs and 2^9 outputs. That' a lot of "information", far
> more than any code that would compute the result. How does the
> information in a look-up table compare to that in an algorithm?
I figured for a specification, you know the specification is complete
when it provides every valid input paired with every valid output. I'm
interested in information content of programs, but I don't currently
care about information content of specs, I just want to make sure
they're correct.
The information in an algorithm can be much smaller than that in a
lookup table provided that the pattern is reasonably simple and the
table has a lot of entries. The less information there is in the table
the more room there is for an algorithm to do better. Short lookup
tables might do better than algorithms, and arbitrary lookup tables
might be hard to find good algorithms for.
> Above I wrote "2^16 - 2^8". I think that's more informative than
> 65,280.
> 65,280 is more compact (even with the comma) but I dislike magic
> numbers. Does the information about where the number came from count?
Since we're talking about information content, I presented a method to
compare programs based on their information content. I don't think it is
the only important thing. Since I don't care what philosophical insights
a processor gets from its code, I don't at all mind magic numbers in
object code. But a program is better when programmers can easily
understand it, so it's useful for source code to be easily readable.
Sometimes redundancy aids understanding. If there's something you need
to understand and it's presented several different ways, and presented
at each place in the program that might get you to stumble because of
it, that might help. If it's presented only once in a manner that
minimises the information content, that seems obviously harder to read.
I think it's easier to decide how important the information content is
if you actually have a measure. And so I say what matters is not just
the information content but the relative information content among
correct programs that do the same thing. I'm pretty sure it isn't the
only thing that's worth caring about, but I think this is potentially
more useful than the absolute shannon information value compared to FD
articles or musical compositions.
Bruce McFarling
It really should be kept in mind that the "information" in
"information theory" is about information transmission capacity. There
is, in other words, more information in that information theory sense
of the word in a Brittney Spears video than in the article in which
Einstein published the General Theory of Relativity.
So "information content" in an information theoretic quantitative
sense is more about compressibility than about meaningfulness.
And then there is the issue of Absolute and Relative H and S - a well
factored program could well see its relative information to go up as
its absolute information goes down - that is, its a smaller
information packet if fully compressed but a less compressible one.
It may well be that the entropy that good programming is about
minimizing is more the system entropy as a consequence of running the
program, since a well-organized system is a further-from-randomly-
ordered system. Of course, on the thermodynamic dimension total
entropy cannot be reduced overall, but a more efficient process is one
that has a smaller increase in total thermodynamic entropy as a
consequence of the specific decrease in entropy within the boundaries
of the system.
Aleksej Saushev
> It may well be that the entropy that good programming is about
> minimizing is more the system entropy as a consequence of running the
> program, since a well-organized system is a further-from-randomly-
> ordered system. Of course, on the thermodynamic dimension total
> entropy cannot be reduced overall,
How do you know that? Just because Clausius said so?
> but a more efficient process is one
> that has a smaller increase in total thermodynamic entropy as a
> consequence of the specific decrease in entropy within the boundaries
> of the system.
You don't know thermodynamics, if you make such an elementary mistake.
--
HE CE3OH...
Bruce McFarling
> Bruce McFarling <agil...@netscape.net> writes:
> > It may well be that the entropy that good programming is about
> > minimizing is more the system entropy as a consequence of running the
> > program, since a well-organized system is a further-from-randomly-
> > ordered system. Of course, on the thermodynamic dimension total
> > entropy cannot be reduced overall,
> How do you know that? Just because Clausius said so?
If you have a concrete physical process for reducing total entropy, as
opposed to reducing entropy within the boundaries of a system at the
expense of a greater increase in entropy outside the boundaries of
that system, implement it in the form of a perpetual motion machine
and make your millions of dollars/euros/yen etc..
> > but a more efficient process is one
> > that has a smaller increase in total thermodynamic entropy as a
> > consequence of the specific decrease in entropy within the boundaries
> > of the system.
> You don't know thermodynamics, if you make such an elementary mistake.
I believe you have not read sufficiently carefully, or else are using
a definition of "efficient" that is distinct from the normal
engineering output/input.
Aleksej Saushev
> On Dec 6, 7:28О©╫am, Aleksej Saushev <a...@inbox.ru> wrote:
>> Bruce McFarling <agil...@netscape.net> writes:
>> > It may well be that the entropy that good programming is about
>> > minimizing is more the system entropy as a consequence of running the
>> > program, since a well-organized system is a further-from-randomly-
>> > ordered system. Of course, on the thermodynamic dimension total
>> > entropy cannot be reduced overall,
>
>> How do you know that? Just because Clausius said so?
>
> If you have a concrete physical process for reducing total entropy, as
> opposed to reducing entropy within the boundaries of a system at the
> expense of a greater increase in entropy outside the boundaries of
> that system, implement it in the form of a perpetual motion machine
> and make your millions of dollars/euros/yen etc..
You don't know anything about "total" entropy, since everything
thermodynamics refers to is bounded systems. "Total" entropy may
decrease and it's out of scope of consideration in thermodynamics.
>> > but a more efficient process is one
>> > that has a smaller increase in total thermodynamic entropy as a
>> > consequence of the specific decrease in entropy within the boundaries
>> > of the system.
>
>> You don't know thermodynamics, if you make such an elementary mistake.
>
> I believe you have not read sufficiently carefully, or else are using
> a definition of "efficient" that is distinct from the normal
> engineering output/input.
In industrial thermodynamics the measure of process efficiency is
relative exergy, it isn't entropy. If entropy were useful to perform
needed analysis, there would be no need to introduce another quantity.
Entropy itself isn't useful in practice, you use derived quantities
almost exclusively, free energy or enthalpy depending on if it is work
or heat what you need in result.
--
HE CE3OH...
jacko
> Bruce McFarling <agi...@netscape.net> writes:
>
> > On Dec 6, 7:28О©╫am, Aleksej Saushev <a...@inbox.ru> wrote:
> >> Bruce McFarling <agil...@netscape.net> writes:
> >> > It may well be that the entropy that good programming is about
> >> > minimizing is more the system entropy as a consequence of running the
> >> > program, since a well-organized system is a further-from-randomly-
> >> > ordered system. Of course, on the thermodynamic dimension total
> >> > entropy cannot be reduced overall,
> >
> >> How do you know that? Just because Clausius said so?
> >
> > If you have a concrete physical process for reducing total entropy, as
> > opposed to reducing entropy within the boundaries of a system at the
> > expense of a greater increase in entropy outside the boundaries of
> > that system, implement it in the form of a perpetual motion machine
> > and make your millions of dollars/euros/yen etc..
>
> You don't know anything about "total" entropy, since everything
> thermodynamics refers to is bounded systems. "Total" entropy may
> decrease and it's out of scope of consideration in thermodynamics.
>
hi folks. total entropy in a physical sense will by many orders of
magnitude most likely increase. if high entropy causes universe heat
death then slowing its rate of increase is efficient in the well lived
sense. I think universal entropy was what total refered to.... ....
and a few more words for one less nano second or less.....
> >> > but a more efficient process is one
> >> > that has a smaller increase in total thermodynamic entropy as a
> >> > consequence of the specific decrease in entropy within the boundaries
> >> > of the system.
> >
> >> You don't know thermodynamics, if you make such an elementary mistake.
> >
> > I believe you have not read sufficiently carefully, or else are using
> > a definition of "efficient" that is distinct from the normal
> > engineering output/input.
>
> In industrial thermodynamics the measure of process efficiency is
> relative exergy, it isn't entropy. If entropy were useful to perform
> needed analysis, there would be no need to introduce another quantity.
>
> Entropy itself isn't useful in practice, you use derived quantities
> almost exclusively, free energy or enthalpy depending on if it is work
> or heat what you need in result.
>
>
> --
> HE CE3OH...
in information entropy the maximum in 1 bit per bit.. so ... and bit
refridgeration does not even have to move to the sink. even if it
did...how does it ever exceed 1. does it reflect around 1 or just melt
into noop space??
cheers jacko
Bruce McFarling
> > I believe you have not read sufficiently carefully, or else are using
> > a definition of "efficient" that is distinct from the normal
> > engineering output/input.
> In industrial thermodynamics the measure of process efficiency is
> relative exergy, it isn't entropy. If entropy were useful to perform
> needed analysis, there would be no need to introduce another quantity.
See, as I said, you have not read sufficiently carefully. Saying that
an entropy reduction within an open system is at the expense of a
greater entropy increase outside that system does not imply that
entropy would be used to *measure* that efficiency, whether in an
industrial process or an ecosystem.