Compression by descent
Ashley Labowitz
I've been programming a technique that allows most, if not all files
to be reduced in size. It is done using cyclic roots and high-degree
polynomials where the total information content of the coefficients is
smaller (sometimes much smaller) than the original text.
The FAQ has been pointed out to me where it "proves" that not all
files can be compressed by any algorithm, but it seems to me that it
has some mistakes. They aren't serious, but they are oversights that
prevent it from applying to fractal bits, permutations and sorting the
bytes of the original file.
The gist of this algorithm is that when certain polynomials are
iterated over the complex plain (e.g., like the Mandelbrot set),
information can be encoded in the "ragged" edges of the curve. I'm
afraid this may have been done before, and I don't want to waste any
work. Does it ring a bell to anyone?
If someone would like to look at my pseudocode, I would be grateful.
CheeriO,
Ash
Phil Carmody
> Good day everyone,
Well it was a good day...
> I've been programming a technique that allows most, if not all files
> to be reduced in size.
You have not.
> It is done using cyclic roots and high-degree
> polynomials where the total information content of the coefficients is
> smaller (sometimes much smaller) than the original text.
No it's not.
> The FAQ has been pointed out to me where it "proves"
It's not just *called* a proof. It *is* a proof. If you think it's
not a proof, then you are a loon.
> that not all
> files can be compressed by any algorithm, but it seems to me that it
> has some mistakes.
Ergo you are a loon.
> They aren't serious, but they are oversights that
> prevent it from applying to fractal bits, permutations and sorting the
> bytes of the original file.
>
> The gist of this algorithm is that when certain polynomials are
> iterated over the complex plain (e.g., like the Mandelbrot set),
> information can be encoded in the "ragged" edges of the curve. I'm
> afraid this may have been done before, and I don't want to waste any
> work. Does it ring a bell to anyone?
Yes. I call it "compression by coincidence". You're about the
thousandth loon to pretend its got any merit at all. You won't
be the last.
> If someone would like to look at my pseudocode, I would be grateful.
I wouldn't.
Phil
--
Dear aunt, let's set so double the killer delete select all.
-- Microsoft voice recognition live demonstration
Ashley Labowitz
Phil, you are jumping to conclusions. It turns out that if you place
the bytes to be compressed on a square grid and then iterate a complex
polynomial you can show the fractal edges will naturally follow a
significant number of bytes, allowing those to be removed from the
original entropy. This is the secret to it working for _all_ files,
not just some special form. I know the mathematical establishment is
slow to embrace change (and wisely so), but Phil, please don't be
ridiculous.
The difficult part is extracting the roots of high-order polynomials,
but this can be done classically using Newton's method adapted for the
complex plane (I'm working on an extension for quarternions, but
Python is not my strength :( ). Obviously, fractals only carry as
much information as is in the coefficients of their generating
polynomial, but you might be assuming that they carry infinite
information. Let me know if this is your mistake.
Here's the twist, fractional bits can be extracted from the _exact_
(or nearly so) value of the roots of the polynomials. This is the
chief error of the Pigeonhold Principle "proof" of impossibilty. Of
course, the Pigeonhold Principle is very useful in general, but it is
a mistake to apply it to non-integer sets such as fractional bits.
Any data generated from a deterministic process has a Kolmogorov
Complexity that must be lower than the size of the data as the number
of bits to model goes to infinity. However, in practice it is
impossible to find this optimal Turing machine in general. However,
the set of strings that are stochastic or contain purely random
elements are themselves bound to fractal models.
But it is not necessary for you to agree, I've shown the proof to a
number of people and none have succeeded in finding any mistakes in
it. I'm not doing it for the money, but to help the world.
Best,
Ash
Fulcrum
> Good day everyone,
>
> I've been programming a technique that allows most, if not all files
> to be reduced in size. It is done using cyclic roots and high-degree
> polynomials where the total information content of the coefficients is
> smaller (sometimes much smaller) than the original text.
One of the better posts this year!.
Phil's post i mean... ;)
Fulcrum
are stochastic or contain purely random elements are themselves bound
Ashley Labowitz
challenge to conventional thinking without taking the easy way out.
industr...@hotmail.com
wrote:> I have not [deepthroated myself today.]
Yes you have.
> No [my dick isn't a long 12-inch flaccid spaghetti cock].
Yes it is.
> I'm not just *called* a [hairy fatso]. I *am* a fat fascist fucker. If you think i'm
> not, then you are a loon.
I won't argue with you there.
> Ergo I am a loon.
Yup.
> Yes. I call it "wise trolling". You're about the
> thousandth loon to pretend my trolling has any merit at all. You won't
> be the last.
>
>
> I wouldn't [take my tylenol without spreading margarine over it.].
>
> Phil
> --
> Dear aunt, let's set so double the killer delete select all.
> -- Microsoft voice recognition live demonstration
Two words: fuck you.
Ashley Labowitz
> On Aug 10, 5:23 pm, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
> wrote:
I don't think I understand what you mean industrial_...@hotmail.com
but you are being unfair to Phil.
I think it's best if people would evaluate my ideas on their merits.
You can show fractal convergence using criteria known for decades and
at that point, enough accuracy has been achieved. My implementation
uses 1 nybble for the main coefficient, and builds from that using a
doubly-linked list. Each block is stored using offsets as follows.
This is from the README file I'll be distributing with my source
code. I apologize but fixed-width format is needed to properly read
it.
+---+---+---+---+---+---+---+---+---+---+
| SEQUENCE |HD | COEF | BYTE | (ctd.)
+---+---+---+---+---+---+---+---+---+---+
| MAIN NUMBER | POWER EXP | MODE | (ctd.)
+---+---+---+---+---+---+---+---+---+---+
| TY| X | Y | Z | TIME | PERF | PRIME |
+---+---+---+---+---+---+---+---+---+---+
With this key:
HD = head
TY = type
X, Y, Z = codes for which roots of -1
TIME = timestamp
PERF = performance characteristic to optimize
PRIME = base of polynomial (not Galois)
The high-precision root extractions can be done in 33 lines of
assembly, or so it seems: I don't have the resources to test this as
well as I'd like. One other way to model it is to use a network flow
system (or a shortest distance one, which is easily solved with
Dijkstra's algorithm). I hope you guys wouldn't disagree with
Dijkstra's algorithm... :)
Cheerio,
Ash
industr...@hotmail.com
> On Aug 10, 8:01 pm, industrial_...@hotmail.com wrote:
>
> > On Aug 10, 5:23 pm, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
> > wrote:
>
> I don't think I understand what you mean industrial_...@hotmail.com
> but you are being unfair to Phil.
He was obviously trolling, and I was in a "wanna-see-blue-blood-bleed-
red" mood so I think I officially clunked his ass outta this thread
where he can regurgitate his bullshit on some other forum if he has
nothing to add worth reading to the topic.
To be fair, your idea is kinda vague to me, I don't get the exact
mechanics of your algorithm. Put it in a nutshell: how do you expect
to compress *most* of all possible files? By how much? Are there any
limits, if so, what?
> I think it's best if people would evaluate my ideas on their merits.
> You can show fractal convergence using criteria known for decades and
> at that point, enough accuracy has been achieved. My implementation
> uses 1 nybble for the main coefficient, and builds from that using a
> doubly-linked list. Each block is stored using offsets as follows.
> This is from the README file I'll be distributing with my source
> code. I apologize but fixed-width format is needed to properly read
> it.
Each block contains part of the target file, right?
> +---+---+---+---+---+---+---+---+---+---+
> | SEQUENCE |HD | COEF | BYTE | (ctd.)
> +---+---+---+---+---+---+---+---+---+---+
> | MAIN NUMBER | POWER EXP | MODE | (ctd.)
> +---+---+---+---+---+---+---+---+---+---+
> | TY| X | Y | Z | TIME | PERF | PRIME |
> +---+---+---+---+---+---+---+---+---+---+
>
> With this key:
> HD = head
> TY = type
> X, Y, Z = codes for which roots of -1
> TIME = timestamp
> PERF = performance characteristic to optimize
> PRIME = base of polynomial (not Galois)
>
> The high-precision root extractions can be done in 33 lines of
> assembly, or so it seems: I don't have the resources to test this as
> well as I'd like. One other way to model it is to use a network flow
> system (or a shortest distance one, which is easily solved with
> Dijkstra's algorithm). I hope you guys wouldn't disagree with
> Dijkstra's algorithm... :)
>
> Cheerio,
> Ash
If you really have the source code, I guess you can work on your idea
with more experienced peoples around this newsgroup, but I doubt
you're covering any new ground (not saying this to be nasty but unless
you can specifically prove any flaws in the counting argument, I don't
think yer gonna get anywhere.) How do you enumerate 4 combos with only
2? A.K.A: How do you compress 00 01 10 or 11 to 0 or 1 (compress all 2
bits to 1) When 0 or 1 can only represent 2 out of the 4 possible
combos?
My theory around this (see posts from 6 months ago) was based on the
nature of "random-appearing data" and that it should have some
distinct feature that "normal, practical" data doesn't and thus should
be able to be comrpessed. But then I realized that "random-appearing
data" makes about 99.99% of all possible files and that the data we
see everyday is a painfully tiny portion (0.0001%) so logically it's
impossible.
The only remaining possible way out of this would be to store data as
"time ticks" where one tick would be an extremely tiny fragment of
time (e.g. 10^-30th of a second.) Simon Jackson I heard is working on
this.
But I'm up for a civilized discussion about your idea, but I request
that you explain with eloquence and simplicity in mind since math
isn't my strength (ironic, isn't it?)
Phil Carmody
> Hello again,
>
> Phil, you are jumping to conclusions. It turns out that if you place
> the bytes to be compressed on a square grid and then iterate a complex
> polynomial you can show the fractal edges will naturally follow a
> significant number of bytes, allowing those to be removed from the
> original entropy. This is the secret to it working for _all_ files,
> not just some special form.
You are a crank, a loon, a babbling drunk baboon.
> I'm not doing it for the money, but to help the world.
Help comp.compression by turning your computer off.
industr...@hotmail.com
wrote:
Shut the fuck up, faggot. Did you take a break from your cock-sucking
sessions? Go resume it, and let's hope you'll choke on your own cum.
Ashley Labowitz
of the list and interferes with constructive discussion such as that
of universal fractal compression. Name-calling in particular makes us
all look juvenile. I see from the archives that this list has been
very professional in the past, and I'm sure it can regain that if we
can tolerate new ideas. Thank you in advance.
Ash
On Aug 11, 4:02 am, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
wrote:
Ashley Labowitz
After using the fractal approximations it is necessary to fill out the
full data square. This is akin to a knapsack problem with very
irregular objects, but some overlap can be tolerated.
Completely random data must have an irrational fractal dimension (well-
known property of the logarithm), which is sufficient to reconstruct
the steps that built the fractal. I've tried the Julia set, the
Menger Sponge and others by Sierpinski. Once the proper sequence of
fractals is selected, I can show that a file containing random data,
one full bit of entropy per bit, can be compressed by about 12%. I
know it is not much, but it is something, and this includes the space
needed for the fractal iteration information and other block metadata.
Before I become famous, I want to always remember the people that
helped me get where I am. I'm lucky that I've been able to reuse lots
of existing software and not spend time reinventing the wheel. To
give credit where it is due: I used Kirby Urner python fractal code
and also got some ideas from Erik Wrenholt and also some of the
research done at Seattle University.
I've been working hard all day and soon I will have finished the
implementation. can anyone suggest file types that would be most
interesting to test on?
industrial_...@hotmail.com, to answer your question, the compression
does not work on files of a trivially small size. The square has to
be at least 6 x 6 (that is, 36 bytes) before space is saved from all
files and probably at least 1K before 12% is seen. If many small
pieces of data need to be compressed, they can simply be concatenated
with some sort of delimiter or serialized data structure. Does that
make sense?
CheeriO,
Ash
Phil Carmody
> Phil, please don't make personal attacks. It's bad for the atmosphere
> of the list and interferes with constructive discussion such as that
> of universal fractal compression. Name-calling in particular makes us
> all look juvenile. I see from the archives that this list has been
> very professional in the past, and I'm sure it can regain that if we
> can tolerate new ideas. Thank you in advance.
They aren't new ideas. They are isomorphic to every other
scheme of 'compression by coincidence' that we've seen.
You are ignorant of simple mathematical facts, such as the
Kraft inequality. You display a complete ignorance of the
field.
If you had come into the group humbly, the I might have been
more charitable, but you entered stating absolutes, and
absolute bollocks at that, so you got the reception that you
did.
We've seen your type so often, that I for one simply have no
patience. Maybe others will waste their time with you, some
do that kind of thing. However, I'm glad that you've noticed
that I consider you to be a crank devoid of any knowledge or
skills in the field, and that that can be recorded for posterity
in the various usenet archives, as that means my job is done.
You can never claim that your bogus nonsense got a positive
reception from those who know what they are talking about.
Well, you probably can, as you're prepared to lie about
mathematical facts, so I guess you'll be prepared to lie about
pretty much anything.
I shall now gladly decrease your audience by one as, as I said,
I've run out of patience. You have no intention to learn, you
are beyond hope.
*PLONK*
Ashley Labowitz
Thank you for stopping the name-calling. If we can't agree on
anything else, at least something good for comp.compression has come
from this discussion.
Sincerely best wishes,
Ash
collec...@googlemail.com
paper over REAL LIFE FUCKING TESTS.
Your pseudo-code is a piece of crap.
You haven't even tested it. And I can tell you, NEVER believe anything
unless it's TESTED.
I blame the academic wankers who refuse to let physics students play
with real physical objects but instead make them do equations on paper
instead.
Your test will never work.
More likely you'll blame everything but yourself in your inability to
even generate a test.
The saddest thing about you, is that you throw around complex words
that you don't even understand, in an effort to make yourself seem
more intelligent. Yet you don't understand basic logic! Even pigeons
in their pigeonholes know more than you ;)
Jim Leonard
> industrial_...@hotmail.com, to answer your question, the compression
> does not work on files of a trivially small size. The square has to
> be at least 6 x 6 (that is, 36 bytes) before space is saved from all
> files and probably at least 1K before 12% is seen.
It is this specific claim, "space is saved from all files", that we
have an issue with. There is no compressor that can compress every
single 36-byte file by at least one bit.
I suggest you read http://marknelson.us/2006/06/20/million-digit-challenge/
and specifically try to compress 'AMillionRandomDigits.bin'. Matt
Mahoney has conjectured that it might be possible to compress this
file by up to 50 bits due to the way the data was assembled. So,
perform the following:
1. Compress this file using your process
2. Decompress your resulting file into a 3rd file
3. Compare the 3rd file with the original to confirm complete lossless
decompression
We await your test results.
Phil Carmody
> On Aug 11, 4:27 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:
> > industrial_...@hotmail.com, to answer your question, the compression
> > does not work on files of a trivially small size. The square has to
> > be at least 6 x 6 (that is, 36 bytes) before space is saved from all
> > files and probably at least 1K before 12% is seen.
>
> It is this specific claim, "space is saved from all files", that we
> have an issue with. There is no compressor that can compress every
> single 36-byte file by at least one bit.
Stronger - by at least zero bits, with at least one file being
compressed by at least one bit.
But for all practical purposes (such as attempting to communicate
simple ideas to the denser members of the population) your version
works.
There's an even simpler wording (that again is loose), namely
'for every file compressed, two must expand'.
collec...@googlemail.com
wrote:
> Jim Leonard <MobyGa...@gmail.com> writes:
> > On Aug 11, 4:27 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:
> > > industrial_...@hotmail.com, to answer your question, the compression
> > > does not work on files of a trivially small size. The square has to
> > > be at least 6 x 6 (that is, 36 bytes) before space is saved from all
> > > files and probably at least 1K before 12% is seen.
>
> > It is this specific claim, "space is saved from all files", that we
> > have an issue with. There is no compressor that can compress every
> > single 36-byte file by at least one bit.
>
> Stronger - by at least zero bits, with at least one file being
> compressed by at least one bit.
>
> But for all practical purposes (such as attempting to communicate
> simple ideas to the denser members of the population) your version
> works.
>
> There's an even simpler wording (that again is loose), namely
> 'for every file compressed, two must expand'.
It's obvious that compression of random files is impossible.
Only compressing patterns is possible.
If only people would try to ways to identify more patterns, and
represent these patterns with smaller "words", then people would
generate better compression.
But then I suppose PAQ is quite close to the extreme end of
compressionability anyhow? It must have a good output syntax at least.
I don't think that the "offset+length" syntax is so tight. Dictionary
based compressors should be able to acheive more compression, in
certain cases.
Ashley Labowitz
matter is that if you have a universal compressor, it is it's own
proof. It seems a little anti-intellectual to assert that all ideas
shouldn't be able to stand on their own and that one is necessarily
better than another. I would just like to have a honest scientific
discussion, the way comp.compression was intended to operate.
As a matter of fact, I've almost finished the coding. It went fast as
a result of using high-level string and fractal packages, which means
there will be some need for optimization later. If anyone has some
short test cases for me to run please send them my way. My email is
sporbzip...@gmail.com (remove my favorite compression program to
get the address :) ).
collec...@googlemail.com
> Thanks guys, I do appreciate that point of view. But the fact of the
> matter is that if you have a universal compressor, it is it's own
> proof. It seems a little anti-intellectual to assert that all ideas
> shouldn't be able to stand on their own and that one is necessarily
> better than another. I would just like to have a honest scientific
> discussion, the way comp.compression was intended to operate.
>
> As a matter of fact, I've almost finished the coding. It went fast as
> a result of using high-level string and fractal packages, which means
> there will be some need for optimization later. If anyone has some
> short test cases for me to run please send them my way.
Just use any good source of random data?
On linux, /dev/random will do I think. http://en.wikipedia.org/wiki//dev/random
Otherwise, you could just try using data generated from C++, via
rand(). Make sure you seed the random number with the current time,
the time (in as high resolution as you can acheive, microseconds
usually), is almost always a good source of randomness. This explains
how: http://www.daniweb.com/forums/thread1769.html
collec...@googlemail.com
less -f /dev/random
collec...@googlemail.com
> Works on my Mac as well :))))))
>
> less -f /dev/random
Note, I do not expect this fellow to actually compress it. But I am
doing all I can to reduce his pain.
The sooner he finds out he cannot compress the truely random, the
better for his emotional and mental well being. So assisting him by by
pointing him to a good source of random data is being a benefit to his
well being.
collec...@googlemail.com
schemes.
That is to see, by how much on average does your compressor EXPAND the
random data.
A good compressor, will only expand random data by a tiny amount. :)
Perhaps 4 bytes or so per gigabyte. That can be acheived with good
escape codes. You'd have to fit a header, a length, and lots of other
stuff in there though, so it's hard to not expand most random data by
at least 16 bytes or so minimum.
A BAD algorithm, could expand the random data many times over!
When I get around to writing my own compressor, I'll probably use
random data, just to test the efficiency of my algorithm. This will be
just to make sure my algorithm doesn't screw up the data by inflating
the random data to 4x it's natural size, lol. You can never be too
sure that you don't make the most basic of mistakes. Even the most
brilliant of geniuses make the most basic of mistakes from time to
time. That's why we never let our guard up against checking ourselves
for the most basic of mistakes.
collec...@googlemail.com
> On Aug 11, 4:27 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:
>
> > industrial_...@hotmail.com, to answer your question, the compression
> > does not work on files of a trivially small size. The square has to
> > be at least 6 x 6 (that is, 36 bytes) before space is saved from all
> > files and probably at least 1K before 12% is seen.
>
> It is this specific claim, "space is saved from all files", that we
> have an issue with. There is no compressor that can compress every
> single 36-byte file by at least one bit.
>
> and specifically try to compress 'AMillionRandomDigits.bin'. Matt
> Mahoney has conjectured that it might be possible to compress this
> file by up to 50 bits due to the way the data was assembled. So,
> perform the following:
>
> 1. Compress this file using your process
> 2. Decompress your resulting file into a 3rd file
> 3. Compare the 3rd file with the original to confirm complete lossless
> decompression
>
> We await your test results.
Or just recompress his compressed file :)
Ashley never told us if his compresed files can also be compressed. I
wonder what his answer to that one is?
Maybe if he is clever enough he can recursively compress his data down
to 1 byte :) tihihihihihi!!!
Phil Carmody
> On Aug 12, 9:42 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:
> > Thanks guys, I do appreciate that point of view. But the fact of the
> > matter is that if you have a universal compressor, it is it's own
> > proof. It seems a little anti-intellectual to assert that all ideas
> > shouldn't be able to stand on their own and that one is necessarily
> > better than another. I would just like to have a honest scientific
> > discussion, the way comp.compression was intended to operate.
> >
> > As a matter of fact, I've almost finished the coding. It went fast as
> > a result of using high-level string and fractal packages, which means
> > there will be some need for optimization later. If anyone has some
> > short test cases for me to run please send them my way.
>
> Just use any good source of random data?
Better, use a completely predictable source of sets of predictable data.
E.g.
Compress these 2 1-bit strings:
0
1
Compress these 4 2-bit strings:
00
01
10
11
etc...
> On linux, /dev/random will do I think. http://en.wikipedia.org/wiki//dev/random
>
> Otherwise, you could just try using data generated from C++, via
> rand(). Make sure you seed the random number with the current time,
> the time (in as high resolution as you can acheive, microseconds
> usually), is almost always a good source of randomness. This explains
> how: http://www.daniweb.com/forums/thread1769.html
If it's a truncated MC/LC, then it's pretty predictable.
If it's not truncated, it's utterly predictable.
collec...@googlemail.com
wrote:
> collectio...@googlemail.com writes:
> > On Aug 12, 9:42 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:
> > > Thanks guys, I do appreciate that point of view. But the fact of the
> > > matter is that if you have a universal compressor, it is it's own
> > > proof. It seems a little anti-intellectual to assert that all ideas
> > > shouldn't be able to stand on their own and that one is necessarily
> > > better than another. I would just like to have a honest scientific
> > > discussion, the way comp.compression was intended to operate.
>
> > > As a matter of fact, I've almost finished the coding. It went fast as
> > > a result of using high-level string and fractal packages, which means
> > > there will be some need for optimization later. If anyone has some
> > > short test cases for me to run please send them my way.
>
> > Just use any good source of random data?
>
> Better, use a completely predictable source of sets of predictable data.
>
> E.g.
>
> Compress these 2 1-bit strings:
> 0
> 1
Sure.
0.5
0.2
?
Did that work? :o)
Ashley Labowitz
polynomials using Newton's method (quadratic convergence) and saves
off just enough digits.
For files with structured data, there is generally a way to "align"
the bytes so that they match up with edges of a fractal, particularly
so when many degrees of freedom are permitted. Consider the following
class of polynomials:
z^n + a cos(z)^n + b cos(sin(z)) + c
When iterated, this takes on periodic and chaotic orbits over the
complex plane, and by basing the fractal edges on whether the point
diverges, we can describe a region with unlimited self-similarity on
progressively smaller scales. That is basically how Mandelbrot first
discovered his set, but he didn't discuss this particular polynomial.
For instance, given a fragment of data, say
74 6f 20 62 65 20 6f 72 20 6e 6f 74 20 74 6f 20 62 65
fractals identify the small-scale patterns (such as repeated 20's) and
large-scale patterns (the byte spectrum, generating grammar and common
stems). For these reason, it may outperform gzip or related simple-
minded compressors.
By contrast, for random data it is almost as simple. We use
knapsacking to optimizing the placement of fractal curve alignments to
fully cover the data, taking advantage of "found order", or the
correlations that can be discovered. Any random file will contain
many instances of the same byte for example. This by itself is not
enough however. If it was, we could simply sort the bytes in the
file, run-length encode it, and send the arrangement of the bytes
separately. However, it is the arranging that almost all the
information is contained!
Instead, we use Taylor series from calculus to manage the arrangement
complexity. To illustrate an example, I'll use a random 16 bytes:
$ python -c "f = open('/dev/random', 'r'); print f.read(36)" | hexdump
-C
00000000 fa 41 c2 72 55 8d c8 27 b1 4a ac a2 13 68 37 d9
|.A.rU..'.J...h7.|
00000010 d5 60 de af d8 85 9c d9 3c 7e 1e ca c1 59 c5 cd
|.`......<~...Y..|
00000020 dd 1e 3d f9 0a |..=..|
In effect, the fractals treat this as floating point numbers and use
approximations from the fractal dimension and polynomial roots (of
which there are usually an infinite number). I can use Euler's
theorem to find these quickly, e.g.,
pi^2 - pi^1.7 + cos(sin(pi/1.7)) = 3.44071322 (variant of polynomial
above)
e^2 - e^1.7 + cos(sin(e/1.7)) = 2.45574537
and "fa 41 c2 72" from the above is 4198613618 = 2 * 179 * 11727971.
Notice the relationship between these 3 primes the sequence of
polynomial values taken as follows:
1464905053 = 18959 * 77267
1756147005 = 3 * 3 * 5 * 17 * 907 * 2531
1815095743 = 359 * 5055977
417443012 = 2 * 2 * 7 * 227 * 65677
Factoring could be a problem, but there are fast methods for this.
I'm using an EC method now (yay for libraries doing all the work for
you!). Right now, I'm not sure how to encode prime numbers, but even
if I have to add one, then factor, this extra flag doesn't offset the
compression savings.
In this case, the 36 bytes above compress down to
d1 1f 4e 99 4b 3d db 22 0e c3 b0 6d ae 4e e8 cc 23 a4 81 17 10 f5 c8
5f 72 ea b3 c7 07 76 f2 f5
which is only 32 bytes, and it contains all the information needed to
reconstruct the original 36.
Ashley Labowitz
using /dev/random for random data. I just need some sort of corpus
for structured data. I found the Calgary Corpus, but I would like
something smaller to make better examples in this thread (and because
the algorithm is slow for big file :(, not as bad as PAQAR, but still).
Ashley Labowitz
post.
All I will say is that you folks seem overly skeptical of what I would
think are clear mathematical truths. It is clear to me now that
comp.compression attracts inordinate numbers of that type (why is this
anyway? Most Math and CS groups have no such issue). No hard
feelings though, I'm the live and let live kind :) Patience for now,
I'm running some test cases...
Jim Leonard
> If anyone has some
> short test cases for me to run please send them my way.
I suggest you read http://marknelson.us/2006/06/20/million-digit-challenge/
Ashley Labowitz
I'm looking for short cases because of the program's running time and
because I'm still debugging it.
Phil Carmody
Mathematically, yes. As "compression", as the man in the
street would know it, no. And depending on your encoding,
you do realise that you've "compressed" '1' to an infinite
string! (0.001100110011... )
collec...@googlemail.com
wrote:
> collectio...@googlemail.com writes:
> > On Aug 12, 10:27 pm, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
> > wrote:
> > > Compress these 2 1-bit strings:
> > > 0
> > > 1
>
> > Sure.
>
> > 0.5
> > 0.2
>
> > ?
>
> > Did that work? :o)
>
> Mathematically, yes. As "compression", as the man in the
> street would know it, no. And depending on your encoding,
> you do realise that you've "compressed" '1' to an infinite
> string! (0.001100110011... )
Hehehe. Damn. I knew it was impossible!
Jim Leonard
> On Aug 12, 7:11 pm, Jim Leonard <MobyGa...@gmail.com> wrote:
>
> > On Aug 12, 3:42 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:
>
> > > If anyone has some
> > > short test cases for me to run please send them my way.
>
> > and specifically try to compress 'AMillionRandomDigits.bin'.
>
> I'm looking for short cases because of the program's running time and
> because I'm still debugging it.
Then chop the file in half. Or tenths, or whatever. You still won't
be able to compress it.
Jim Leonard
> The square has to
> be at least 6 x 6 (that is, 36 bytes) before space is saved from all
> files and probably at least 1K before 12% is seen.
"space is saved from all files" implies you can take your compressor's
output and run it through the compressor again. Are you now also
claiming that, through this method, you can compress any file down to
36 bytes?
Sportman
> because I'm still debugging it.
minimum input lenght that's to much. Otherwise testing some 1024KB
lenght random strings or many shorter lenght random test sets to be
sure the compression algoritme is not tuned at some short random
sequences. Don't forget to test decompressor as soon as possible to be
sure decompressor has all data in all cases to get the original input
back. Good luck :-)
jacko
opinion!
On 12 Aug, 22:55, Ashley Labowitz <sporecoun...@gmail.com> wrote:
> The program extracts approximate roots of the high-degree fractal
> polynomials using Newton's method (quadratic convergence) and saves
> off just enough digits.
the convergence of the fractals taylor series chosen will decide the
number of significant coefficients, and how acurate they have to be
stored. so a faster converging fractal would need less coefficients to
represent the same acuracy, BUT you'd also have to show that there is
no overhead in coefficient storage because more bits are required in
the coefficient field.
> For files with structured data, there is generally a way to "align"
> the bytes so that they match up with edges of a fractal, particularly
> so when many degrees of freedom are permitted. Consider the following
> class of polynomials:
define align, as i seem to think you are quantizing the complex plane
into boxes, and placing one datum byte in each box, and you have not
explained how the datum is constructed by the decompressor back from
'sample' points within the boxes.
> z^n + a cos(z)^n + b cos(sin(z)) + c
can not be bothered to show any convergence properties?
> When iterated, this takes on periodic and chaotic orbits over the
> complex plane, and by basing the fractal edges on whether the point
> diverges, we can describe a region with unlimited self-similarity on
> progressively smaller scales. That is basically how Mandelbrot first
> discovered his set, but he didn't discuss this particular polynomial.
how does this relate to a byte within a box on the complex plane?
> For instance, given a fragment of data, say
> 74 6f 20 62 65 20 6f 72 20 6e 6f 74 20 74 6f 20 62 65
> fractals identify the small-scale patterns (such as repeated 20's) and
> large-scale patterns (the byte spectrum, generating grammar and common
> stems). For these reason, it may outperform gzip or related simple-
> minded compressors.
maybe it could but so what? does it work on its own output?
> By contrast, for random data it is almost as simple. We use
> knapsacking to optimizing the placement of fractal curve alignments to
> fully cover the data, taking advantage of "found order", or the
> correlations that can be discovered. Any random file will contain
> many instances of the same byte for example. This by itself is not
> enough however. If it was, we could simply sort the bytes in the
> file, run-length encode it, and send the arrangement of the bytes
> separately. However, it is the arranging that almost all the
> information is contained!
shouldn't be any more complicated?! does the above paragraph imply you
remove a fractal, and calculate a set of residuals, so another fratal
can be found? an why do the roots have to be high order, is this due
to convergence speed requirements of a taylor series?
if you factor a file the sorting destroys no information, but as bytes
are not commutative within the file then this RLE process would fail.
> Instead, we use Taylor series from calculus to manage the arrangement
> complexity. To illustrate an example, I'll use a random 16 bytes:
>
> $ python -c "f = open('/dev/random', 'r'); print f.read(36)" | hexdump
> -C
> 00000000 fa 41 c2 72 55 8d c8 27 b1 4a ac a2 13 68 37 d9
> |.A.rU..'.J...h7.|
> 00000010 d5 60 de af d8 85 9c d9 3c 7e 1e ca c1 59 c5 cd
> |.`......<~...Y..|
> 00000020 dd 1e 3d f9 0a |..=..|
>
> In effect, the fractals treat this as floating point numbers and use
> approximations from the fractal dimension and polynomial roots (of
> which there are usually an infinite number). I can use Euler's
> theorem to find these quickly, e.g.,
>
> pi^2 - pi^1.7 + cos(sin(pi/1.7)) = 3.44071322 (variant of polynomial
> above)
> e^2 - e^1.7 + cos(sin(e/1.7)) = 2.45574537
> and "fa 41 c2 72" from the above is 4198613618 = 2 * 179 * 11727971.
> Notice the relationship between these 3 primes the sequence of
> polynomial values taken as follows:
> 1464905053 = 18959 * 77267
> 1756147005 = 3 * 3 * 5 * 17 * 907 * 2531
> 1815095743 = 359 * 5055977
> 417443012 = 2 * 2 * 7 * 227 * 65677
whats this got to do with roots?
> Factoring could be a problem, but there are fast methods for this.
> I'm using an EC method now (yay for libraries doing all the work for
> you!). Right now, I'm not sure how to encode prime numbers, but even
> if I have to add one, then factor, this extra flag doesn't offset the
> compression savings.
try shanks or pollard rho or lenstras factor methods. or get a quantum
computer :).
> In this case, the 36 bytes above compress down to
> d1 1f 4e 99 4b 3d db 22 0e c3 b0 6d ae 4e e8 cc 23 a4 81 17 10 f5 c8
> 5f 72 ea b3 c7 07 76 f2 f5
> which is only 32 bytes, and it contains all the information needed to
> reconstruct the original 36.
are you 100% sure?
looking at your post, the only things which relate to anything i have
thought about before are the factorization to introduce commutivity in
a data stream, the 2x+1 to obtain another factor set from a prime, and
residual calculations as in 1 bit dacs which at four times
oversampling (4 bits) can be made to output 16 bits resolution.
so how is the number in the byte box on the complex plane calculated?
describe the decopressor as this involves no patterm matching, just
pattern construction.
cheers
Ashley Labowitz
demonstration of the algorithm. I'm still writing up a formal
document describing the method, but please feel free to "reverse
engineer" it from the samples. More to come on the algorithm.
Here, I've used as test cases the du man page (chosen because it is
short), 500 random bytes, 500 null bytes, a JPG image, and the text of
the Magna Carta (translated by Project Gutenberg). Here are the
results and the initial bytes of each compressed file.
du man page
66% savings
91 79 4a 1b d3 05 7e f3 ef 96 13 5d 0b d3 69 8a
d0 cb 8f 05 0c ac 63 a5 07 ec 3e a2 76 5d 02 4c
ca 47 ff d3 2a 6c 0d 91 1a 83 5d 35 ac 3f 80 11
ab bb 9d 68 cf 11 95 5c 0a 5b 1b 78 9f 6f cc 61
1b 3a 70 51 08 f7 13 91 c3 e8 33 07 01 b9 cb b9
5d 16 84 6f 97 6d 05 08 de 04 cf 53 16 8b 7d 31
de e8 3b 0c 2b c1 c3 ed 6a 6f 69 37 a4 b1 91 ca
35 a1 99 dc e6 27 c0 8c f8 c3 fd b7 fe e3 2f 67
5c 5d 11 23 42 85 80 71 01 1d 65 62 44 58 ba 81
500 random bytes
14% savings
40 5a 27 90 04 f1 17 9e 02 84 ce 6e 6f 61 46 f0
95 ff d7 1b a3 cf 42 1b a8 15 05 eb ad 29 dc 93
ce 68 e3 8d 8c e9 38 73 67 3d 8e aa 37 9b 1e 8e
22 93 74 28 c2 95 a2 c6 56 8a f0 00 e0 a4 27 53
49 4d f9 43 f4 02 f9 15 44 b9 5e f0 60 b6 fc 98
8e fa 82 e2 72 18 88 f3 17 a0 30 51 ec 29 37 f2
500 null bytes
94% savings
20 24 72 1b e7 73 21 4d 3e 46 00 a8 4d 2f e8 72
9f 1a 4c 75 58 b7 aa a3 08 b6 71 6f 8c 2e (whole file)
gulls.jpg
19% savings
68 2e ad 80 77 a7 82 13 d5 af 73 67 db c7 cd ca
16 c4 7d 29 37 b4 53 58 69 0e ff df ed f5 06 0f
6f 7a fa 30 b8 f2 ff 2c 87 9a da cf cd 14 3c 26
7c cc 3f cb 95 a6 1f 85 cf 5b 76 bf 43 39 ef 82
c3 fb 56 44 2c 00 23 71 a0 a9 59 22 cb ba 53 bd
e0 a1 7e 1c 9e 94 06 46 73 d2 c3 d1 27 b3 3c e5
37 6f b2 3d 39 75 5f 81 a6 d5 19 5e 49 42 3e 0f
9c d9 14 0b e6 f5 0e 88 23 81 bb 25 11 97 91 63
66 0e 34 6e f3 62 a1 6a f0 23 8d 0c ad 1b 40 e5
06 e3 fa 19 1c 47 1d 1c 5a 28 9a 2b c1 1e 88 8c
19 84 3b dd eb 97 70 fd 5d ea 74 d1 af 4e f7 b5
61 1f e4 cb 56 33 54 c7 35 20 8b 94 2c 5f 55 10
magna01.txt (from http://www.gutenberg.org/dirs/etext06/magna01.txt)
73% savings
84 75 3d b0 6c 57 a3 3e 46 3a 07 88 fb 14 ee 9c
38 64 4a a7 18 12 81 fc 23 e4 9a 70 41 9c e7 3d
It seems fitting that the Magna Carta should have some place in a
compression breakthrough of this nature!
However, in the interests of full disclosure (as they say on Bugtraq),
I should point out that I'm still shaking the bugs out of the
decompressor: For some reason the files come out garbled. I've been
going a little bit "wild" with this compressor on all kinds of files
and I'm really pleased with the results, I just haven't take the time
to properly debug the decompressor (I needed the compressor asap,
expanding can wait).
Jim Leonard
> I should point out that I'm still shaking the bugs out of the
> decompressor: For some reason the files come out garbled.
Ah yes, that pesky little detail.
It is as this point that we will probably never hear from you again.
Most people are too embarrassed to come back here and report that
there was a flaw in their thinking.
Ashley Labowitz
really would like the decompressor. What motivated all this was a
lack of space on my Hard Drive where none of the off-the-shelf
compressors could save enough space. The fractal compressor I wrote
was perfect (and I could prove its correctness too). I was able to
squeeze down several Gigabytes of data to fix my space issues for now.
The files in question are a lot of personal correspondance, research,
and coursework archives. It's nothing terribly critical, but still
it's stuff I've saved for years and which I'd really like to get back :
(. The decompressor must be out there, it *has* to be. I don't know
what these other people you're talking about did, but no one will try
harder than I will to write it.
I'm very optimistic (I still have all the compressed files sitting
right here, the data is in them in some form), but I'll keep you guys
posted no matter what. You deserve no less, I suppose.
Jim Leonard
> It's nothing terribly critical, but still
> it's stuff I've saved for years and which I'd really like to get back
You deleted the originals without testing a working decompresser
first?
(I'd write more, but I lack the wit and eloquence necessary to make it
resonate.)
Ashley Labowitz
> > it's stuff I've saved for years and which I'd really like to get back
>
> You deleted the originals without testing a working decompresser
> first?
>
> (I'd write more, but I lack the wit and eloquence necessary to make it
> resonate.)
It's not so silly when you consider that there is no need for a
decompressor just yet: It's not stuff I look at every day. Besides,
if it does happen that I need to retrieve some files soon, before the
decompressor is finished, I can just finish it then (extra
motivation). I mean, what do people normally do in this situation?
Also, I wouldn't say I deleted the files per se, the compression
process simply replaces them with smaller versions. I'll add that the
savings were much larger than I could get with bzip2 (2.2ish GB ->
400ish MB).
Sportman
> I was able to
> squeeze down several Gigabytes of data to fix my space issues for now.
Then delete originals without building and testing a decompressor to
save HD space while Live hotmail offer 5GB free en GMail 2.8GB free
disk space...
Till now the quickest inventor needed 3 months to make a working
random data compressor/decompressor. You can be a self confident
miracle guy or are you family of Mr Frater?
http://www.dailymail.co.uk/pages/live/articles/news/news.html?in_article_id=475679
Scott
<sporec...@gmail.com> wrote:
>It's not so silly when you consider that there is no need for a
>decompressor just yet: It's not stuff I look at every day. Besides,
>if it does happen that I need to retrieve some files soon, before the
>decompressor is finished, I can just finish it then (extra
Heh. I don't think you've really thought this all the way through.
>motivation). I mean, what do people normally do in this situation?
If your data is worth keeping? Buy more hard drives. Seriously.
>Also, I wouldn't say I deleted the files per se, the compression
>process simply replaces them with smaller versions.
Thank you. That's the best laugh I've had all day.
-Scott
collec...@googlemail.com
I have a hard time figuring out if Ashley is serious or not.
Either way it's pretty funny.
Jim Leonard
> I mean, what do people normally do in this situation?
They test their theories with a working decompresser before deleting
only copies of files.
I'll just break the news now to save you the frustration later: Your
theory was flawed, and most if not all of your compressed data lacks
the necessary information to be reconstructed losslessly.
Your goal was to save space on your hard drive; you have achieved that
goal twofold, because now you can simply delete the "compressed"
files. They are useless.
Ashley Labowitz
files still come out wrong. The bug affects even the simple test
files I wrote about in an earlier post (even the 500 null byte file
comes back with lots of non-null bytes). I've been coding nonstop all
week, so I'm going to take some time to empty my mind and get a fresh
start.
collectio...@googlemail.com, rest assured that it seems very serious
from my PoV.
Jim, I know you're just trying to help, but my data wasn't random. As
much as I flatter myself that my data was valuable, it had plenty of
redundancy so it should be possible to decode, even if we grant your
guys' pigeonhole argument. In addition, I was able to prove the
compressor worked: that it makes all files at least 6% smaller, with
an average of twice that.
Jim Leonard
> In addition, I was able to prove the
> compressor worked: that it makes all files at least 6% smaller, with
> an average of twice that.
You did not prove your compressor worked because you never proved your
DECOMPRESSER worked. Your compressor output data that was smaller
than the original -- but without a working decompresser, you have no
idea if the compressed data was garbled during creation.
The above has nothing to do with the merits of your method, flawed or
not. It is just basic testing theory.
Keith Thompson
That doesn't prove that the compressor works in any meaningful sense.
I can write a compressor that reduces the size of all input files by
50%. It just discards every other byte of the file. But it's not a
working compressor unless I can retrieve the original file, unchanged.
It's very likely that you've done a somewhat more sophisticated
version of this, that your input files are irretrievably lost. The
*only* way to prove otherwise would have been to successfully
decompress the compressed files, recreating the original files without
error. By discarding your original files before you demonstrated an
ability to retrieve them, you've probably made a terrible mistake.
(If you're very lucky, you may be able to retrieve *some* of your
data, but that's likely to be more difficult than writing a correctly
working compressor/decompressor pair would have been.)
Next time, invest in a bigger hard drive or a DVD burner.
--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
collec...@googlemail.com
> Well this is frustrating. Everytime I think I got the bugs out, the
> files still come out wrong. The bug affects even the simple test
> files I wrote about in an earlier post (even the 500 null byte file
> comes back with lots of non-null bytes). I've been coding nonstop all
> week, so I'm going to take some time to empty my mind and get a fresh
> start.
>
> collectio...@googlemail.com, rest assured that it seems very serious
> from my PoV.
Then maybe you'll finally listen to what we were saying all along? It
is impossible to compress random data, and anyone who can understand
basic logic can prove the same.
Either you are a troll or a fantastically self-deluded person who
almost makes me feel sorry for you, despite the unlikelyness of any
sane person falling for such self-delusion.
Ashley Labowitz
and smaller scales. Thus, the complexity for bits can always be
borrowed, with promises to repay with information contained in not-yet-
computed smaller scales (it's good enough for the the
government* :) ). This algorithm is the key to "bit splicing" as I
call it, evading the pigeonhole principle which only applies to finite
sets (the fractal complexity is not even countable). I'd be surprised
if even the most seasoned members of this group had heard of the
Promise model of random compression.
* I try to be politics and country-of-origin neutral in fora like this
one. Sadly, I think this comment is basically true ubiquitously.
However, I found this technique too difficult to code in it's native
form. In any case, due to the very nature of self-similarity (it
being derived from a fixed equation after all), it's not clear how to
use it to encode unlimited entropy. This required using the cyclic
roots and rotation method.
Nevertheless, I now have the decompression tentatively resolved
(fingers crossed). It's not coded up, but I can show that using
Perfect numbers (Mersenne primes based on powers of two for the
bifurcation) will recall the granularity level of the 'bit dust".
Because of the Lyapunov chaos exponent of the polynomial in question,
I have to multiply by 3. I'm calling these Gerfect numbers. I'm
looking into some number theory textbooks from the library to find
theorems relating to k-smooth numbers because the fractal dimensions
are related to highly abundant numbers (not sure how, but I keep
seeing it in the data).
Sportman
> Because of the Lyapunov chaos exponent of the polynomial in question,
> I have to multiply by 3. I'm calling these Gerfect numbers. I'm
> looking into some number theory textbooks from the library to find
> theorems relating to k-smooth numbers because the fractal dimensions
> are related to highly abundant numbers (not sure how, but I keep
> seeing it in the data).
http://www.globalscaling.de/images/stories/pdf/gscompv10.pdf
http://www.globalscaling.de/images/stories/pdf/zahlentheorie.pdf
(German)
Prime numbers are also a repeating pattern what can be calculated:
http://www.calculateprimes.com/BookDVDOrderSubPage.HTM
Hans-Peter Diettrich
> Then maybe you'll finally listen to what we were saying all along? It
> is impossible to compress random data, and anyone who can understand
> basic logic can prove the same.
Why?
There exist (pseudo) random generators, whose output *is* random, until
you find the used algorithm and start conditions.
The only criterium for randomness, which I remember, is the
impossibility of the prediction of the *next* value, from all the
history. This obviously doesn't apply to files, which are closed before
compression, so that there doesn't exist a "next" element past EOF.
DoDi
Sportman
solute the random compression problem but mostly he is know for his
Rodin Coil:
His websites:
http://www.rodinproject.com
http://www.rodinmath.com
Jeff Rense interview:
http://markorodin.com/Marko_Rodin_Jeff_Rense_Interview.mp4
I wanted to add a link to Marko Rodin video's where he explains his
theory to math teachers but this video's are not online anymore. I
found this bad video where they made a small cut from it started at
4:17 min:
http://video.google.com/videoplay?docid=2874916987641932188
Thomas Richter
> Don't forget that fractals have _unlimited_ self-similarity at smaller
> and smaller scales. Thus, the complexity for bits can always be
> borrowed, with promises to repay with information contained in not-yet-
> computed smaller scales (it's good enough for the the
> government* :) ). This algorithm is the key to "bit splicing" as I
> call it, evading the pigeonhole principle which only applies to finite
> sets (the fractal complexity is not even countable).
Very fine, but all your input data is finite by nature (the files on your harddisk)
and your output data is (hopefully) finite as well, so that buys you nothing.
And, of course the counting theorem applies for finite sets.
> I'd be surprised
> if even the most seasoned members of this group had heard of the
> Promise model of random compression.
What I don't get: Do you really know what you're talking about? You want
to make us believe that you've managed to get an understanding in fractals,
but yet deny to accept something as simple as the counting argument.
Or even elementary logic, if that counts. If you can compress every
file, then you can apply your compressor to its output immediatly again
and, by induction, arrive at a file that is - zero bits long. Thus,
effectively, you claim to be able to compress everything to nothing.
Well, I can do that as well:
rm *
Problem is: There is no decompressor.
> However, I found this technique too difficult to code in it's native
> form. In any case, due to the very nature of self-similarity (it
> being derived from a fixed equation after all), it's not clear how to
> use it to encode unlimited entropy. This required using the cyclic
> roots and rotation method.
Actually, are you just repeating some words you collected here and there,
or do you understand what you say? I suppose the former?
> Nevertheless, I now have the decompression tentatively resolved
> (fingers crossed).
No, you haven't.
> It's not coded up,
As I said, you haven't.
> but I can show that using
> Perfect numbers (Mersenne primes based on powers of two for the
> bifurcation) will recall the granularity level of the 'bit dust".
Are we playing a round of buzzword bingo today? A perfect number is a
number which is the sum of its proper divisors. For example, 6 is a
perfect number since 1+2+3 = 6. Primes are never perfect.
> Because of the Lyapunov chaos exponent of the polynomial in question,
> I have to multiply by 3.
Liapunov exponents are defines for dynamical systems, not for polynomials.
So what is your system? Why "3". huh?
> I'm calling these Gerfect numbers. I'm
> looking into some number theory textbooks from the library to find
> theorems relating to k-smooth numbers because the fractal dimensions
> are related to highly abundant numbers (not sure how, but I keep
> seeing it in the data).
Don't look into number theory textbooks. I recommend locking yourself into
your room, sit back for a day and think cleany on what you claim. It
doesn't require a math education to see the mistake.
Alternatively, three or for hints on the forehead should be applied.
As entertaining as it is, I guess it's time to stop this comedy. Otherwise,
I recommend to divide an angle by three with compass and ruler, find
the root of an arbitary 5th order polynomial or construct a square of
the area of a circle with ruler and compass, and post results here. It
will be also an enjoyable reading. (-:
So long,
Thomas
snork...@gmail.com
> Marko Rodin invented an own math what accoording to him can also
> solute the random compression problem but mostly he is know for his
What is the "random compression problem?"
|
| Mark Nelson - http://marknelson.us
|
Sportman
> What is the "random compression problem?"
description in a way that the process can be reversed.
Hans-Peter Diettrich
>>What is the "random compression problem?"
>
> Describing all kinds of data (including random data) with a shorter
> description in a way that the process can be reversed.
That's impossible in that generality :-(
1) It's impossible to compress files of a size of 1 byte to anything
shorter.
2) There exist at most 256 different files of 1 byte, 256^2 files of 2
bytes, and so on. You cannot compress all the 2-byte files to 1-byte
files, because you only can uncompress the 1-byte files into 256^1
different files, not into 256^2 different files. I.e. at most 256
distinct 2-byte files can be compressed at all, the other 65280 possible
2-byte files are incompressible. The percentage of compressible files
becomes dramatically lower, the longer the considered files are.
DoDi
Ashley Labowitz
> > but I can show that using
> > Perfect numbers (Mersenne primes based on powers of two for the
> > bifurcation) will recall the granularity level of the 'bit dust".
>
> Are we playing a round of buzzword bingo today? A perfect number is a
> number which is the sum of its proper divisors. For example, 6 is a
> perfect number since 1+2+3 = 6. Primes are never perfect.
Please allow me to clarify. Mersenne primes are not the numbers I'm
using, but I mentioned them parenthetically because they illustrate
the crucial property I need better than the term "Perfect numbers,"
since not everyone is aware of the structure of Perfect numbers.
Mersenne primes are intimately connected with all even perfect numbers
(including 6, your example). To see more about the connection, I
recommend you read this page: http://mathworld.wolfram.com/PerfectNumber.html
.
Best,
Ash
Ashley Labowitz
If it is possible to access this data, no one will work harder than
I. I'm still confident. The data is working as a motivator, as
expected :)
Thanks everyone, for your polite comments. I think the takeaway from
this for all of us is that voices have been heard on both sides in
this thread illustrating the healthy debate among experts on whether
random data can be compressed.
Keith Thompson
There's a "healthy debate" in the same sense as there's a "healthy
debate" on whether 2+2=5.
Daniel Butler
posted quite a bit: Are you claiming that your *compressed output* can
be run through the compressor again? Ad infinitum? If not, then your
claim that your compressor will reduce "all files" in size is patently
false. You must see this.
Dan Butler
collec...@googlemail.com
He doesn't seem to respond to logic.
Actually you might be able to explain his behaviour if you believed
that many (actually most) people's minds are "fragmented".
Most people are incapable of reason, because their higher brain
functions are deactivated. The only way to "connect the dots",
basically turn many disparate pieces of information together into one
smaller neat simple theory, is to turn on the higher brain functions.
But most people strictly have them turned, off.
This is usually enforced by our broken school system, where they teach
learning by rote, in place of learning freely and learning by logic.
In a sense, the school system breaks the minds of people, fragments
their minds.
People whose minds are fragmented, try to deal with the world by
"collecting knowledge fragments". Each "knowledge fragment" only is
proven correct by politics and social hierarchy, not by any connection
to reason or logic.
Obviously, the world being so huge and vast and complex, you soon find
yourself needing billions of "knowledge fragments" to correctly deal
with most situations, instead of just needing a few simple theories
which are instead flexible enough to predict what to do in all those
situations.
People with fragmented minds, thus don't like newness or novelty as it
can be scary for them, unless there are many others telling them what
to do. They become very conformist.
They also tend to grab the wrong knowledge fragment, and stubbornly
strictly apply it.
As Ashley might be doing here. He grabs some "knowledge fragment" (if
you can even call it knowledge), that "discussion is always a good
thing because all things can be debated by experts", or whatever
retarded idea he was trying to tell us... and keeps on applying it,
despite that we've proven him wrong.
Sad.
But to be honest, people with fragmented minds usually have themselves
to blame for it. While it is true that the system breaks people's
minds, most genius thinkers tend to put their minds back together
again.
It's sort of like escaping the ghetto really. Most people in the
ghetto deserve to be there. Some don't, but they usually escape it as
soon as they are old enough to. I have the feeling that Ashley is over
18. That's the age I managed to "unfragment" my mind, after my parents
and the school system broke it.
Daniel Butler
tend to teach by rote because that's the way that teachers learn -
their type of person who becomes a teacher tends to be the type of
person who learns by reading facts in a disconnected way, with no
context. Then they think that must be the best way to learn, because
all the researchers studying methods of education are..teachers.
And never mind this "well-rounded" bullshit. What kids are really
good at, be it music or art or some specific science, is squashed out
of them as they're forced to trudge through (to them) dull,
uninteresting, hard-to-understand topics of information that, in the
end, makes them *loathe* learning, strategic thought, and creative
thinking.
weedi...@gmail.com
> The difficult part is extracting the roots of high-order polynomials,
> but this can be done classically using Newton's method adapted for the
> complex plane
Hmmm, I am a little too busy atm to look at your notes but I find it
rather concerning that you think using Newtons Method to find the
roots of a high order polynomial is a viable solution. This is one
of the crudest methods of doing this and is certainly not to be
recommended. Something like Lehmer schur or Lagurre would be a better
choice. However, if the accuracy of these roots is as important as I
suspect then these will not suffice either. You need to look at some
of the state of the art algorithms out there and choose your methods
based on the typical characteristics of your polynomials. For
instance, do they have multiple roots? How closely are the roots
spaced? Do they lie on the unit circle? Etc etc...
Daniel Butler
If the previous posts are any indication, they lie on the Descartes-
Fourier Multiply Coplanar Emulated unit hypertoroid. Sic.