Good day everyone,

I've been programming a technique that allows most, if not all files
to be reduced in size.  It is done using cyclic roots and high-degree
polynomials where the total information content of the coefficients is
smaller (sometimes much smaller) than the original text.

The FAQ has been pointed out to me where it "proves" that not all
files can be compressed by any algorithm, but it seems to me that it
has some mistakes.  They aren't serious, but they are oversights that
prevent it from applying to fractal bits, permutations and sorting the
bytes of the original file.

The gist of this algorithm is that when certain polynomials are
iterated over the complex plain (e.g., like the Mandelbrot set),
information can be encoded in the "ragged" edges of the curve.  I'm
afraid this may have been done before, and I don't want to waste any
work.  Does it ring a bell to anyone?

If someone would like to look at my pseudocode, I would be grateful.

CheeriO,
Ash

Well it was a good day...

You have not.

No it's not.

It's not just *called* a proof. It *is* a proof. If you think it's
not a proof, then you are a loon.

Ergo you are a loon.

Yes. I call it "compression by coincidence". You're about the
thousandth loon to pretend its got any merit at all. You won't
be the last.

I wouldn't.

Phil
--
Dear aunt, let's set so double the killer delete select all.
-- Microsoft voice recognition live demonstration

Hello again,

Phil, you are jumping to conclusions.  It turns out that if you place
the bytes to be compressed on a square grid and then iterate a complex
polynomial you can show the fractal edges will naturally follow a
significant number of bytes, allowing those to be removed from the
original entropy.  This is the secret to it working for _all_ files,
not just some special form.  I know the mathematical establishment is
slow to embrace change (and wisely so), but Phil, please don't be
ridiculous.

The difficult part is extracting the roots of high-order polynomials,
but this can be done classically using Newton's method adapted for the
complex plane (I'm working on an extension for quarternions, but
Python is not my strength :( ).  Obviously, fractals only carry as
much information as is in the coefficients of their generating
polynomial, but you might be assuming that they carry infinite
information.  Let me know if this is your mistake.

Here's the twist, fractional bits can be extracted from the _exact_
(or nearly so) value of the roots of the polynomials.  This is the
chief error of the Pigeonhold Principle "proof" of impossibilty.  Of
course, the Pigeonhold Principle is very useful in general, but it is
a mistake to apply it to non-integer sets such as fractional bits.

Any data generated from a deterministic process has a Kolmogorov
Complexity that must be lower than the size of the data as the number
of bits to model goes to infinity.  However, in practice it is
impossible to find this optimal Turing machine in general.  However,
the set of strings that are stochastic or contain purely random
elements are themselves bound to fractal models.

But it is not necessary for you to agree, I've shown the proof to a
number of people and none have succeeded in finding any mistakes in
it.  I'm not doing it for the money, but to help the world.

Best,
Ash

One of the better posts this year!.
Phil's post i mean...  ;)

I would also like to nominate this sentence : 'the set of strings that
are stochastic or contain purely random elements are themselves bound
to fractal models'  :)

Thanks Fulcrum.  It's good that someone can appreciately respond to a
challenge to conventional thinking without taking the easy way out.

On Aug 10, 7:35 pm, Fulcrum <werner.bergm...@gmail.com> wrote:

On Aug 10, 5:23 pm, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
wrote:

Yes you have.

Yes it is.

I won't argue with you there.

Yup.

Two words: fuck you.

On Aug 10, 8:01 pm, industrial_...@hotmail.com wrote:

I don't think I understand what you mean industrial_...@hotmail.com
but you are being unfair to Phil.

I think it's best if people would evaluate my ideas on their merits.
You can show fractal convergence using criteria known for decades and
at that point, enough accuracy has been achieved.  My implementation
uses 1 nybble for the main coefficient, and builds from that using a
doubly-linked list.  Each block is stored using offsets as follows.
This is from the README file I'll be distributing with my source
code.  I apologize but fixed-width format is needed to properly read
it.

         +---+---+---+---+---+---+---+---+---+---+
         | SEQUENCE  |HD |     COEF      | BYTE  | (ctd.)
         +---+---+---+---+---+---+---+---+---+---+
         | MAIN NUMBER   |   POWER EXP   | MODE  | (ctd.)
         +---+---+---+---+---+---+---+---+---+---+
         | TY| X | Y | Z | TIME  | PERF  | PRIME |
         +---+---+---+---+---+---+---+---+---+---+

With this key:
HD = head
TY = type
X, Y, Z = codes for which roots of -1
TIME = timestamp
PERF = performance characteristic to optimize
PRIME = base of polynomial (not Galois)

The high-precision root extractions can be done in 33 lines of
assembly, or so it seems: I don't have the resources to test this as
well as I'd like.  One other way to model it is to use a network flow
system (or a shortest distance one, which is easily solved with
Dijkstra's algorithm).  I hope you guys wouldn't disagree with
Dijkstra's algorithm... :)

Cheerio,
Ash

On Aug 10, 7:46 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:

He was obviously trolling, and I was in a "wanna-see-blue-blood-bleed-
red" mood so I think I officially clunked his ass outta this thread
where he can regurgitate his bullshit on some other forum if he has
nothing to add worth reading to the topic.

To be fair, your idea is kinda vague to me, I don't get the exact
mechanics of your algorithm. Put it in a nutshell: how do you expect
to compress *most* of all possible files? By how much? Are there any
limits, if so, what?

Each block contains part of the target file, right?

If you really have the source code, I guess you can work on your idea
with more experienced peoples around this newsgroup, but I doubt
you're covering any new ground (not saying this to be nasty but unless
you can specifically prove any flaws in the counting argument, I don't
think yer gonna get anywhere.) How do you enumerate 4 combos with only
2? A.K.A: How do you compress 00 01 10 or 11 to 0 or 1 (compress all 2
bits to 1) When 0 or 1 can only represent 2 out of the 4 possible
combos?

My theory around this (see posts from 6 months ago) was based on the
nature of "random-appearing data" and that it should have some
distinct feature that "normal, practical" data doesn't and thus should
be able to be comrpessed. But then I realized that "random-appearing
data" makes about 99.99% of all possible files and that the data we
see everyday is a painfully tiny portion (0.0001%) so logically it's
impossible.

The only remaining possible way out of this would be to store data as
"time ticks" where one tick would be an extremely tiny fragment of
time (e.g. 10^-30th of a second.) Simon Jackson I heard is working on
this.

But I'm up for a civilized discussion about your idea, but I request
that you explain with eloquence and simplicity in mind since math
isn't my strength (ironic, isn't it?)

You are a crank, a loon, a babbling drunk baboon.

Help comp.compression by turning your computer off.

Phil
--
Dear aunt, let's set so double the killer delete select all.
-- Microsoft voice recognition live demonstration

On Aug 11, 3:02 am, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
wrote:

Shut the fuck up, faggot. Did you take a break from your cock-sucking
sessions? Go resume it, and let's hope you'll choke on your own cum.

Phil, please don't make personal attacks.  It's bad for the atmosphere
of the list and interferes with constructive discussion such as that
of universal fractal compression.  Name-calling in particular makes us
all look juvenile.  I see from the archives that this list has been
very professional in the past, and I'm sure it can regain that if we
can tolerate new ideas.  Thank you in advance.

Ash

On Aug 11, 4:02 am, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
wrote:

Happy Saturday everyone!

After using the fractal approximations it is necessary to fill out the
full data square.  This is akin to a knapsack problem with very
irregular objects, but some overlap can be tolerated.

Completely random data must have an irrational fractal dimension (well-
known property of the logarithm), which is sufficient to reconstruct
the steps that built the fractal.  I've tried the Julia set, the
Menger Sponge and others by Sierpinski.  Once the proper sequence of
fractals is selected, I can show that a file containing random data,
one full bit of entropy per bit, can be compressed by about 12%.  I
know it is not much, but it is something, and this includes the space
needed for the fractal iteration information and other block metadata.

Before I become famous, I want to always remember the people that
helped me get where I am.  I'm lucky that I've been able to reuse lots
of existing software and not spend time reinventing the wheel.  To
give credit where it is due: I used Kirby Urner python fractal code
and also got some ideas from Erik Wrenholt and also some of the
research done at Seattle University.

I've been working hard all day and soon I will have finished the
implementation.  can anyone suggest file types that would be most
interesting to test on?

industrial_...@hotmail.com, to answer your question, the compression
does not work on files of a trivially small size.  The square has to
be at least 6 x 6 (that is, 36 bytes) before space is saved from all
files and probably at least 1K before 12% is seen.  If many small
pieces of data need to be compressed, they can simply be concatenated
with some sort of delimiter or serialized data structure.  Does that
make sense?

CheeriO,
Ash

They aren't new ideas. They are isomorphic to every other
scheme of 'compression by coincidence' that we've seen.
You are ignorant of simple mathematical facts, such as the
Kraft inequality. You display a complete ignorance of the
field.

If you had come into the group humbly, the I might have been
more charitable, but you entered stating absolutes, and
absolute bollocks at that, so you got the reception that you
did.

We've seen your type so often, that I for one simply have no
patience. Maybe others will waste their time with you, some
do that kind of thing. However, I'm glad that you've noticed
that I consider you to be a crank devoid of any knowledge or
skills in the field, and that that can be recorded for posterity
in the various usenet archives, as that means my job is done.
You can never claim that your bogus nonsense got a positive
reception from those who know what they are talking about.
Well, you probably can, as you're prepared to lie about
mathematical facts, so I guess you'll be prepared to lie about
pretty much anything.

I shall now gladly decrease your audience by one as, as I said,
I've run out of patience. You have no intention to learn, you
are beyond hope.

*PLONK*

Phil
--
Dear aunt, let's set so double the killer delete select all.
-- Microsoft voice recognition live demonstration

Phil,

Thank you for stopping the name-calling.  If we can't agree on
anything else, at least something good for comp.compression has come
from this discussion.

Sincerely best wishes,
Ash

Another product of a failed education system, that values work on
paper over REAL LIFE FUCKING TESTS.

Your pseudo-code is a piece of crap.

You haven't even tested it. And I can tell you, NEVER believe anything
unless it's TESTED.

I blame the academic wankers who refuse to let physics students play
with real physical objects but instead make them do equations on paper
instead.

Your test will never work.

More likely you'll blame everything but yourself in your inability to
even generate a test.

The saddest thing about you, is that you throw around complex words
that you don't even understand, in an effort to make yourself seem
more intelligent. Yet you don't understand basic logic! Even pigeons
in their pigeonholes know more than you ;)

On Aug 11, 4:27 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:

It is this specific claim, "space is saved from all files", that we
have an issue with.  There is no compressor that can compress every
single 36-byte file by at least one bit.

I suggest you read http://marknelson.us/2006/06/20/million-digit-challenge/
and specifically try to compress 'AMillionRandomDigits.bin'.  Matt
Mahoney has conjectured that it might be possible to compress this
file by up to 50 bits due to the way the data was assembled.  So,
perform the following:

1. Compress this file using your process
2. Decompress your resulting file into a 3rd file
3. Compare the 3rd file with the original to confirm complete lossless
decompression

We await your test results.

Stronger - by at least zero bits, with at least one file being
compressed by at least one bit.

But for all practical purposes (such as attempting to communicate
simple ideas to the denser members of the population) your version
works.

There's an even simpler wording (that again is loose), namely
'for every file compressed, two must expand'.

Phil
--
Dear aunt, let's set so double the killer delete select all.
-- Microsoft voice recognition live demonstration

On Aug 12, 7:46 pm, Phil Carmody <thefatphil_demun...@yahoo.co.uk>
wrote:

It's obvious that compression of random files is impossible.

Only compressing patterns is possible.

If only people would try to ways to identify more patterns, and
represent these patterns with smaller "words", then people would
generate better compression.

But then I suppose PAQ is quite close to the extreme end of
compressionability anyhow? It must have a good output syntax at least.
I don't think that the "offset+length" syntax is so tight. Dictionary
based compressors should be able to acheive more compression, in
certain cases.

Thanks guys, I do appreciate that point of view.  But the fact of the
matter is that if you have a universal compressor, it is it's own
proof.  It seems a little anti-intellectual to assert that all ideas
shouldn't be able to stand on their own and that one is necessarily
better than another. I would just like to have a honest scientific
discussion, the way comp.compression was intended to operate.

As a matter of fact, I've almost finished the coding.  It went fast as
a result of using high-level string and fractal packages, which means
there will be some need for optimization later.  If anyone has some
short test cases for me to run please send them my way.  My email is
sporbzipecoun...@gmail.com (remove my favorite compression program to
get the address :) ).

On Aug 12, 9:42 pm, Ashley Labowitz <sporecoun...@gmail.com> wrote:

Just use any good source of random data?

On linux, /dev/random will do I think. http://en.wikipedia.org/wiki//dev/random

Otherwise, you could just try using data generated from C++, via
rand(). Make sure you seed the random number with the current time,
the time (in as high resolution as you can acheive, microseconds
usually), is almost always a good source of randomness. This explains
how: http://www.daniweb.com/forums/thread1769.html

Works on my Mac as well :))))))

less -f /dev/random

On Aug 12, 9:58 pm, collectio...@googlemail.com wrote:

Note, I do not expect this fellow to actually compress it. But I am
doing all I can to reduce his pain.

The sooner he finds out he cannot compress the truely random, the
better for his emotional and mental well being. So assisting him by by
pointing him to a good source of random data is being a benefit to his
well being.

Randomness has only one real valid use with respect to compression
schemes.

That is to see, by how much on average does your compressor EXPAND the
random data.

A good compressor, will only expand random data by a tiny amount. :)
Perhaps 4 bytes or so per gigabyte. That can be acheived with good
escape codes. You'd have to fit a header, a length, and lots of other
stuff in there though, so it's hard to not expand most random data by
at least 16 bytes or so minimum.

A BAD algorithm, could expand the random data many times over!

When I get around to writing my own compressor, I'll probably use
random data, just to test the efficiency of my algorithm. This will be
just to make sure my algorithm doesn't screw up the data by inflating
the random data to 4x it's natural size, lol. You can never be too
sure that you don't make the most basic of mistakes. Even the most
brilliant of geniuses make the most basic of mistakes from time to
time. That's why we never let our guard up against checking ourselves
for the most basic of mistakes.

On Aug 12, 7:15 pm, Jim Leonard <MobyGa...@gmail.com> wrote:

Or just recompress his compressed file :)

Ashley never told us if his compresed files can also be compressed. I
wonder what his answer to that one is?

Maybe if he is clever enough he can recursively compress his data down
to 1 byte :) tihihihihihi!!!