Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

perpetual-compression

2,343 views
Skip to first unread message

Jules Gilbert

unread,
Jul 8, 2002, 3:11:38 PM7/8/02
to
Hello folks:

I have decided to sell my 'random' compressors.

This is what I mean when I use the word 'random' or
'random-appearing'.

1) My compressor technology ONLY compress'es random-appearing-data.
It is not able to reduce the filesize of files containing information
that can be compressed using conventional compressors', say or ARJ or
GZIP.

(While one can XOR the input of say, a text file, and that will enable
the material to be compressed, it is much better to first compress
such a file with a good quality compressor, say an ARJ compressor or
other high quality CONVENTIONAL compressor.) After all, XOR'ing does
not reduce filesize. Compression, even conventional compression,
does!

2) Only buffer's of substantial size can be compressed. The amount
of computer time required to compress smaller files, say 64k, is so
high so as to make the effort impractical.

3) As the program exists today, it accepts one or more buffers of 8MB
of previously compressed data and produces smaller buffers as output.
It does not seem terribly difficult to me for the method to be
extended to handle ordinary files. In fact, I do this now (not well,
though).


For various reasons, I am interested in restricting the use of my
products (now only one program) to the United States. Further, I am
not enamored of the patenting process -- it is really a mechanism for
managing information disclosure, and very little protection is
provided from really bad people -- people I would not do business with
in any case. So I am offering to license my program by means of trade
secret ONLY.

My program is suited for use with data that is written once and read
many times, such as movies or other entertainment data, etc. It is
probably not suitable for data that is written once and read only
several times unless space is a critical problem constraint.

Our demonstrating process involves two machines, not connected by wire
or other means. Compression occurs on one machine, the results are
transferred by floppy and the result is de-compressed on another
machine. These machines can be inspected by a technician, they are
ordinary PC's running FreeBSD.

If you represent commercial interests, please contact me to learn
more.

Sincerely,
Jules Gilbert

Glenn Gordon

unread,
Jul 8, 2002, 10:47:45 PM7/8/02
to

"Jules Gilbert" <j...@medg.lcs.mit.edu> wrote in message
news:496bb05e.02070...@posting.google.com...
> Hello folks:
...

> Our demonstrating process involves two machines, not connected by wire
> or other means. Compression occurs on one machine, the results are
> transferred by floppy and the result is de-compressed on another
> machine. These machines can be inspected by a technician, they are
> ordinary PC's running FreeBSD.
...

As a technician I do not need to inspect the machines. Your demonstration
process demonstrates nothing, we technicians only believe in independently
achieved results on multiple platforms with data of our choosing and
analysis of source code. We only buy black-box solutions that implement
proven algorithms and methods, and avoid the proprietary until it has large
market share and/or a guarentee of long term support by an established and
respected company, and even then we give it a more rigorous test than your
"demonstration".

And yes, we technicians embaress the vacuum cleaner salespeople too. We
understand the principles involved, which can be a real downer for the
amateur stage magicians in sales, and even moreso for the frauds.

If you are really affiliated with MIT you have many more viable options for
marketing working technology than posting to a newsgroup.


Mark Nelson

unread,
Jul 8, 2002, 11:04:35 PM7/8/02
to
Hi Jules,

I think I mentioned this in private correspondence, but I don't believe you
responded.

My challenge to anyone who has a random compressor is as follows:

There are one million random decimal digits at the following web site:

http://www.rand.org/publications/classics/randomdigits/

The actual digits are here:

http://ftp.rand.org/software_and_data/random/digits.txt

Write a program that can generate those decimal digits in the same order
that they appear on the web site.

Put your program and any data files it requires on a floppy disk and run it
on a non-networked computer that I control

If you can do this with fewer than 415,241 bytes, you will have demonstrated
the ability to compress random data.

Of course, the degree to which you limbo under that number will determine
how impressed we should be.

Any reason you can't pull it off with 100K of java source?

Caveat: there are probably many ways to pull a con job on this test. I
haven't thought of all of them, but I'm sure readers of this NG can help me
tighten it up a bit.

Challenge: if you can't do this, you are just blowing smoke, go away and
quit bothering us. None of your excuses will convince me that you are doing
anything other than avoiding your comeuppance.

--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info


|
"Jules Gilbert" <j...@medg.lcs.mit.edu> wrote in message
news:496bb05e.02070...@posting.google.com...

The 3 Wise Men US

unread,
Jul 9, 2002, 1:05:14 AM7/9/02
to
j...@medg.lcs.mit.edu (Jules Gilbert) wrote in message news:<496bb05e.02070...@posting.google.com>...

> Hello folks:
>
> I have decided to sell my 'random' compressors.

How much do they cost? Are you going to sell the decompressors too?

> This is what I mean when I use the word 'random' or
> 'random-appearing'.
>
> 1) My compressor technology ONLY compress'es random-appearing-data.

How much does it compress the data by? Can the output be further
compressed by repeated runs?

> It is not able to reduce the filesize of files containing information
> that can be compressed using conventional compressors', say or ARJ or
> GZIP.

It doesn't compress this data at all, or not as well as conventional
compressors?

> (While one can XOR the input of say, a text file, and that will enable
> the material to be compressed, it is much better to first compress
> such a file with a good quality compressor, say an ARJ compressor or
> other high quality CONVENTIONAL compressor.) After all, XOR'ing does
> not reduce filesize. Compression, even conventional compression,
> does!
>
> 2) Only buffer's of substantial size can be compressed. The amount
> of computer time required to compress smaller files, say 64k, is so
> high so as to make the effort impractical.

So the process takes longer with smaller files, or it takes the same
amount of time with all files? How long would it take to compress
a 64k file and an 8MB file on a machine with a 1GHz processor?

> 3) As the program exists today, it accepts one or more buffers of 8MB
> of previously compressed data and produces smaller buffers as output.
> It does not seem terribly difficult to me for the method to be
> extended to handle ordinary files. In fact, I do this now (not well,
> though).
>
>
> For various reasons, I am interested in restricting the use of my
> products (now only one program) to the United States. Further, I am
> not enamored of the patenting process -- it is really a mechanism for
> managing information disclosure, and very little protection is
> provided from really bad people -- people I would not do business with
> in any case. So I am offering to license my program by means of trade
> secret ONLY.
>
> My program is suited for use with data that is written once and read
> many times, such as movies or other entertainment data, etc. It is
> probably not suitable for data that is written once and read only
> several times unless space is a critical problem constraint.

How fast is the decompressor?

SCOTT19U.ZIP_GUY

unread,
Jul 9, 2002, 9:54:56 AM7/9/02
to
ma...@ieee.org (Mark Nelson) wrote in <7psW8.456211$cQ3.36412@sccrnsc01>:

>Hi Jules,
>
>I think I mentioned this in private correspondence, but I don't believe
>you responded.
>
>My challenge to anyone who has a random compressor is as follows:
>
>There are one million random decimal digits at the following web site:
>
>http://www.rand.org/publications/classics/randomdigits/
>
>The actual digits are here:
>
>http://ftp.rand.org/software_and_data/random/digits.txt
>
>Write a program that can generate those decimal digits in the same order
>that they appear on the web site.
>
>Put your program and any data files it requires on a floppy disk and run
>it on a non-networked computer that I control

Shucks I was going to write a program that used the URL to get the
data by reading it off the net. Since the URL is really what
the data compresses to. But this last statement wrecks that plan.
I suspect it wrecks the other guys plan too.

I do have a back plan but it costs money I need to bribe the people
at the web site to put up the random data that I want then I could
take you up on the challenge, But I lack the amount of cash needed
for this con. Maybe your friend Jules has another trick up his sleeve
but I suspect he will just stall you and hope other fools provide
him with the cash.


David A. Scott
--
SCOTT19U.ZIP NOW AVAILABLE WORLD WIDE "OLD VERSIOM"
http://www.jim.com/jamesd/Kong/scott19u.zip old version
My Crypto code http://radiusnet.net/crypto/archive/scott/
My Compression code http://bijective.dogma.net/
**TO EMAIL ME drop the roman "five" **
Disclaimer:I am in no way responsible for any of the statements
made in the above text. For all I know I might be drugged.
As a famous person once said "any cryptograhic
system is only as strong as its weakest link"

Jules Gilbert

unread,
Jul 9, 2002, 3:12:54 PM7/9/02
to
Mark:

It's possible that I've forgotten but I do not remember your remarks.

As to the demo, my friends and I have arrived at a 'standard' demo.

We arrive with two floppies, one containing a data file and the other
a computer program. The program reads the data and explodes the data
to CD's. The images produced are standard 650MB CD's containing many
files, almost all of which are
highly compressed using gzip and various other types of space reducing
programs.

The 'we' includes a policeman for security purposes and someone who
actually does the demo.

We've chosen this demo because we think it best describes exactly what
our product does. Our assertion is that client data can be readily
compressed and distributed, then decompressed.

It's not very fast -- each decompression stage reads the prior stage
datafile and produces a subsequent file, then deleting the prior stage
datafile. My hope is that once a customer is identified, that
improvements will be made that will make this inefficiency obsolete --
but today, this is the situation. So even though the decompression
process is exceedingly fast, the 'actual' decompression time FOR ALL
THE STAGES is not great.

Compression varies of course, but each stage delivers as much as 15%
compression until the input data is less than half a megabyte.

Mark, as to delivering a decompressor to you. No thanks. So far as I
know, no one else has a working 'perpetual' compressor (no, my system
isn't really perpetual, either! -- that's marketing), but no one else
has accomplished anything even remotely like this.

I'm not great at business. I'm just a technician, Mark. I'm going to
let a small group of friends help me manage my program. If you have
helpful suggestions, let us know -- you know most of their e-mails.


the3wi...@yahoo.com (The 3 Wise Men US) wrote in message news:<dd9a1641.02070...@posting.google.com>...

Willem

unread,
Jul 9, 2002, 4:47:47 PM7/9/02
to
Jules wrote:
) We arrive with two floppies, one containing a data file and the other
) a computer program. The program reads the data and explodes the data
) to CD's. The images produced are standard 650MB CD's containing many
) files, almost all of which are
) highly compressed using gzip and various other types of space reducing
) programs.

Ah-HA! So *you yourself* provide the data that will be compressed.

) The 'we' includes a policeman for security purposes and someone who
) actually does the demo.
)
) We've chosen this demo because we think it best describes exactly what
) our product does. Our assertion is that client data can be readily
) compressed and distributed, then decompressed.

Given this demo, I will assume that your product can compress some
carefully-selected data, which is very much tailored to this compression
scheme of yours.
Your assertion is not validated in any way whatsoever by such a demo.

If you want to sell your product, you're going to have to substantially
change this demo procedure to at least include allowing the client to
provide the data that will be compressed.

It is my opinion that you're either stupid or a fraud.


SaSW, Willem (at stack dot nl)
--

Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be

drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

Mark Nelson

unread,
Jul 9, 2002, 9:42:53 PM7/9/02
to
>, But I lack the amount of cash needed for this con.

Fortunately, there is no cash reward for achieving the nearly-impossible,
but just in case I'd better get a copy of the file into escrow somewhere!

--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|

"SCOTT19U.ZIP_GUY" <david_...@emailv.com> wrote in message
news:924659914H110W...@216.168.3.40...

Mark Nelson

unread,
Jul 9, 2002, 9:46:37 PM7/9/02
to
>We've chosen this demo because we think it best describes exactly what

Jules, I'm feeling pretty smug about the correctness of my challenge. Anyone
who can really compress random data can meet my challenge with absoultely no
fear of reverse-engineering. I think you can come up with legitimate
procedures that will satisfy you of your safety.

Prove random compression with your own data sets? Ha!

Let me know when you cook up something that can meet the challenge. Until
then, well, the last paragraph of my original response stands!

--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|

"Jules Gilbert" <j...@medg.lcs.mit.edu> wrote in message

news:496bb05e.0207...@posting.google.com...

Mark Nelson

unread,
Jul 9, 2002, 10:01:05 PM7/9/02
to
By the way, my smarmy tone wasn't meant for you specifically, as my post
said, it was to the anonymous mass of folks who fit the description of
"anyone who claims to be able to compress random data." So while I don't
think you can do as you say, I certainly don't intend to be ugly about it.
:-)

But really Jules, I think the bit about the policeman puts you over the top!

--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|
"Jules Gilbert" <j...@medg.lcs.mit.edu> wrote in message
news:496bb05e.0207...@posting.google.com...

Chris

unread,
Jul 9, 2002, 10:13:25 PM7/9/02
to
Hi Mark

I agree with you and the pointer to that dataset is very useful thanks

But...

please qualify this statement

> absoultely no fear of reverse-engineering. I think you can come up


Java not reverse engineerable?

I thought it was one of the easiest

Chris


"Mark Nelson" <ma...@ieee.org> wrote in
news:1mMW8.453103$352.71255@sccrnsc02:

Gordon V Cormack

unread,
Jul 9, 2002, 10:36:36 PM7/9/02
to
Jules -

Where have you been?

Your proposed medicine show hasn't changed since '97. [See comp.compression
FAQ, section 9.4]

Are you still planning to sue anybody that challenges your claims?
--
Gordon V. Cormack CS Dept, University of Waterloo, Canada N2L 3G1
gvco...@uwaterloo.ca http://cormack.uwaterloo.ca/cormack

The 3 Wise Men US

unread,
Jul 10, 2002, 12:20:01 AM7/10/02
to
j...@medg.lcs.mit.edu (Jules Gilbert) wrote in message news:<496bb05e.0207...@posting.google.com>...

> Compression varies of course, but each stage delivers as much as 15%
> compression until the input data is less than half a megabyte.

This is for lossless compression of an 8MB file right?
I understand that your compression scheme only works on
certain files, but based on the pigeon hole principle and
the number of 8MB files compared to the number of 0.5MB
files, your scheme would appear to be so selective that
the chances of you finding a client with just one
suitable data file are probably less than the chances
of you completing Mark's challenge.

John

unread,
Jul 10, 2002, 2:04:12 PM7/10/02
to
Errr... how about pkzip?

"Mark Nelson" <ma...@ieee.org> wrote in message
news:1mMW8.453103$352.71255@sccrnsc02...

Mark Nelson

unread,
Jul 10, 2002, 10:31:16 PM7/10/02
to
>Java not reverse engineerable?

Yes, but I don't get to keep a copy of the code. He brings a floppy, inserts
it into the computer, runs his program, creates the output file, removes his
floppy, turns off the PC, and leaves.

On a non-hacked machine he's safe.

So all you have to do to make the test safe is find a way to guarantee the
machine is non-hacked, which I believe is possible.


--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|

"Chris" <re...@removethis.rebel.com.au> wrote in message
news:Xns92477838E4B73re...@66.40.56.91...

Mark Nelson

unread,
Jul 10, 2002, 10:32:47 PM7/10/02
to
"John" <som...@nothing.com> wrote in message
news:10262918...@assimilated.linearg.com...
>Errr... how about pkzip?

Well, if you read the challenge, I think you'll decide you really need
pkunzip. Whey don't you donwload a copy of the file and see what size you
can achieve with the data set and pkunzip.exe. Methinks you won't make it.


--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|

"John" <som...@nothing.com> wrote in message
news:10262918...@assimilated.linearg.com...

Chris

unread,
Jul 11, 2002, 8:38:29 PM7/11/02
to
"Mark Nelson" <ma...@ieee.org> wrote in
news:T56X8.462702$352.74269@sccrnsc02:

>>Java not reverse engineerable?
>
> Yes, but I don't get to keep a copy of the code. He brings a floppy,
> inserts it into the computer, runs his program, creates the output
> file, removes his floppy, turns off the PC, and leaves.
>
> On a non-hacked machine he's safe.
>
> So all you have to do to make the test safe is find a way to guarantee
> the machine is non-hacked, which I believe is possible.

So you are assuming he is local to you
or do you pay the airfare?

I'd like to take you up on this challenge

but please I am in Oz

so if you can get me a return ticket please send it to me
Oh and I might as well look around while I am there, so can you make the
period of stay two weeks :)

would be an awful shame if I lost the bet (he he he)

Mark Nelson

unread,
Jul 11, 2002, 10:04:49 PM7/11/02
to
>So you are assuming he is local to you
>or do you pay the airfare?

I think I'll have to say the test is FOB Dallas, Texas, you get here any way
you can. So you're welcome to come from Oz, once you pass the test you'll
undoubtedly get some licensing agreements and can fly back home in your own
747 with John Travolta as the pilot!


--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|

"Chris" <tera...@removethis.rebel.com.au> wrote in message
news:Xns9249683C11ED0re...@66.40.56.91...

John

unread,
Jul 13, 2002, 7:06:12 AM7/13/02
to
How does one convert the "random" [it's not truly random] data into a text
file simply and easily on a PC?

"Mark Nelson" <ma...@ieee.org> wrote in message

news:j76X8.476898$cQ3.40537@sccrnsc01...

SCOTT19U.ZIP_GUY

unread,
Jul 12, 2002, 10:23:34 PM7/12/02
to
som...@nothing.com (John) wrote in
<10265259...@assimilated.linearg.com>:

>How does one convert the "random" [it's not truly random] data into a
>text file simply and easily on a PC?

Thanks for the lead in. You use my latest and greatest.
It will decompress any file into a text file that looks
like english to second order. If you compress back it goes
to your original file.

Mark Nelson

unread,
Jul 15, 2002, 10:47:54 PM7/15/02
to
"John" <som...@nothing.com> wrote in message
news:10265259...@assimilated.linearg.com...

>How does one convert the "random" [it's not truly random] data into a text
>file simply and easily on a PC?

First, you say "it's not truly random."

If it isn't random, you should be able to meet the challenge. By virtually
any definition I've seen in this NG, "not random" means "compressible'. Once
the ASCII text is converted to a binary number it's pretty incompressible as
far as I can tell.

Second, how to convert the data into a text file? A little bit of code, I
suppose. A very little bit on top of the generation of the binary data.


--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|

"John" <som...@nothing.com> wrote in message
news:10265259...@assimilated.linearg.com...

Jules Gilbert

unread,
Jul 17, 2002, 8:26:31 AM7/17/02
to
Hello all:

Some folks who regularly participate in this newsgroup realize that
compression is the flip side of prediction, and to quote a friend of
mine, a scientist at MIT:

"Jules, if you can compress, you can predict".

He said this several years ago, at a time when G. Cormack (see below)
and many other foolish people were beating on me in an effort to get
me to disclose.

(For the uninitiated, supplying someone with a copy of the
decompressor subjects my algorithm (the program implementation) to
reverse engineering by way of disassembling the decompressor load
module.)

Anyway, I thought about that... About prediction. Something I had
not previously thought much of. But he was right. Compression works
precisely because one can characterize (or predict) the next bit in
the input stream. If not with certainty, then with sufficient
probability that it becomes practical to expect one or more 'future'
bits, and to code for their minimization on output.

And so I decided to concentrate on prediction. My compression product
was not going anywhere -- not just because of the nasty people here,
but as I kept saying, that version was clearly too slow to be compete
with practical programs such as ARJ or other well written commercial
versions, some of which are quite good. (With large buffer's though,
it may be practical to use if implemented in hardware.)

Now though, I have a system that is not DSP based. No floating point
operations. NONE. In fact, it could work fine on a Z80 with say, 64k
of RAM. More important, it's FAST.

TRUE -- to produce really spectacular results, it's still necessary to
iterate through several stages, and each one produces a file, the
output file. Which is read by the subsequent compressor stage to
produce a smaller output file, with the prior stage file being
deleted. After several iterations of this, one really does wind up
with a smaller file which, by reversing this process, and assuming no
errors in the intermediate stages, one can reproduce the original
large file.

Ignoring the above (ie., just concentrating on a single stage of the
process, not the many stages), the process is quite speedy!, and is
faster than gzip or ARJ or any other compressor I have access to (I've
actually only compared it to gzip.)

About the 8MB issue: Sorry, those references were to my OLDER
version, a DSP based program based around very different principles.
For superior performance, that program does require substantially
large input buffers.

So now I come to my point: Does this audience consider accurate
predictions (say, of a particular stock's performance tomorrow to be
significant?) If I produce a few predictions of stocks, wherein each
prediction is correct with respect to the direction, will that cause
Gordon V. Cormack to protect future generations of CS student's by
resigning his position?

How about it Gordon?

You would name a day for the following week. My task would be to
produce a list of say 10 stocks, and each would be among the top
earners' that day, using the measure of the CLOSE:OPEN price FOR THAT
SELECTED MARKET DAY. I will only name large US companies, publicly
traded on the NYSE or NASDAQ exchanges.

If I perform as described above you will resign your position the same
day and agree to never work again a s a professor of computer science
or in any other engineering capactity at a college or university.

If I lose, I'll not post here again.

So, Gordon: "Do you feel lucky?"

gvco...@uwaterloo.ca (Gordon V Cormack) wrote in message news:<agg6jk$7ta$1...@speedy.uwaterloo.ca>...

SCOTT19U.ZIP_GUY

unread,
Jul 17, 2002, 9:20:18 AM7/17/02
to

>How about it Gordon?
>
>You would name a day for the following week. My task would be to
>produce a list of say 10 stocks, and each would be among the top
>earners' that day, using the measure of the CLOSE:OPEN price FOR THAT
>SELECTED MARKET DAY. I will only name large US companies, publicly
>traded on the NYSE or NASDAQ exchanges.
>
>If I perform as described above you will resign your position the same
>day and agree to never work again a s a professor of computer science
>or in any other engineering capactity at a college or university.
>
>If I lose, I'll not post here again.
>
>So, Gordon: "Do you feel lucky?"

Lets see if its a fair bet. First of all its a given
you chances most likely will not succeed. But lets make
the very rash assumpution that your a man of your word
on this wager. To be a fair bet or a bet favorable to
your oppenent you have to see the expected value to him.

If he wins and most liekly he will what does he get.
Nothing. No personnel gain whatsoever. So that means
the gain from winning for him is worthless. What does
it lose if he loses. Well the salary of a professor is
very high compared to zero so he has a lot to lose.
So even if you where honest and even through the odds
greatly farvor him its a dumb foolish bet.

So why did you make the bet. I would guess its like
a liberal trick. So in some later post you can gloat
that you must be correst since he did not take you
up on the wager.

But like a liberal the real truth that it wasn't even
close to a fair bet will be ignored. At least thats
how it looks to me.

>
>gvco...@uwaterloo.ca (Gordon V Cormack) wrote in message
>news:<agg6jk$7ta$1...@speedy.uwaterloo.ca>...
>> Jules -
>>
>> Where have you been?
>>
>> Your proposed medicine show hasn't changed since '97. [See
>> comp.compression FAQ, section 9.4]
>>
>> Are you still planning to sue anybody that challenges your claims?
>

s...@nospam.unt.edu

unread,
Jul 17, 2002, 10:11:08 AM7/17/02
to
Jules Gilbert <j...@medg.lcs.mit.edu> wrote:

> He said this several years ago, at a time when G. Cormack (see below)
> and many other foolish people were beating on me in an effort to get
> me to disclose.

> (For the uninitiated, supplying someone with a copy of the
> decompressor subjects my algorithm (the program implementation) to
> reverse engineering by way of disassembling the decompressor load
> module.)

I believe you are the one I talked to about my challenge, and offered
to sign a non-disclosure agreement. That offer still stands....

--
Steve Tate - srt[At]cs.unt.edu | "A computer lets you make more mistakes faster
Dept. of Computer Sciences | than any invention in human history with the
University of North Texas | possible exceptions of handguns and tequila."
Denton, TX 76201 | -- Mitch Ratliffe, April 1992

Earl Colby Pottinger

unread,
Jul 17, 2002, 11:31:15 AM7/17/02
to
j...@medg.lcs.mit.edu (Jules Gilbert) :

> Hello all:

>
> So now I come to my point: Does this audience consider accurate
> predictions (say, of a particular stock's performance tomorrow to be
> significant?) If I produce a few predictions of stocks, wherein each
> prediction is correct with respect to the direction, will that cause
> Gordon V. Cormack to protect future generations of CS student's by
> resigning his position?

I don't! The original claim was that you could compress random appearing
numbers.

Now you claim not to compress them, but to predict them, fair enough. But
the stock markets is rarely random, in the short term value are hard to
predict but they rarely change by large amounts. If IBM is selling for $150
a share today, I know of no change outside WWIII that will cause it to sell
for $50 or $1000 the following day. Trend prediction is a common use of
computers today!

Looks like you are trying to change the test to me!

Earl Colby Pottinger

--
Hydrogen Peroxide Rockets, BePrint, BePrinter, RAMDISK, Cabin Raising,
Camping, BoatBuilding, Girlfriend. What happened to the time?
http://webhome.idirect.com/~earlcp

Gordon V Cormack

unread,
Jul 17, 2002, 1:06:13 PM7/17/02
to
In article <496bb05e.02071...@posting.google.com>,
Jules Gilbert <j...@medg.lcs.mit.edu> wrote:
>Hello all:

>
>How about it Gordon?
>
>You would name a day for the following week. My task would be to
>produce a list of say 10 stocks, and each would be among the top
>earners' that day, using the measure of the CLOSE:OPEN price FOR THAT
>SELECTED MARKET DAY. I will only name large US companies, publicly
>traded on the NYSE or NASDAQ exchanges.
>
>If I perform as described above you will resign your position the same
>day and agree to never work again a s a professor of computer science
>or in any other engineering capactity at a college or university.
>
>If I lose, I'll not post here again.


Jules,

You have nothing of value to offer me, so no bet.

If you can predict stocks, do so. Place your bets with your broker.

Mark Nelson

unread,
Jul 17, 2002, 10:35:44 PM7/17/02
to
Jules, I've been watching your stock picks for some time, and to be honest,
I'm not impressed.

It seems like a very large percentage of the time you will declare victory
based on the following scenario: A stock that you call a winner will wobble
around a trading range of plus or minus 1% over the course of the day. You
declare that since it was up 1% at some time a sell order right then would
earn you 1% during the day.

Now, making trades on that thin a margin is pretty tough, particularly since
you may well not get a sell order through at that price. And all it takes is
one pretty good loser to wipe out a whole bunch of gains. To avoid that you
have to put in a stop-loss order, just like Martha Stewart. And how low do
you put it? -5%? -10%? Make it too close and you'll sell at a loss for a
stock that might make it up later that day.

Not only that, but quite frequently you aren't going to be able to get the
opening price. A stock that is going up when market opens may be impossible
to buy at last night's closing.

Your second victory scenario goes like this: your top pick of the day
finishes down 12%, but it was a crummy day in the market and some disaster
drove the DJIA down 200 points. So you claim victory. That's great, but you
just lost a bunch of money. The correct thing to do would have been to not
buy. But your program can't predict what Greenspan is going to say, can it?

But, if you want to make public predictions, do this. Every day, list the
stocks that you will purchase at opening. List the dollar amounts on your
sell orders as well. Set your purchase price as what you pay at 9:30, not at
opening.

In any case, I dispute the whole notion. I believe that the performance of a
stock on a given day is pretty near independent of its previous
performance - mathematically it looks like a random walk. The factors that
really matter - corporate announcements, financial news, world events,
aren't simulated in your program.

To be really useful, write a program that will predict that best increase in
price over a six-month period. That way you would be able to get real gains,
insted of trying to sniff the thing vapors seen in the daily thermal
fluctuations. But I don't think you can do that.

I hope you don't feel that this naysaying is mean or cruel. I just don't
believe in your method, and nothing I've seen has done even the tiniest bit
to change my mind.

--
|
| Mark Nelson - ma...@ieee.org - http://MarkNelson.us
| DataCompression.info - http://www.datacompression.info
|

"Jules Gilbert" <j...@medg.lcs.mit.edu> wrote in message

news:496bb05e.02071...@posting.google.com...

Message has been deleted
Message has been deleted

danceswi...@gmail.com

unread,
Apr 27, 2020, 2:19:56 PM4/27/20
to
On Monday, July 8, 2002 at 1:11:38 PM UTC-6, Jules Gilbert wrote:
> Hello folks:
>
> I have decided to sell my 'random' compressors.
>
> This is what I mean when I use the word 'random' or
> 'random-appearing'.
>
> 1) My compressor technology ONLY compress'es random-appearing-data.
> It is not able to reduce the filesize of files containing information
> that can be compressed using conventional compressors', say or ARJ or
> GZIP.
>
> (While one can XOR the input of say, a text file, and that will enable
> the material to be compressed, it is much better to first compress
> such a file with a good quality compressor, say an ARJ compressor or
> other high quality CONVENTIONAL compressor.) After all, XOR'ing does
> not reduce filesize. Compression, even conventional compression,
> does!
>
> 2) Only buffer's of substantial size can be compressed. The amount
> of computer time required to compress smaller files, say 64k, is so
> high so as to make the effort impractical.
>
> 3) As the program exists today, it accepts one or more buffers of 8MB
> of previously compressed data and produces smaller buffers as output.
> It does not seem terribly difficult to me for the method to be
> extended to handle ordinary files. In fact, I do this now (not well,
> though).
>
>
> For various reasons, I am interested in restricting the use of my
> products (now only one program) to the United States. Further, I am
> not enamored of the patenting process -- it is really a mechanism for
> managing information disclosure, and very little protection is
> provided from really bad people -- people I would not do business with
> in any case. So I am offering to license my program by means of trade
> secret ONLY.
>
> My program is suited for use with data that is written once and read
> many times, such as movies or other entertainment data, etc. It is
> probably not suitable for data that is written once and read only
> several times unless space is a critical problem constraint.
>
> Our demonstrating process involves two machines, not connected by wire
> or other means. Compression occurs on one machine, the results are
> transferred by floppy and the result is de-compressed on another
> machine. These machines can be inspected by a technician, they are
> ordinary PC's running FreeBSD.
>
> If you represent commercial interests, please contact me to learn
> more.
>
> Sincerely,
> Jules Gilbert

Using the randbetween function in excel produced the target of 1 million "random" integers between 0 - 9, I then ran the stream through a transform, copied the results and pasted them into word to remove spaces and carriage returns and any punctuation's. I then copied it into word pad, saved the file and compressed it using .rar The file compressed down to 246,040 KB. Thinking there was an error I un zipped and restored it without errors, reversed the transform to its original state.

I am not a programmer but worked on this project with a co-worker Kelly D. Crawford Ph.D for eight years day and night. He passed away weeks before it could be finished. I finished, and this was the result. I would like to give him and his family any posthumous credit but I am left thinking this could be an error considering the various steps.

Fibonacci Code

unread,
Feb 7, 2021, 9:18:54 AM2/7/21
to
Neural networks running thru a set of random data will never able to adapt to all data.
Further, you need to keep the neural network smaller than the output + the errors.

So, I could only say I admire your persistency, but only hope if you could use it in a better way, then stuck in perpetual compression.


Regards,
Raymond

Fibonacci Code

unread,
Feb 7, 2021, 9:19:44 AM2/7/21
to
Fibonacci

Evert Pot

unread,
Jan 9, 2023, 2:55:44 PM1/9/23
to
I case you're still curious. If you store a list of numbers in a text file there's a TON of repetition. It's not raw binary data, it's ASCII data for which almost every byte is going be in the range 48-57. Highly compressible even though the numbers themselves might be properly random.

Keith Thompson

unread,
Jan 9, 2023, 3:26:30 PM1/9/23
to
Evert Pot <ev...@badgateway.net> writes:
> On Monday, April 27, 2020 at 2:19:56 PM UTC-4, danceswi...@gmail.com wrote:
[...]
>> Using the randbetween function in excel produced the target of 1
>> million "random" integers between 0 - 9, I then ran the stream
>> through a transform, copied the results and pasted them into word to
>> remove spaces and carriage returns and any punctuation's. I then
>> copied it into word pad, saved the file and compressed it using .rar
>> The file compressed down to 246,040 KB. Thinking there was an error
>> I un zipped and restored it without errors, reversed the transform
>> to its original state.

Some of those numbers must be incorrect. 1 million decimal digits
represented in ASCII would be about 977 kilobytes (assuming a "kilobyte"
is 1024 bytes, not 1000 bytes). If the decimal digits are random, I'd
expect it to compress to about 294 kilobytes *at best*. The reported
size of "246,040 KB" either indicates an error of a factor of at least
1000, or uses "," as a decimal separator. Even if it's 246 KB, that's
much better compression than I'd expect.

>> I am not a programmer but worked on this project with a co-worker
>> Kelly D. Crawford Ph.D for eight years day and night. He passed away
>> weeks before it could be finished. I finished, and this was the
>> result. I would like to give him and his family any posthumous
>> credit but I am left thinking this could be an error considering the
>> various steps.
>
> I case you're still curious. If you store a list of numbers in a text
> file there's a TON of repetition. It's not raw binary data, it's ASCII
> data for which almost every byte is going be in the range
> 48-57. Highly compressible even though the numbers themselves might be
> properly random.

To be precise, a byte (using the most common meaning of the term)
is 8 bits and can can hold any of 256 values. There are only 10
decimal digit values, so in a sequence of decimal digits represented
in ASCII, 246 of the 256 possible byte values are never used.
A very simple compression method, binary-coded decimal, stores each
decimal digit in 4 bits, leaving 6 unused values for each 4-bit unit.
A slightly more sophisticated compression method could use 10 bits
for each b3 decimal digits, using 1000 of 1024 possible values.
Both of these work only for decimal digits.

An ideal compression algorithm could store each decimal digit in
log2(10) bits, or about 3.32 bits per digit, yielding an output about
30.1% the size of the input (plus metadata). A general-purpose
compression algorithm might come reasonably close to that, though
the best I've managed is about 44% with `xz -9`. (I don't have a
rar compressor.) (I used /dev/urandom, not Excel's randbetween,
to generate the input.)

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */
0 new messages