Generalized Image Compression (GIF -> BIF)
Edward Hamrick
overcomes the problems of LZW in GIF and TIFF in a very general
way.
My proposal is in two parts. The first part is that uncompressed
image formats be stored in ZIP files, using pkzip's deflate
compression. For instance,
.TIF -> .TIZ (in MS-DOS, just type "pkzip file.tiz file.tif")
.PCX -> .PCZ
.BMP -> .BMZ
.TGA -> .TGZ (conflicts with .tar.gz mapping, but not a big deal)
.BIF -> .BIZ (see below)
.??? -> .ZIP (viewer has to auto-recognize)
Anybody who doesn't have a .TIZ reader can just unzip it and use
their .TIF reader. It's a clean approach. (The TIFF files would
be baseline TIFF, without any LZW compression.)
The second part of this proposal is that a new format called
BIF (Binary Image Format) be used to replace GIF. This format
is _identical_ to GIF87a/GIF89a except for:
1) The first byte is a "B" instead of a "G".
2) GIF's LZW compressed data is stored as packed bytes
(in the same bit order as the LZW codes). The number
of "Bits Per Sample" is put into the "LZW Code Size"
byte, and the packed pixel data is put into the subsequent
data blocks, followed by the 0 block terminator.
Valid values for "Bits Per Sample" are 1, 2, 4, and 8.
That's all there is to it. Note that transforming a GIF to
a BIF, and then the BIF back to a GIF, results in exactly the
same image (modulo small differences due to LZW compressors).
This is the biggest advantage of this approach. This second
proposal is only useful if the first proposal is implemented
(uncompressed image files within ZIP files).
I am developing a library that will allow easy integration of
the capability to read from a ZIP file. The routines are
zopen(), zread(), zgetc(), zseek(), and zclose(). They have
exactly the same arguments as fopen(), fread(), fgetc(),
fseek(), and fclose(), so they're easy to plug into an existing
viewer.
This is a bit tricky, but it's all ANSI C and very portable.
If the zopen() routine sees a ZIP header, it transparently
decompresses it (on demand) and passes the data back to the calling
application, just as if the caller were reading the uncompressed
file. If the zopen() routine doesn't see the zip header, it
just reads it like a normal image file. From the application's
point of view, it is always just reading uncompressed files.
This code will be freely available, with the same kind of copyright
notice as Sam Leffler's TIFF library (basically, it's freely usable
by anybody as long as it's use is acknowledged).
The core inflate.c routine is 6.5K of code, and 320 bytes of
static data. I don't have the measurements yet on the zxxx()
routines, but I expect they'll add another 2K. I plan to
finish debugging the inflate.c routine today, and then complete
the zxxxx() routines by the end of this week.
I could use some help on this. I need a volunteer to produce
a freeware gif2bif.c and bif2gif.c. As part of the BIF and
embedded ZIP spec, I plan to have a section containing a list
of people who endorse this approach, and the names of the
programs they commit to supporting ZIP embedding and BIF with.
If you like the idea, please e-mail me a one-line endorsement with:
your name, company name [name of software products]
I don't support PNG because it isn't invertible with GIF, i.e.
gif2png followed by png2gif doesn't get the original GIF file
back. This is a fatal flaw in my opinion. How many BBS
operators are going to be willing to run a non-invertible
program on all their GIF files?
The zip-compressed BIF files will always be at least as small
as the comparable PNG file, and usually 10% smaller than the
original GIF file. Programs that process zip-compressed BIF
files will also be able to read zip-compressed TIFF, TGA (Targa),
BMP, and PCX files. I will make freely usable code available
to do all this.
Jump on board the bandwagon, it'll be fun.
Regards,
Ed Hamrick
Lee Daniel Crocker
>I don't support PNG because it isn't invertible with GIF, i.e.
>gif2png followed by png2gif doesn't get the original GIF file
>back. This is a fatal flaw in my opinion. How many BBS
>operators are going to be willing to run a non-invertible
>program on all their GIF files?
You are mistaken about this. I have already designed my
png2gif program to store the extra PNG data in a GIF extension
that is read by the gif2png program. 100% lossless both ways.
Of course, only in native PNG format will you will get the
benfit of the extra capabilities, simplicity, and better
compression (probably better than your "BIF" because of the
improved spatial coherence and prediction before zipping).
--
Lee Daniel Crocker /o)\ "When people are influenced by rulers,
lcro...@netcom.com \(o/ they do what the rulers do, not what
finger for PGP key they say."--Masters of Huainan
Jeremy Wohl
Edward Hamrick <ham...@Primenet.Com> wrote:
>
> [munch, munch]
>
>The second part of this proposal is that a new format called
>BIF (Binary Image Format) be used to replace GIF. This format
>is _identical_ to GIF87a/GIF89a except for:
>
I already released the Free Graphics Format (FGF) near the beginning
of January. It looks like a GIF file except for zip-compressed
image data. Has a 24-bit truecolor extension to boot. Source + utils.
Entirely compatible (sans image data).
Check out <ftp://ftp.cs.sunysb.edu/pub/fgf/>.
HOWEVER, I was persuaded that the Unisys fiasco was a grand opportunity
to create a more efficient, streamlined format for a broader class
of image types. Thus PNG.
-jeremy
Edward Hamrick
Lee Daniel Crocker <lcro...@netcom.com> wrote:
>ham...@Primenet.Com (Edward Hamrick) writes:
>>I don't support PNG because it isn't invertible with GIF, i.e.
>>gif2png followed by png2gif doesn't get the original GIF file
>>back. This is a fatal flaw in my opinion. How many BBS
>>operators are going to be willing to run a non-invertible
>>program on all their GIF files?
>
>You are mistaken about this. I have already designed my
>png2gif program to store the extra PNG data in a GIF extension
>that is read by the gif2png program. 100% lossless both ways.
You can't recover the subimage data this way, and there's
no plan to store them in PNG. I guess you could just store
the whole GIF image in a PNG chunk, and that would make it
invertible <smile>, but it sort of defeats the whole purpose
of PNG.
>Of course, only in native PNG format will you will get the
>benfit of the extra capabilities, simplicity, and better
>compression (probably better than your "BIF" because of the
>improved spatial coherence and prediction before zipping).
"Spatial coherence and prediction?" Nonsense. The compression
of BIF will be almost identical to PNG, and probably a little
better because the palette and GIF extension blocks will be
compressed in BIF as well. PNG only compresses the image
data, which is another weakness (although for the life of
me I can't see why the PNG group doesn't just encapsulate
it within a ZIP format).
It'll be about 5 lines of code to modify an existing GIF
decoder to read BIF as well, using the zipio.h package
I'm working on. The zipio.h is a direct replacement for
stdio.h, and can be plugged right in to existing GIF
viewers.
I've uploaded the first version of zipio.zip to:
ftp://ftp.primenet.com/users/h/hamrick/zipio/zipio.zip
It includes:
dezip1.c - sample program for reading compressed zip file
inflate.c - general purpose inflate decompression routine
crc.c - CRC-32 computation routine
inflate.h - interface for inflate.c
crc.h - interface for crc.c
zipio.h - zip I/O routine for replacing stdio.h
I hope to get zipio.c working today, and will make
it available when it's working.
Regards,
Ed Hamrick
Tom Lane
> I don't support PNG because it isn't invertible with GIF
Um, Ed, who the hell cares?
GIF is dead. It hasn't quite stopped moving yet, but it's dead.
No one will care about converting PNG back to GIF.
regards, tom lane
organizer, Independent JPEG Group
Oliver Fromme (TBH)
Edward Hamrick (ham...@Primenet.Com) wrote (7 Feb 1995 19:08:18 GMT):
> My proposal is in two parts. The first part is that uncompressed
> image formats be stored in ZIP files, using pkzip's deflate
> compression. For instance,
That would make using the files a pain.
Just one example: There is a utility GIFCMD.COM which
can add and extract comments from GIF files. Its size
is 2.5 Kb, and it is _very_ fast. It can extract the
comments from about 200 GIF files in a directory in about
3 seconds. On a slow 386SX.
Using your approach to wrap the entire uncompressed file
into a ZIP file would have the following result: The size
of the utility would be at least 20 Kb (provided that your
statement about 6.5 Kb code size for inflate is correct,
and the deflate would be probably larger than that).
To extract the comments of 200 "BIZ" files, it would have
to read and decompress the entire files. It will take
_MINUTES_ at least. Compare to 3 seconds. Great.
> Anybody who doesn't have a .TIZ reader can just unzip it and use
> their .TIF reader. It's a clean approach. (The TIFF files would
> be baseline TIFF, without any LZW compression.)
How are you going to prevent anybody from using
compressed (LZW or something else) TIFF files?
> Valid values for "Bits Per Sample" are 1, 2, 4, and 8.
GIF also allows 3, 5, 6 and 7. So your BIF is not
fully invertible with GIF. I've seen several 3 and 6
bit GIFs, and even a few 5 and 7 bit GIFs.
> The core inflate.c routine is 6.5K of code, and 320 bytes of
> static data.
What about deflate? A command adding or palette modifying
utility would need that, too (unlike GIF or PNG).
> I don't support PNG because it isn't invertible with GIF, i.e.
> gif2png followed by png2gif doesn't get the original GIF file
> back.
About 99.9% of all GIF files _are_ invertible (those which
don't contain multiple images). Multiple image GIFs could
be converted to MPNG (multiple image extension of PNG),
so there's no problem either.
> The zip-compressed BIF files will always be at least as small
> as the comparable PNG file
Wrong. PNG uses additional filters (predictors) which can
improve compression by up to 50%, compared to normal
compression (like pkzip)!
> Jump on board the bandwagon, it'll be fun.
Maybe it's fun, but it seems useless to me.
I will _not_ support BIF or BIZ.
PNG is the way to go.
Regards,
Oliver Fromme, author of QPEG
--
Oliver Fromme, Leibnizstr. 18-61, 38678 Clausthal, Germany
Internet/Usenet email address: fro...@rz.tu-clausthal.de
WWW server: http://www.rz.tu-clausthal.de/~inof/Welcome.html
Edward Hamrick
Tom Lane <t...@netcom.com> wrote:
>ham...@Primenet.Com (Edward Hamrick) writes:
>> I don't support PNG because it isn't invertible with GIF
>
>Um, Ed, who the hell cares?
>
>GIF is dead. It hasn't quite stopped moving yet, but it's dead.
>No one will care about converting PNG back to GIF.
I thought it was obvious, but I'll restate it. Non-invertibility
implies data loss. I'm glad you admit that gif->png->gif loses
data, since many people argue that this isn't true. Many people
won't care about this data loss, and some will.
GIF isn't actually dead - when I use WWW all I see are GIF
files and a few JPEG files.
What I think you meant to say is that you believe there will be
a rapid move from GIF to PNG, both on WWW and online services.
There are many factors working against this. I'll use two
examples, but could just as easily come up with more.
Imagine you sell a WWW browser that decodes GIF files. You're
trying to compete with netscape, and you have a technically
better product than netscape. You have two independant decisions
to make - whether to take out GIF, and whether to add PNG.
You can't sell your program today without GIF, so you put GIF
support into it. Now you have to make the decision whether
to add PNG - let's say you add it, but some of your competitors
don't add it.
Imagine you're the author of a Web page. You have to decide
whether to include GIF graphics, PNG graphics, or JPEG
graphics (for the little buttons and other larger graphics).
You know that most existing Web viewers can display GIF,
some display PNG, and some display both (but all display
JPEG). What are you going to do? The rational decision
is to use JPEG for most graphics files, and maybe leave
the unimportant ones as GIF (and hope viewers that don't
support GIF just display a little box).
Now imagine you're a customer, looking to buy a Web
browser. You know that some Web pages will have GIF,
some will have JPEG, and some will have PNG (and some
may have .BMP, some .TGA, some .TIF). You have to
decide which Web browser you want to buy with your $40.
Faced with the decision between buying a Web browser
with GIF costing $40.50, and one that doesn't have
GIF that costs $40, what will your decision be? Even
if you know only 1 in 100 Web pages you'll access
will have a GIF file, you'll still buy the one that
has GIF support, all other things being equal.
In fact, customers will be extremely biased towards
buying Web browsers with GIF, and will be quite
unwilling to buy Web browsers without GIF.
If there's a good chance that most Web browsers will
support GIF, then there's a good chance that people
writing Web pages will continue to use GIF graphics -
it's a vicious cycle.
My main point about PNG is that it doesn't go far
enough to break this vicious cycle. Imagine if PNG
were simply BIF with extension blocks. (BIF is
GIF with the first "G" replaced with a "B", and
the packed pixel data being put into the same format
as the current LZW codes.) All the new PNG info
could be put into the GIF extension blocks. Existing
GIF readers could continue (with about 5 lines of
new code) to read BIF files.
The main power of this approach is that an 8-bit
GIF application can continue to read BIF files,
even BIF files with 24-bit data in the extension
blocks. The 8-bit BIF (uncompressed) would _be_
the thumbnail! The zip wrapper would be the
compression.
The approach of using optional zip wrappers for all
uncompressed image formats is orthogonal to the
BIF concept (if BIF were done like PNG, I'd just
recommend replacing the LZW codes with deflate
codes).
The upward compatibility story here is powerful,
and I'm surprised you just don't get it. The whole
reason of pushing BIF is that it's an easier
sales job for replacing GIF, since it's easier to
convince people that it's really the same as GIF.
As you can tell from the above discussion, I'm pretty
sure that even if every person you or I have ever
exchanged e-mail with supported PNG (or BIF), that
it's still unlikely that either PNG (or BIF) would
take over for GIF. It's simple economics. I happen
to think that BIF has a better chance than PNG, but
I still think it's a 1 in 100 longshot.
It's like convincing an entire country to voluntarily
switch from driving on the left hand side of the road
to the right hand side. Those who change sides
prematurely get in wrecks (those who remove GIF support
from their programs). Those who try to convince the
world that you need to repave all the roads in the
process (add 24-bit lossless) just get laughed at.
Replacing GIF is a big enough job as it is. BIF means
just switching sides of the road. We don't need to
repave the roads at the same time.
Regards,
Ed Hamrick
P.S. I actually hope my competitors drop GIF support
from their applications - it'll just mean more
money for me <smile>. Why do you think I've
been working so hard on freely usable PNG source
code <grin>.
Edward Hamrick
Oliver Fromme (TBH) <in...@obelix.rz.tu-clausthal.de> wrote:
>Edward Hamrick (ham...@Primenet.Com) wrote (7 Feb 1995 19:08:18 GMT):
> > My proposal is in two parts. The first part is that uncompressed
> > image formats be stored in ZIP files, using pkzip's deflate
> > compression. For instance,
>That would make using the files a pain.
>Just one example: There is a utility GIFCMD.COM which
>can add and extract comments from GIF files. Its size
>is 2.5 Kb, and it is _very_ fast. It can extract the
>comments from about 200 GIF files in a directory in about
>3 seconds. On a slow 386SX.
200 GIF files in 3 seconds is 66 files per second. This
means it can read about one file per disk rotation in this
example, and I suspect it actually isn't CPU-bound, but
rather is limited by disk rotation speed.
The encapsulated GIF code decompresses at 400 KByte/sec,
which results in about 5 milliseconds of CPU time for
1 16K block. It takes a disk 16 milliseconds to rotate,
and even longer to seek (which is typical when reading
200 files - each normally requires a seek).
This 5 millisecond overhead just isn't significant
in this kind of application because it is seek-limited
not CPU limited.
>Using your approach to wrap the entire uncompressed file
>into a ZIP file would have the following result: The size
>of the utility would be at least 20 Kb (provided that your
>statement about 6.5 Kb code size for inflate is correct,
>and the deflate would be probably larger than that).
Gee, adding another 10K of space really makes life tough
for you <smile>. Haven't you noticed memory prices?
>To extract the comments of 200 "BIZ" files, it would have
>to read and decompress the entire files. It will take
>_MINUTES_ at least. Compare to 3 seconds. Great.
Run pkunzip -t on a .ZIP file with 200 small files -
it just doesn't take that long.
> > Anybody who doesn't have a .TIZ reader can just unzip it and use
> > their .TIF reader. It's a clean approach. (The TIFF files would
> > be baseline TIFF, without any LZW compression.)
>How are you going to prevent anybody from using
>compressed (LZW or something else) TIFF files?
I wouldn't _prevent_ anyone from putting _anything_ into these
encapsulated zip files. I added the zipio.c library to my viewer
(VuePrint) yesterday. It now automatically detects this zip
wrapper on any GIF, BMP, PCX, TGA, JPG, or TGA file.
I can even transparently read a JPEG file, encapsulated in a
Macintosh PICT format, encapsulated in a zip file. I can read
an LZW-compressed GIF file encapsulated in a zip file. I can
read an LZW-compressed TIFF file encapsulated in a zip file.
It all works - I just replaced <stdio.h> with "zipio.h" in
all the image reading programs, and replaced fopen with
zopen, fread with zread, fseek with zseek, ftell with ztell,
getc with zgetc, and fclose with zclose. That's it - and
it all works. One hour's work.
The performance when reading a file that doesn't have the
zip wrapper is the same as before. zgetc is just as fast
as getc, and zread is just as fast as fread.
> > Valid values for "Bits Per Sample" are 1, 2, 4, and 8.
>GIF also allows 3, 5, 6 and 7. So your BIF is not
>fully invertible with GIF. I've seen several 3 and 6
>bit GIFs, and even a few 5 and 7 bit GIFs.
You're right, since I wrote the previous e-mail, I realized that
since applications already support unpacking LZW codes of
any size up to 8 bits, they might as well support packed
pixels of any size up to 8 bits.
> > The core inflate.c routine is 6.5K of code, and 320 bytes of
> > static data.
>What about deflate? A command adding or palette modifying
>utility would need that, too (unlike GIF or PNG).
I haven't done deflate.c yet. It's probably another
10K of code. What are your measurements of deflate.c?
> > I don't support PNG because it isn't invertible with GIF, i.e.
> > gif2png followed by png2gif doesn't get the original GIF file
> > back.
>About 99.9% of all GIF files _are_ invertible (those which
>don't contain multiple images). Multiple image GIFs could
>be converted to MPNG (multiple image extension of PNG),
>so there's no problem either.
Yes, most are invertible. However, on CompuServe there are
30,333 GIF files (I checked today). If only 0.5% aren't
invertible, this is only 151 files. I agree it's no big deal.
> > The zip-compressed BIF files will always be at least as small
> > as the comparable PNG file
>Wrong. PNG uses additional filters (predictors) which can
>improve compression by up to 50%, compared to normal
>compression (like pkzip)!
PNG doesn't use filters on palette-compressed data. Comparing
BIF files with PNG files is comparing 8-bit palette data only,
so your argument about filters is irrelevant.
> > Jump on board the bandwagon, it'll be fun.
>Maybe it's fun, but it seems useless to me.
>I will _not_ support BIF or BIZ.
>PNG is the way to go.
There has been a lot of discussion lately about how important
24-bit lossless images are. Many people have indicated that
they think this is important, and the PNG file format is being
largely driven by the need to add a 24-bit lossless image
format.
I got some statistics that I think shed a very interesting
light on the importance of 24-bit lossless images. I used
CompuServe's Graphics File Finder (GO GRAPHFF) to get
statistics on how many files of what type are on-line.
Of a total of 44,942 files that were in all the Graphics
forums on CompuServe, 34,303 have an extension of GIF, BMP,
PCX, TGA, JPG, or TIF. 4515 of these files have an extension
of ZIP.
Of the 34,303 files with a specific image file extension,
here's the distribution:
GIF 30,333 88.43%
JPG 3,867 11.27%
BMP 58 0.17%
TIF 25 0.07%
PCX 12 0.03%
TGA 8 0.02%
Of the 4,515 files with a ZIP extension, the following had
specific image file keywords, although inspection shows that
fewer than 1 in 10 with the keyword were zip-encapsulated
image files:
GIF 250
TGA+TARGA 177
PCX 144
BMP 128
TIF+TIFF 104
JPG 36
I actually only found one zipped image file (a PCX file),
but the above numbers establish an upper bound on the number
of 24-bit lossless images on CompuServe.
There are some obvious conclusions you can draw from this
large sample of raw data:
1) 99.5% of the image files on CompuServe are GIF and JPEG
2) 0.5% of of the image files on CompuServe are 24-bit lossless
3) Any new image format to replace GIF should focus very strongly
on GIF replacement, and less strongly on 24-bit lossless features.
I believe that the above statistics are clear evidence that
using the BIF format is a better approach to replacing GIF than PNG.
Some may differ, but the arguments advanced for PNG are all related
to PNG's 24-bit lossless features. I hope the above statistics
put an end to this argument.
If the PNG proponants have any other statistics (i.e. WWW or BBS
statistics), please post them. If the PNG proponants have any
statistics showing a trend towards a rapid increase in the use of
24-bit lossless images, please post them. In fact if PNG proponants
have statistics that show anything other than the trend towards JPEG
replacing GIF, please post them.
Regards,
Ed Hamrick
Oliver Fromme (TBH)
> Oliver Fromme (TBH) <in...@obelix.rz.tu-clausthal.de> wrote:
> >Just one example: There is a utility GIFCMD.COM which
> >can add and extract comments from GIF files. Its size
> >is 2.5 Kb, and it is _very_ fast. It can extract the
> >comments from about 200 GIF files in a directory in about
> >3 seconds. On a slow 386SX.
>
> 200 GIF files in 3 seconds is 66 files per second. This
> means it can read about one file per disk rotation in this
> example, and I suspect it actually isn't CPU-bound, but
> rather is limited by disk rotation speed.
Of course I assumed that a disk cache with read-ahead is
installed, like Smartdrive or Hyperdisk.
To read the comment from a GIF file, normally only the
first cluster (2 Kb) has to be read.
> The encapsulated GIF code decompresses at 400 KByte/sec,
On which type of CPU? Certainly _not_ on a 386SX.
Do you require people to have a Pentium?
> which results in about 5 milliseconds of CPU time for
> 1 16K block. It takes a disk 16 milliseconds to rotate,
Normally less than that. Do you have a hard disk which
rotates at 3600 rpm? Must be pretty old.
> >Using your approach to wrap the entire uncompressed file
> >into a ZIP file would have the following result: The size
> >of the utility would be at least 20 Kb (provided that your
> >statement about 6.5 Kb code size for inflate is correct,
> >and the deflate would be probably larger than that).
>
> Gee, adding another 10K of space really makes life tough
> for you <smile>. Haven't you noticed memory prices?
If every small utility grows by 20 Kb -- _that_ makes
life tough indeed.
> >To extract the comments of 200 "BIZ" files, it would have
> >to read and decompress the entire files. It will take
> >_MINUTES_ at least. Compare to 3 seconds. Great.
> Run pkunzip -t on a .ZIP file with 200 small files -
> it just doesn't take that long.
Run pkunzip -t on 200 ZIP files, each about 200 Kb.
It will take _that_ long. 200 Kb is a pretty normal
size for GIF files.
> I wouldn't _prevent_ anyone from putting _anything_ into these
> encapsulated zip files. I added the zipio.c library to my viewer
> (VuePrint) yesterday. It now automatically detects this zip
> wrapper on any GIF, BMP, PCX, TGA, JPG, or TGA file.
If you are really that money-centric as you said,
you will add PNG support anyway. Everybody is going
to add PNG support to his/her software.
If you don't jump on the train, you'll be left alone
on your "BIF" island. And you'll be certainly unable
to stop the train. So why are you still trying?
So far I haven't heard a _single_ reasonable argument
from you why PNG is that bad.
> > > The zip-compressed BIF files will always be at least as small
> > > as the comparable PNG file
> >Wrong. PNG uses additional filters (predictors) which can
> >improve compression by up to 50%, compared to normal
> >compression (like pkzip)!
>
> PNG doesn't use filters on palette-compressed data. Comparing
> BIF files with PNG files is comparing 8-bit palette data only,
> so your argument about filters is irrelevant.
First, my argument _is_ very relevant to 24 bit images.
Second, you can not say that "PNG doesn't use filters
on palette-compressed data". True, the 7th Draft doesn't
allow it, but that might change. For example, SUB or XOR
might help on many color mapped images.
> There has been a lot of discussion lately about how important
> 24-bit lossless images are. Many people have indicated that
> they think this is important, and the PNG file format is being
> largely driven by the need to add a 24-bit lossless image
> format.
>
> [statistics about CompuServe deleted]
Your statistics are irrelevant. Compuserve is an online
service, which most people use over slow modem lines.
Therefore file should be as small as possible, and JPEG
is very well suited for that job. Most CIS users don't
need lossless images.
But there are _many_ people who do need lossless 24 bit.
If you are going to edit, crop or rescale an image, you
need to store it lossless, because each change to the
image requires a decompression/compression cycle, and
with JPEG each such cycle is lossy and degrades the
quality of the image.
We _do_ need a standard which supports 24 bit truecolor
with appropriate _lossless_ compression.
PNG will be that standard. Many software authors and
companies (including CompuServe and all the major Web
browsers) have already acknowledged support of PNG.
I don't want to prevent you from going your own "BIF"
way. It's your choice. Good luck.
Regards,
Oliver Fromme
Edward Hamrick
Oliver Fromme (TBH) <in...@majestix.rz.tu-clausthal.de> wrote:
>Edward Hamrick (ham...@Primenet.Com) wrote (10 Feb 1995 15:33:51 GMT):
>installed, like Smartdrive or Hyperdisk.
>To read the comment from a GIF file, normally only the
>first cluster (2 Kb) has to be read.
Read-ahead doesn't work generally when reading through a
number of files - a seek will almost always be needed to
get the first block of each file.
If you only need to read the first 2 KB of a GIF file
to get the comment, then you only need to read the first
2 KB of a BIF file to get the comment. My zipio.c
library only compresses the minimal amount necessary
to satisfy a read request. I finished integrating this
library with VuePrint yesterday, and made it publicly
available on
ftp://ftp.primenet.com/users/h/hamrick/zipio/zipio.zip
You can do this testing yourself if you don't believe me.
> > The encapsulated GIF code decompresses at 400 KByte/sec,
>On which type of CPU? Certainly _not_ on a 386SX.
>Do you require people to have a Pentium?
Go ahead and measure the performance of zipio.c - you
may be surprised. Make sure to compile with optimization
and to add -DMEMCPY to the compilation line.
> > which results in about 5 milliseconds of CPU time for
> > 1 16K block. It takes a disk 16 milliseconds to rotate,
>Normally less than that. Do you have a hard disk which
>rotates at 3600 rpm? Must be pretty old.
Even 4500 rpm disks take 13 milliseconds to rotate.
> > Gee, adding another 10K of space really makes life tough
> > for you <smile>. Haven't you noticed memory prices?
>
>If every small utility grows by 20 Kb -- _that_ makes
>life tough indeed.
Disks cost $0.50 per MByte nowdays. If you take the
number of utilities likely to need this additional 10K
of space (use your own assumptions), you'll need
200 of these utilities to even cost you a dollar.
I doubt that you can make a convincing case that you
currently even have 10 different programs that read
GIF files withough reading the LZW data.
> > >To extract the comments of 200 "BIZ" files, it would have
> > >to read and decompress the entire files. It will take
> > >_MINUTES_ at least. Compare to 3 seconds. Great.
>
> > Run pkunzip -t on a .ZIP file with 200 small files -
> > it just doesn't take that long.
>Run pkunzip -t on 200 ZIP files, each about 200 Kb.
>It will take _that_ long. 200 Kb is a pretty normal
>size for GIF files.
Note tha above argument - zipio.c only reads the minimum
necessary of the BIF file, and you stated that only the
first 2K would need to be read on average. The disk
latency clearly dominates.
> > I wouldn't _prevent_ anyone from putting _anything_ into these
> > encapsulated zip files. I added the zipio.c library to my viewer
> > (VuePrint) yesterday. It now automatically detects this zip
> > wrapper on any GIF, BMP, PCX, TGA, JPG, or TGA file.
>If you are really that money-centric as you said,
>you will add PNG support anyway. Everybody is going
>to add PNG support to his/her software.
>If you don't jump on the train, you'll be left alone
>on your "BIF" island. And you'll be certainly unable
>to stop the train. So why are you still trying?
I'm planning on making money with BIF, you just don't
see how yet. I've posted a description on CompuServe's
format discussion area, titled GIF on CD-ROM.
A short-summary - Unisys is saying they're going to
charge CD-ROM vendors a fairly substantial percentage
of the gross selling price of CD-ROM's that contain
commercial, shareware, or _freeware_ gif viewers.
I will have a product that CD-ROM vendors can bundle
with their CD-ROM's (VuePrint 3.6 Zip/16 Edition)
that views zipped BIF files, zipped TIF files (without
LZW compression), and JPEG files. This product is
already working, and I'll release it in the next two
weeks (time is money <smile>)
I'll also make public bif2gif and gif2bif programs,
including freely usable source code. CD-ROM vendors
can offer several ways of distributing these freeware
converters for those customers who want to be able
to use the files on the CD-ROM as GIF files. Unisys
prevents them from distributing these bif2gif and
bif2gif utilities on the CD-ROM itself, but the
CD-ROM vendor can let people know where to find
these utilities.
>So far I haven't heard a _single_ reasonable argument
>from you why PNG is that bad.
PNG isn't all that bad, and if it becomes popular, I'll
add it to VuePrint. However, the way I decide when to
add new formats is when 10 different customers ask for
me to do so - I prioritize all development based on
what customers want (and CD-ROM vendors are customers).
>Your statistics are irrelevant. Compuserve is an online
>service, which most people use over slow modem lines.
>Therefore file should be as small as possible, and JPEG
>is very well suited for that job. Most CIS users don't
>need lossless images.
You may live in an ideal world where you don't pay anything
for the time you're on-line, but the target market for a
new image format is people who are paying per-hour charges.
I know that you're lucky since you have a university account
and don't pay fees for transferring data, but this is the
exception, not the rule.
If you have to pay per-hour charges to transfer images, you're
less likely to use 24-bit lossless images, but instead to
have a strong preference for using JPEG. The CompuServe
statistics clearly show this - 24-bit lossless images just don't
get used when you have to pay for connect time.
>But there are _many_ people who do need lossless 24 bit.
>If you are going to edit, crop or rescale an image, you
>need to store it lossless, because each change to the
>image requires a decompression/compression cycle, and
>with JPEG each such cycle is lossy and degrades the
>quality of the image.
That's why all major image editing programs support TIFF.
ZIP-encapsulated TIFF is actually a nice solution, and
people can freely use baseline (i.e. non-LZW) TIFF files
to exchange images - just zip them, send them, and then
unzip them.
>We _do_ need a standard which supports 24 bit truecolor
>with appropriate _lossless_ compression.
>PNG will be that standard. Many software authors and
>companies (including CompuServe and all the major Web
>browsers) have already acknowledged support of PNG.
TIFF has been a standard 24-bit truecolor format for
years. Every major image editing program supports it.
You just don't find TIFF files on services where people
have to pay to transfer the images - they're just too
big. You won't find 24-bit PNG files on these services
either.
>I don't want to prevent you from going your own "BIF"
>way. It's your choice. Good luck.
Ich hoffe dass Sie auch viel Gluck haben.
Gruess aus Phoenix,
Ed Hamrick
Edward Hamrick
Guy Eric Schalnat <gro...@interserv.com> wrote:
>As we all consider BIF as a GIF replacement, I am concerned about the
>acceptability of BIF in the long term. One of the things the PNG group
>has been very careful of is getting feedback from those people who
>currently use GIF. We have been talking to CompuServe, to the Mosaic
>people, to developers of freeware and shareware products, to other
>companies which utilize GIF, and to users, asking what they wanted,
>what they needed, and what they would accept in a GIF replacement.
>While it is impossible to make everyone happy, awareness of the needs
>and desires of everyone currently using GIF will help make PNG accepted
>across the industry.
BIF has it's place, and because it's easy to convert gif<->bif, there's
no risk to using it. The converse isn't true - complete convertibility
of gif<->png can't be shown, especially with the current PNG spec. I
know there are a lot of people saying "trust me, it's easy, it's no
problem, it's invertible", and others (like Tom Lane) saying "it doesn't
matter that gif<->png isn't invertible" (I'm paraphrasing, I think his
actual words were something like "who the hell cares" <smile>).
BIF is demonstrably easier to integrate with existing GIF applications.
For those who may have missed the earlier thread explaining what BIF
is, it's just GIF with the following changes:
1) instead of the first three bytes being "GIF" they're "BIF"
2) interpret the LZW codes as raw pixel values
3) optionally zip this file
I've released a software library that makes reading the first
file in a zip file transparently. To use it, just replace the
#include <stdio.h> with #include "zipio.h", and replace calls
to fopen(), fread(), getc(), fseek(), ftell(), and fclose with
calls to zopen(), zread(), zgetc(), zseek(), and ztell(), and zclose().
This library automatically recognizes whether a file has a zip
wrapper or not, and hides the details of reading uncompressed
data from an optionally zipped file.
I've already integrated this library with my VuePrint GIF/JPEG viewer,
and can now read any kind of zipped image file, including TIFF, JPEG,
GIF, TGA, and BMP. This week I plan to add the logic for reading and
writing BIF files, and I'm going to throw in an unzip option in the
File menu while I'm at it (the zipio library only needs one more
routine to make this work).
The zipio source code is all very portable ANSI C, and is freely
usable by any application. It can be found with anonymous ftp on:
ftp://ftp.primenet.com/users/h/hamrick/zipio/zipio.zip
>Has Ed done this? Does he offer any reason to believe that everyone
>will accept his proposal? Sure, it may sound good to you, but does it
>meet everyone's needs? Some of these groups have needs that are not
>well met by the current form of GIF. If they have to add code anyway,
>they'd rather add code that meets their needs.
I like the "Sure, it may sound good to you" part. It sounds good to
a lot of people.
Your argument for PNG appears to be that "We, the cognitive elite,
have decided that there shall be PNG, and you peons should be more
deferential to our wisdom. Everybody who's important has signed on
to PNG, and those who disagree with us are oviously not important."
>Some examples: CompuServe wants a 24 bit, streamable format that you
>can put comments into. They also want a format that current GIF's can
>be converted too without any loss of data. BIF doesn't do this
>(not 24 bit). Compressed TARGA doesn't do this (no extension blocks,
>no multiple images). Compressed TIFF doesn't do this (not streamable).
>In order to meet CompuServe's requirements, you would have to implement
>at least two formats. They will find this unacceptable.
There has been a lot of discussion lately about how important
24-bit lossless images are. Many people have indicated that
they think this is important, and the PNG file format is being
largely driven by the need to add a 24-bit lossless image
format.
I got some statistics that I think shed a very interesting
of 24-bit lossless images on CompuServe. Some people claim
that the Graphics File Finder doesn't find a mythical cache
of 24-bit lossless images on CompuServe, but when pressed
have been unable to point out where they are.
There are some obvious conclusions you can draw from this
large sample of raw data:
1) 99.5% of the image files on CompuServe are GIF and JPEG
2) 0.5% of of the image files on CompuServe are 24-bit lossless
3) Any new image format to replace GIF should focus very strongly
on GIF replacement, and less strongly on 24-bit lossless features.
I believe that the above statistics are clear evidence that
using the BIF format is a better approach to replacing GIF than PNG.
Some may differ, but the arguments advanced for PNG are all related
to PNG's 24-bit lossless features. I hope the above statistics
put an end to this argument.
Some argue that the reason you don't find more 24-bit lossless
images on CompuServe is that there isn't much software available
for 24-bit lossless images, and that there isn't wide availability
of 24-bit display hardware. Both arguments are hogwash - compressed
TIFF for storing 24-bit lossless images has been widely available
for years, and all major image editing and viewing programs support it.
Hardware for 24-bit and 16-bit color displays has been widely available for
years.
The reason you don't find 24-bit lossless images on CompuServe is
simple - it just costs too much and takes too long to download them
(most people pay for their download time). This is the same
reason you won't in the future find many 24-bit lossless PNG files
on CompuServe, no matter how much CompuServe pushes 24-bit lossless images.
Since 24-bit lossless images are demonstrably unimportant for
a GIF replacement format, and since BIF is simpler to implement, and
since invertibility of gif<->xxx is important, BIF is a better way to go.
If the PNG proponants have any other statistics (i.e. WWW or BBS
statistics), please post them. If the PNG proponants have any
statistics showing a trend towards a rapid increase in the use of
24-bit lossless images, please post them. In fact if PNG proponants
have statistics that show anything other than the trend towards JPEG
replacing GIF, please post them.
It isn't good enough to say that you're wise and all knowing, and
are sure 24-bit lossless images will become oh-so-important on
WWW or on-line services.
Regards,
Ed Hamrick
Lee Daniel Crocker
>BIF has it's place, and because it's easy to convert gif<->bif, there's
>no risk to using it. The converse isn't true - complete convertibility
>of gif<->png can't be shown, especially with the current PNG spec. I
>know there are a lot of people saying "trust me, it's easy, it's no
>problem, it's invertible", and others (like Tom Lane) saying "it doesn't
>matter that gif<->png isn't invertible" (I'm paraphrasing, I think his
>actual words were something like "who the hell cares" <smile>).
The day after the PNG spec is finalized, I will release my GIF2PNG
program (already 90% complete), which is 100% losslessly invertable.
This is not speculation or opinion, it is an unambiguous statement
of my intention.
You are correct that Tom doesn't care about this issue, but I do, and
I've described my intentions more than once. Your continued insistence
that GIF->PNG is non-invertable is either stubbornly ignorant or
maliciously deceptive.
Edward Hamrick
>I've described my intentions more than once. Your continued insistence
>that GIF->PNG is non-invertable is either stubbornly ignorant or
>maliciously deceptive.
It wouldn't be the first time that I've been proven either stubborn or
ignorant <smile>. I look forward to seeing your gif2png and png2gif
source code, and seeing how you handle the sub-images in GIF89a.
Regards,
Ed Hamrick
erik jensen
optimized to compress non-8-bit data? (and would it compress
better if it did?) I think uc2 tries to detect 16/24-bit data
at maximum compression (and probablyuses an alternative
ompression scheme).
Nico de Vries [AIP-NL]
>
> Just a thought on image compression. Is the zip routines
> optimized to compress non-8-bit data? (and would it compress
> better if it did?) I think uc2 tries to detect 16/24-bit data
> at maximum compression
Both -TT (tight) and -TST (supertight) compression support auto-detect,
not just -TST. -TT compression might fail on unusual data, -TST will
always choose the most optimal method.
> (and probablyuses an alternative
> ompression scheme).
UC2 does not exactly use a different compression scheme for 8,16,24 and
32 bit multimedia compression (-TT and -TST mode). It transforms the
data and then uses the core compression engine also used for 'normal'
data.
By the way, it is also possible to skip UC2's auto-detect and use
multi-media compression right away. Options -TM1 (8 bit) to -TM8 (64
bit) should be used for that.
>
Take care,
Nico de Vries [AIP-NL UltraCompressor III development]