JPEG - waiting for the shoe to drop

[rusty craine]

Couple of weeks ago some one mentioned they wished for a JPEG program in
lisp (i guess common lisp). I'd been wanting to read JPEG on a DOS system
for sometime myself. So I got all the documentation I need to figure out
the JPEG file format. Started by reading the header upto the SOS marker
(the actual start of the compressed data). Load the various tables etc with
the header data. Built a function to read scan lines one by one. I
realized there was nothing in the header data to tell you how much "color
area" you'd need so I messed with that a little. Next on the list, the
decompression routines. First I had to get a _clean_ area in memory.
MuLisp (sorry Marco Antoniotti) has a function to allocate DS memory and
fill it (with nulls in this case). The allocate function returns the
address of the data area. So you can actually name it on the variable stack
(PUTD) and muLisp also has a function to write directly to DS offsets using
the PUTD name and a pointer (not sure what GC is going to do to all this
just yet, lots of numbers going to be whirlling around).

Well I got five questions 1). what have I missed, too easy so far? 2) does
CL have functions to directly manage the data segment memory? 3). what
functionality would you except from a JPEG reader.....(changing the size of
that sucker might not be fun unless it just involved cropping, changing to
other file formats would be less than fun) 4). does unix or what every ?nix
handle graphic windowing as DOS does? 5). anybody want to port it to CL
when i'm finished?

Kinda makes ya laugh...been here before. Things are going great, looking
like a shoe in......till the shoe drops.

Waiting for the shoe to drop
Rusty

PS It is a lot faster to write directly to memory and handle the areas with
varible names and offsets than it is to build arrays or tables in muLisp.
Of course that is not too lisp like i guess. The biggest scan line i found
coould be handled in an array with no problems, but I haven't look at many
JPEG's yet. The point is I am keeping the memory write functions very
array looking if someone does want to port it to CL and use arrays.
--
Email address is RCCr...@flash.net. Under some situation the "a" and the
"l" in flash are reversed in the header.
Rusty Craine

[Tim Olson]

In article <73dc5r$7ea$1...@excalibur.flash.net>, "rusty craine"
<rccr...@flash.net> wrote:

| PS It is a lot faster to write directly to memory and handle the areas with
| varible names and offsets than it is to build arrays or tables in muLisp.
| Of course that is not too lisp like i guess. The biggest scan line i found
| coould be handled in an array with no problems, but I haven't look at many
| JPEG's yet. The point is I am keeping the memory write functions very
| array looking if someone does want to port it to CL and use arrays.

Before you start micro-optimizing your code too early, I'd suggest doing
the easiest thing for allocation, etc. You will find that you are
spending most of the time in two routines: the IDCT and the colorspace
conversion, and that what you do in other places really doesn't matter
much to the overall runtime.

--

-- Tim Olson

[Marco Antoniotti]

Weeellll. Having gotten the glove on my face :) :)

"rusty craine" <rccr...@flash.net> writes:

> handle graphic windowing as DOS does? 5). anybody want to port it to CL
> when i'm finished?
>
> Kinda makes ya laugh...been here before. Things are going great, looking
> like a shoe in......till the shoe drops.
>
> Waiting for the shoe to drop
> Rusty
>

> PS It is a lot faster to write directly to memory and handle the areas with
> varible names and offsets than it is to build arrays or tables in muLisp.
> Of course that is not too lisp like i guess. The biggest scan line i found
> coould be handled in an array with no problems, but I haven't look at many
> JPEG's yet. The point is I am keeping the memory write functions very
> array looking if someone does want to port it to CL and use arrays.

If you were using CL in the first place, you'd have arrays from the
start. :)

Cheers

--
Marco Antoniotti ===========================================
PARADES, Via San Pantaleo 66, I-00186 Rome, ITALY
tel. +39 - (0)6 - 68 10 03 17, fax. +39 - (0)6 - 68 80 79 26
http://www.parades.rm.cnr.it

[rusty craine]

Tim Olson wrote in message ...

>Before you start micro-optimizing your code too early, I'd suggest doing
>the easiest thing for allocation, etc. You will find that you are
>spending most of the time in two routines: the IDCT and the colorspace
>conversion, and that what you do in other places really doesn't matter
>much to the overall runtime.

> -- Tim Olson

To be honest I wasn't really tring to optimize at this point. Thu I haven't
put at lot of thought into it yet, dealing directly with memory seemed like
the easiest way to deal with a bit-buffer and a look-ahead-bit-reader to
fetch the next Huffman symbol. Since 95% of the Huffman codes are 8 bits or
fewer long and since my memory pointer is going to point to 8 bits at a time
the exception will be easy to handle in a lisp loop. hmmm I guess I could
always put the bit pattern in to a "float" and use mask and shifts to get
the symbol. Sounds like you have dealt with jpeg before and I am open to
suggestions.

Thanks
Rusty

[Tim Olson]

In article <73fvpb$dcp$1...@excalibur.flash.net>, "rusty craine"
<rccr...@flash.net> wrote:

| To be honest I wasn't really tring to optimize at this point. Thu I haven't
| put at lot of thought into it yet, dealing directly with memory seemed like
| the easiest way to deal with a bit-buffer and a look-ahead-bit-reader to
| fetch the next Huffman symbol. Since 95% of the Huffman codes are 8 bits or
| fewer long and since my memory pointer is going to point to 8 bits at a time
| the exception will be easy to handle in a lisp loop. hmmm I guess I could
| always put the bit pattern in to a "float" and use mask and shifts to get
| the symbol.

I wrote an implementation in Smalltalk. The bitBuffer was simply an
integer to which I shifted in 8 bits from the Huffman-encoded byte stream
whenever the available number of bits dropped to 16 or less:

getBits: requestedBits

| value |
requestedBits > bitsInBuffer
ifTrue:
[self fillBuffer.
requestedBits > bitsInBuffer
ifTrue:
[self error: 'not enough bits available to decode']].
value := bitBuffer >> (bitsInBuffer - requestedBits).
bitBuffer := bitBuffer bitAnd: (1 << (bitsInBuffer - requestedBits) -1).
bitsInBuffer := bitsInBuffer - requestedBits.
^ value

fillBuffer

| byte |
[bitsInBuffer <= 16]
whileTrue:
[byte := self next.
(byte = 16rFF and: [(stream peekFor: 16r00) not])
ifTrue:
[stream position: stream position - 1.
^0].
bitBuffer := bitBuffer << 8 bitOr: byte.
bitsInBuffer := bitsInBuffer + 8].
^ bitsInBuffer

--

-- Tim Olson

[David Steuber The Interloper]

On Mon, 23 Nov 1998 22:22:00 -0600, "rusty craine"
<rccr...@flash.net> claimed or asked:

% Kinda makes ya laugh...been here before. Things are going great, looking
% like a shoe in......till the shoe drops.
%
% Waiting for the shoe to drop

I know of at least two sources for JPEG code. Kodak has an encoder
and decoder written in Java included in the JDK 1.2b4 distribution.
There is C code included in the SuSE 5.3 distribution of Linux for
JPEG and MPEG that is from the source, so to speak.

I just wish the DVD format specification would be made public. There
is also the question of DVD hardware not having a public API so that
DVD software can be written for Linux without shelling out $5000 USD
and signing an NDA.

--
David Steuber (ver 1.31.3a)
http://www.david-steuber.com
To reply by e-mail, replace trashcan with david.

May the source be with you...

[rusty craine]

David Steuber "The Interloper" wrote in message
<3660e3b0...@news.newsguy.com>...

>
>I know of at least two sources for JPEG code. Kodak has an encoder
>and decoder written in Java included in the JDK 1.2b4 distribution.
>There is C code included in the SuSE 5.3 distribution of Linux for
>JPEG and MPEG that is from the source, so to speak.
>

That was a good idea. The C code cleared up some miss-conceptions I had
gotten from my text books. I had alot of time to work on it today. I got
all the markers found I think. I couldn't find jpeg files for some of them
to test; got all the tables built. the defualt one are loaded. I'm working
on SOF0 coding (baseline, the easiest one) first. I coded most of the DCT
stuff today (boy the C code helped there, I really had that screwed up).

I had no idea that were over a dozen different codings. If I'm going to all
this work I want someone to be able to use it besides me. We get our
yearly bonus at work tomorrow; guess I'll get one of the commerical CL's
(personal). I think Franz and Harlequin both make a personal CL for the PC
WIN9?....so which one will it be. got any thoughts. [geeez then maybe I
can play with Marco A. :-)].

BTW that is a thought Marco which lisp do you use? Are you one of those
MacApple guys? Never used a Mac except for McDonalds! "We deserve a break
today at McDon.......". Nah I'll bet your a unix guys or as Dilbert put it
a eunuchs. Gee that is giving up a lot to compute, either way you spell it.

been looking at this screen toooo long
Rusty

[RCCraine]

this is the output of my marker testing program. I will clean it up over the next few days. If anybody is interested I will email it to them to mess with. As soon as I get my CL i'll port it to CL, it might make more sense then. Rusty ps the picture is jill ireland in a swimming suit. if you are going to work on a pic should be a good one. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |output of marker.lps| |written in muLisp 90 ver 7.4 (9/22/98)| |credit to allan n. hessenflow for jpegdump.c must| |be given| |input file name| JILLI.JPG OFFSET X0 SOI MAKER CODE |0xd8| OFFSET X2 APP0 LENGTH 16 MAKER CODE |0xe0| JFIF VERSION 0101, 96 X 96 DPI THUMBNAIL SIZE 0 X 0 OFFSET X14 DQT LENGTH 67 MAKER CODE |0xdb| TABLE 0 PRECISION 8 OFFSET X59 DQT LENGTH 67 MAKER CODE |0xdb| TABLE 1 PRECISION 8 OFFSET X9E SOF0 BASELINE LEN 17 MAKER CODE |0xc0| SAMPLE PRECISION 8 WIDTH 700, HEIGHT 455 COMPONENTS 3 ID 1 HORIZONTAL SAMPLING 2 VERTICAL SAMPLING 2 QUANTIZATION TABLE 0 ID 2 HORIZONTAL SAMPLING 1 VERTICAL SAMPLING 1 QUANTIZATION TABLE 1 ID 3 HORIZONTAL SAMPLING 1 VERTICAL SAMPLING 1 QUANTIZATION TABLE 1 OFFSET XB1 DHT LENGTH 29 MAKER CODE |0xc4| TABLE 0 OFFSET XD0 DHT LENGTH 70 MAKER CODE |0xc4| TABLE 16 OFFSET X118 DHT LENGTH 26 MAKER CODE |0xc4| TABLE 1 OFFSET X134 DHT LENGTH 51 MAKER CODE |0xc4| TABLE 17 OFFSET X169 SOS LENGTH 12 MAKER CODE |0xda| COMPONENTS 3 ID 1 DC TABLE 0, AC TABLE 0 ID 2 DC TABLE 1, AC TABLE 1 ID 3 DC TABLE 1, AC TABLE 1 SS=0, SE=63, AH=0, AL=0 OFFSET XF48B EOI MAKER CODE |0xd9| |rusty craine|

[David Steuber The Interloper]

On 27 Nov 1998 02:56:42 GMT, rccr...@aol.com (RCCraine) claimed or
asked:

% ps the picture is jill ireland in a swimming suit. if you are
% going to work on a pic should be a good one.

Another test pic you might use is this one:

http://www.david-steuber.com/pix/gallery/watersportsgirl.jpg

I'm kind of partial to it as I was the photographer. I haven't
updated my photo gallery yet with other pictures and links.