Question about open/read-byte...
Cory Spencer
I'm a relative Lisp newbie, and for the past several hours have been
struggling with opening a binary file, and reading the data back as a
series of 6-bit bytes. The line that has been giving me problems is:
(setq stream (open "foo" :element-type '(unsigned-byte 6)))
and whever it executes, always prints something along the lines of:
*** - file #P"/home/cspencer/foo" is not an integer file
Grr. Could someone please point out the error of my ways?
Thanks,
--
Cory Spencer
JP Massar
<cspe...@interchange.ubc.ca> wrote:
This may depend on the Lisp you are using and the OS.
Here's something that works using Allegro on Windows:
> (with-open-file (p "C:\\My Documents\\foo" :direction :output
:if-exists :supersede
:element-type '(unsigned-byte 6))
(dotimes (j 100) (write-byte 15 p)))
NIL
> (with-open-file (p "C:\\My Documents\\foo" :direction :input
:element-type '(unsigned-byte 6))
(loop for j from 0 to 99 collecting (read-byte p)))
(15 15 15 15 15 15 15 15 15 15 ...)
Also, opening a random .txt file created normally by the OS using
:element-type '(unsigned-byte 6)
seems to work fine:
(with-open-file (p "C:\\My Documents\\foo.txt" :direction :input
:element-type '(unsigned-byte 6))
(loop for j from 0 to 10 collecting (read-byte p)))
(74 80 9 9 9 53 49 48 45 56 ...)
Also, opening a random file for reading on Unix with Allegro
in the same fashion works for me.
Kent M Pitman
> <cspe...@interchange.ubc.ca> wrote:
> > (setq stream (open "foo" :element-type '(unsigned-byte 6)))
> > *** - file #P"/home/cspencer/foo" is not an integer file
>
> This may depend on the Lisp you are using and the OS.
Yes.
Some OS's only let you open certain files in certain modes.
In general, I'd expect it will read in a given element-type a file
that it itself has written in the same element-type in the same
implementation of Lisp with the same volume of the same file host
using the same OS. Any change to any of these and I'd expect the
possibility of failure outside a few narrowly defined types.
For example, if you use (UNSIGNED-BYTE 6), you may be assuming you're
taking a set of 32 bits and partitioninig it like this:
11111111 11111111 11111111 11111111 11111111 11111111
------=======-------====== ------=======-------====== ...
but in fact Lisp might realize that the OS doesn't have a 6-bit mode
and might write data as:
111111xx 111111xx 111111xx 111111xx 111111xx 111111xx
------ ------ ------ ------ ------ ------ ...
where it wastes the last two bits of every byte. (Or it might put the
byte padded at the right end instead of the left end.) Such a file must
be read/written by Lisp since only the implementation knows what it will
do in terms of padding and endian issues in the case that it's not one
of the "good" element-types.
The spec details a few good element-types that are expected to work in
natural ways so that you can read a file written not-by-Lisp, but it
doesn't include (unsigned-byte 6).
Erik Naggum
You seem to be using CLISP, but have not used CLISP to write the file.
CLISP supports arbitrarily sized bytes in a file by starting the file
with the number of bytes as a 32-bit binary number. It is so sad that
Unix does not support file meta-information. It does not keep the byte
size in the file similarly, so if you lie to open about the element-type,
you will run into a mismatch between the stored size and the actual size
in terms of specified byte size, but fortunately, this does not apply to
multiples-of-8-bit-byte files. If you want to read 6-bit bytes from a
file that does not begin with the prerequisite prefix, you need to copy
its contents to a new file that startes with the 32-bit prefix. Since
you cannot change the element-type of a stream in mid-flight, you may
need to write four 8-bit 0 bytes to your output file, copy the input file
as 8-bit bytes, then reopen as a 32-bit file in overwrite mode, and write
the file size of the input file in 6-bit bytes. Pretty damn ugly, but
such is life when people make bad technical decisions.
///
--
The past is not more important than the future, despite what your culture
has taught you. Your future observations, conclusions, and beliefs are
more important to you than those in your past ever will be. The world is
changing so fast the balance between the past and the future has shifted.