hetrogenous binary streams
David Morse
longs, chars, whatever. I'm trying to read them from lisp, but all I
see is read-byte and read-char, that don't do what I want at all.
I think I want read-double, read-int, etc. But there don't seem to be
such functions.
Okay, so how do I do it? :)
Sent via Deja.com
http://www.deja.com/
Jochen Schmidt
> I have a binary stream with entries that can be ints, doubles, unsigned
> longs, chars, whatever. I'm trying to read them from lisp, but all I
> see is read-byte and read-char, that don't do what I want at all.
>
> I think I want read-double, read-int, etc. But there don't seem to be
> such functions.
Please don't understand me wrong (it's not a joke) but ... implement it if
you want it. It sounds like a good solution to your problem and should not
be so difficult. You can base it on read-byte (or read-char directly for
chars). Integers would be straight forward - only doubles would be a hairy
part, I think you'll need your own binary->double conversion function.
But this would also be only a matter of fiddling out mantiss and exponent
out of the double.
Regards,
Jochen
Valeriy E. Ushakov
> I think I want read-double, read-int, etc. But there don't seem to be
> such functions.
>
> Okay, so how do I do it? :)
Check <http://www.cs.uit.no/~frodef/sw/binary-types/>.
SY, Uwe
--
u...@ptc.spbu.ru | Zu Grunde kommen
http://www.ptc.spbu.ru/~uwe/ | Ist zu Grunde gehen
Erik Naggum
| I have a binary stream with entries that can be ints, doubles, unsigned
| longs, chars, whatever.
Do you know beforehand exactly what bit patterns the stream will
contain, or do you have special encoding of the data types that lets
you read them independent of high-level type expectations?
| I'm trying to read them from lisp, but all I see is read-byte and
| read-char, that don't do what I want at all.
The Lisp reader is built from read-char, and it can build any object.
You can write a reader for your own format that can build any object,
too, using read-byte if the stream is binary. This is not always
trivial, but it can certainly be done.
| I think I want read-double, read-int, etc. But there don't seem to be
| such functions.
No, because binary formats are not defined by the language. Such
formats are either much more complex than textual representation or
they are as machine-dependent as a raw memory dump, which they much
too often actually are. You may be able to dump data to be loaded
back into the same build of the same program on the same machine with
the same version compiler and same version libraries and everything
else being the same, but otherwise, it is highly risky business to
load binary data. Therefore, such complexification as you find in
ASN.1 BER and tagged data formats from .fasl (fast load) files in Lisp
systems are necessary.
| Okay, so how do I do it? :)
You don't write binary data to begin with. :)
If you really have no choice about the data you're reading, you need
to interpret the bit patterns yourself and build Lisp objects from
them. This is not as impossible as it sounds, but some Lisps have
functions that help you do this. E.g., Allegro CL 6.0 has functions
that both hack up floating-point types into several shorts (16-bit
integers) and reassemble them, for use in binary data formats. Do
(apropos :shorts) to find them. You build integers from bytes the
easy way, but you may also specify bytes that are bigger than the
trivial 8 bits. A Lisp "byte" is as large as you want it. Specify
the :element-type when you open the stream.
#:Erik
--
The United States of America, soon a Bush league world power. Yeee-haw!
David Morse
I agree with you in principle, and I would love to write a text file,
but I don't think its possible to write floats as text files without
roundoff errors creeping in. Or am I wrong?
Jochen Schmidt
> but I don't think its possible to write floats as text files without
> roundoff errors creeping in. Or am I wrong?
You can write a float as text to a file with exactly the same precision
as its binary representation has.
The only difference to the binary representation would be a bigger
amount of data per float-value on an average.
Regards
Jochen Schmidt
Joe Marshall
> I agree with you in principle, and I would love to write a text file,
> but I don't think its possible to write floats as text files without
> roundoff errors creeping in. Or am I wrong?
You are wrong. (couldn't resist)
It is possible to write out a float such that when it is read back in
you get the same bits back. Essentially, you write out enough digits
so that when it is rounded you get the same number.
Guy Steele and Jon L. White wrote a paper on how to do that quickly
and efficiently (in Lisp).
>
> Sent via Deja.com
> http://www.deja.com/
-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----
Erik Naggum
| I agree with you in principle, and I would love to write a text file,
| but I don't think its possible to write floats as text files without
| roundoff errors creeping in. Or am I wrong?
Yes. We have an important principle in Common Lisp: Write-read
consistency. That means that whenever an object is printed out and
*print-readably* is true, reading it back in produces an object that
consistent with the object printed out. Achieving this is not always
cheap, but it is a very, very important property of a language that
defines the operations write and read for its objects through textual
representation. Inferior languages may not go the great lengths that
are required to maintain write-read consistency, however.
Tim Bradshaw
>
> I agree with you in principle, and I would love to write a text file,
> but I don't think its possible to write floats as text files without
> roundoff errors creeping in. Or am I wrong?
I believe that not only is this possible (this is kind of obvious if
you think -- you could just write out the bit string...) but that
Common Lisp systems should actually do this -- if you write a float
and then read it back you should get something which is the same
number, assuming you haven't done anything weird with printer control
variables.
However I might be wrong about this, though I don't think so.
There is a famous paper by Jon L White (and others?) on how to do
this reasonably efficiently.
--tim
Tim Olson
| David Morse <david...@my-deja.com> writes:
[printing and reading bit-accurate floating point numbers]
| There is a famous paper by Jon L White (and others?) on how to do
| this reasonably efficiently.
See "Printing Floating Point Numbers Quickly and Accurately" by Robert G.
Burger and R. Kent Dybvig:
http://www.cs.indiana.edu/%7Eburger/FP-Printing-PLDI96.ps.gz
This method is used in many Lisp, Scheme, Haskell, and Smalltalk
implementations.
--
-- Tim Olson
David Combs
Joe Marshall <j...@content-integrity.com> wrote:
><snip>
>It is possible to write out a float such that when it is read back in
>you get the same bits back. Essentially, you write out enough digits
>so that when it is rounded you get the same number.
>
>Guy Steele and Jon L. White wrote a paper on how to do that quickly
>and efficiently (in Lisp).
Anyone have a ref to this paper?
David
Joe Marshall
The MIT Scheme source code mentions the following:
The Dragon4 algorithm is described in "How to print floating point
numbers accurately" by Guy L Steele Jr and Jon L White in ACM SIGPLAN
Conference on Programming Language Design and Implementation 1990
(PLDI '90).
Burger & Dybvig ("Printing Floating-Point Numbers Quickly and
Accurately" by Robert G Burger and R Kent Dybvig, PLDI '96) describe a
variant of the Dragon4 algorithm that addresses some of the efficiency
issues. It is much faster for very large or very small numbers, but
not much different to numbers within a few orders of magnitude of 1.
Kaelin Colclasure
> The MIT Scheme source code mentions the following:
>
> The Dragon4 algorithm is described in "How to print floating point
> numbers accurately" by Guy L Steele Jr and Jon L White in ACM SIGPLAN
> Conference on Programming Language Design and Implementation 1990
> (PLDI '90).
>
> Burger & Dybvig ("Printing Floating-Point Numbers Quickly and
> Accurately" by Robert G Burger and R Kent Dybvig, PLDI '96) describe a
> variant of the Dragon4 algorithm that addresses some of the efficiency
> issues. It is much faster for very large or very small numbers, but
> not much different to numbers within a few orders of magnitude of 1.
I was able to retrieve both of these articles from the ACM Digital Library.
Does anyone happen to have a reference to an implementation of either of
these algorithms in C?
-- Kaelin
Russell Senior
David> In article <hf458a...@content-integrity.com>, Joe Marshall
David> <j...@content-integrity.com> wrote:
>> <snip> It is possible to write out a float such that when it is
>> read back in you get the same bits back. Essentially, you write
>> out enough digits so that when it is rounded you get the same
>> number.
>>
>> Guy Steele and Jon L. White wrote a paper on how to do that quickly
>> and efficiently (in Lisp).
David> Anyone have a ref to this paper?
I think this is probably what you are referring to:
G. L. Steele and J. L. White, "How to Print Floating-Point Numbers
Accurately," pp 112-126 in _Proceedings of the ACM SIGPLAN '90
Conference of Programming Language Design and Implementation_.
White Plains, NY, June 20-22, 1990.
It is a companion to another paper:
W. D. Clinger, "How to Read Floating Point Numbers Accurately,"
pp. 99-101 in _Proceedings of the ACM SIGPLAN '90 Conference of
Programming Language Design and Implementation_. White Plains, NY,
June 20-22, 1990.
A google search of "Steele White Clinger" turned up some related
material.
--
Russell Senior ``The two chiefs turned to each other.
sen...@aracnet.com Bellison uncorked a flood of horrible
profanity, which, translated meant, `This is
extremely unusual.' ''