String to real
Isidro Mendez
I'm really new to LISP and I'm using Lispworks Personal Edition for
learning. I'm looking for a way to translate from string type to
floating point type. In C, for example this is really easy, only using
the function atof(), but I haven't find and equivalente function in Ansi
Common LISP.
Can anyone shed some light on my nightmare?
Anticipated thanks,
Isidro Mendez
iargu...@nexo.es
Tim Bradshaw
> I'm really new to LISP and I'm using Lispworks Personal Edition for
> learning. I'm looking for a way to translate from string type to
> floating point type. In C, for example this is really easy, only using
> the function atof(), but I haven't find and equivalente function in Ansi
> Common LISP.
This comes up quite regularly. There really isn't a good answer --
there is PARSE-INTEGER but not PARSE-FLOAT. People will tell you to
use READ-FROM-STRING, but this is generally bad advice because you
have no idea what's in the string, and so you could get back anything,
or perform arbitrarily complex computations unless you turn off
*READ-EVAL*. So be careful if you do this.
--tim
Erik Naggum
| This comes up quite regularly. There really isn't a good answer --
| there is PARSE-INTEGER but not PARSE-FLOAT. People will tell you to
| use READ-FROM-STRING, but this is generally bad advice because you
| have no idea what's in the string, and so you could get back
| anything, or perform arbitrarily complex computations unless you
| turn off *READ-EVAL*. So be careful if you do this.
This is fairly strange advice. Let's decide to _write_ parse-float,
instead of this repeated lamentation. Somewhere in every Common
Lisp implementation, there's a function that takes a strings of
characters and returns a floating point number. It may be hard to
call, it may require other internals to be set up properly, or it
may be embedded in something else that makes it hard to access right
away, but it's there.
Language design is all about deciding on interfaces to well-defined
functionality, _not_ about implementing them nicely. In fact, we're
allowed to be as implementation-dependent as we could possibly get
if we adhere to the principle of exporting and advertising a clean
interface. Before you object, consider that it is a lot cleaner
than asking people to go off and do all the implementation-dependent
stuff in their _own_ code, because that's just plain gross.
For instance, in Allegro CL, there's a nice little function called
make-float that returns a floating point number if given a
read-buffer with characters satisfying the pattern matching a
floating point number in it, but a read-buffer is an _entrenched_
data type, and accessing it is non-trivial without sources, so only
supported customers who have signed the limited source license can
use it. I'm not super-thrilled about this, but here's how to use
it, given an input string with its usual start and end positions:
#+franz-inc
(defun parse-float (string &key (start 0) end)
(let* ((length (- (or end (length string)) start)))
(excl::make-float (vector (subseq string start end) length 0 length))))
This function will parse a floating point number and return it just
as the reader would (like obeying *read-default-float-format*), but
will return 0.0 if there is no discernible floating point number in
the string you gave it. Caveat emptor or something.
An improvement on this function is to take a string and figure out
where the floating point number ends and return that position, too.
_Or_ you could just write some FFI wrappping around the math library
functions of your favorite operating system's "native" compiler, but
that's completely immaterial to me: All I want is a function called
"parse-float" which takes a string containing the usual read syntax
for floating point numbers and gives me the floating point number
that would print back as that string, if at all possible, or the
nearest floating point model numberน if not.
#:Erik
-------
น That's what they call it in Ada, anyway.
--
If this is not what you expected, please alter your expectations.
Tim Bradshaw
> This is fairly strange advice. Let's decide to _write_ parse-float,
> instead of this repeated lamentation.
Sure, that's the right answer. I just interpreted his `in ANSI CL' to
mean `standardly provided by the language'.
My approach to this has always been to write little hacky single-use
float parsers when I had to: If I know that all numbers my application
has to read look like are [-]n.m where n and m are integers, and I
don't care too much about getting the very best result I can, it's
easy enough to write a little parser function. But the reuse mafia
tend to laugh at me...
--tim
Howard Ding
Hi,
I'm also pretty new to Lisp, and this same question actually also came
up for me today. :-)
Anyway, there appears to be a Lisp implementation of parse-float at the
CMU repository:
http://www-cgi.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/math/0.html
I haven't actually tried it out yet, so I can't necessarily vouch for
how correct/efficient/useful it is. :-)
Howard
had...@worldnet.att.net
>
> Hi all,
>
> I'm really new to LISP and I'm using Lispworks Personal Edition for
> learning. I'm looking for a way to translate from string type to
> floating point type. In C, for example this is really easy, only using
> the function atof(), but I haven't find and equivalente function in Ansi
> Common LISP.
>
Erik Naggum
| Sure, that's the right answer. I just interpreted his `in ANSI CL'
| to mean `standardly provided by the language'.
I'm working with the notion that whatever is in the standard today
is there because it existed outside the standard at some time and
was deemed admissible into it, and I don't consider the concrete
that standards are cast in to be completely solidified. The only
way we can make something _become_ standard is to do the work and
present it to the people who will hopefully measure the technical
merits and accept it. Let's get agreement on useful, good stuff.
(And let's all agree or encourage others to implement it that way.)
I interpreted his "in ANSI CL" to be the name of the language, but
maybe that's because I think programming _solely_ in ANSI CL is a
counter-productive limitation of your universe, just as writing
_solely_ in ISO C is fairly stupid. It's like refusing to use any
software that doesn't come with your operating system or even with
the hardware.
| But the reuse mafia tend to laugh at me...
Well, laugh back: I hate "reuse" almost as much as I hate "API".
I love language design, however.
#:Erik
Russell Senior
I slapped an implementation together of a PARSE-FLOAT this afternoon.
There are a few edge cases where it might behave more nicely, I assume
it is horribly inefficient, and it might go a little overboard on the
use of RADIX, but it is a start anyway. The first three functions are
intended to be helper functions to PARSE-FLOAT.
---cut-here------cut-here------cut-here------cut-here------cut-here---
(defun parse-integer-part (string start end radix)
"A helper function to PARSE-FLOAT, finds the first integer looking
thing in STRING between indices START and END, returning the integer
portion (assuming radix RADIX) and the index of the next character in
the string."
(flet ((integer-char-p (ch)
(or (find ch #(#\- #\+))
(digit-char-p ch radix))))
(let ((integer-part-start (position-if #'integer-char-p
string
:start start
:end end)))
(if integer-part-start
(parse-integer string
:start integer-part-start
:radix radix
:junk-allowed t)
(values 0 start)))))
(defun parse-fraction-part (string start end radix decimal-char)
"A helper function to PARSE-FLOAT, finds a fractional part of STRING
between indices START and END. The character at index START must be
DECIMAL-CHAR. Reads the subsequent digits assuming radix RADIX,
returning the fraction as a rational and the index of the next
character in the string."
(if (< start end)
(let ((ch (elt string start))
(fraction-start (1+ start)))
(multiple-value-bind (fraction-part fraction-end)
(if (and (char= (elt string start) decimal-char)
(< fraction-start end))
(parse-integer string
:start fraction-start
:radix radix
:junk-allowed t)
(values 0 fraction-start))
(let* ((divisor (expt radix (- fraction-end fraction-start)))
(fraction (/ fraction-part divisor)))
(values fraction fraction-end))))
(values 0 start)))
(defun parse-exponent (string start end radix)
"A helper function to PARSE-FLOAT, finds the exponent part of STRING
between indices START and END. The character at index START must be a
recognized exponent designator. Reads the subsequent digits assuming
radix RADIX, returning the exponent and the float-format. If
float-format cannot be determined, float-format is returned NIL."
(let ((ch (and (< start end)
(find (elt string start)
#(#\e #\E #\s #\S #\d #\D #\l #\L)))))
(if ch
(let ((exponent (parse-integer string
:start (+ start 1)
:radix radix
:junk-allowed t))
(float-format (case ch
((#\e #\E) nil)
((#\s #\S) 'short-float)
((#\d #\D) 'double-float)
((#\l #\L) 'long-float))))
(values exponent float-format))
nil)))
(defun parse-float (string &key (start 0)
(end (length string))
(radix 10)
(decimal-char #\.))
"Parses a float from STRING of the form ``[+-]<n>[<d><m>][<e><l>]'',
where <n> is the integer part, <d> is the decimal character, <m> is
the fractional part, <e> is an exponential designator (either `e',
`s', `d', `l' or the uppercase equivalents) and <l> is the exponential
magnitude. The <n>, <m> and <l> portions are given as integers in
radix RADIX. Note also, the exponent is considered to apply to base
RADIX. The float is parsed starting at index START and stopping
before index END. If the exponential designator <e> is `s', `d', or
`l', the float is coerced to the corresponding float type, otherwise
it is coerced to *READ-DEFAULT-FLOAT-FORMAT*."
(multiple-value-bind (integer-part next-start)
(parse-integer-part string start end radix)
(multiple-value-bind (fraction-part next-start)
(parse-fraction-part string next-start end radix decimal-char)
(multiple-value-bind (exponent-part float-format)
(parse-exponent string next-start end radix)
(coerce (* (if (minusp integer-part)
(- integer-part fraction-part)
(+ integer-part fraction-part))
(if exponent-part
(expt radix exponent-part)
1))
(if float-format
float-format
*read-default-float-format*))))))
--
Russell Senior ``The two chiefs turned to each other.
sen...@aracnet.com Bellison uncorked a flood of horrible
profanity, which, translated meant, `This is
extremely unusual.' ''
Erik Naggum
| Anyway, there appears to be a Lisp implementation of parse-float at
| the CMU repository:
Since the correctness of parsing floating point numbers depends
heavily on other parts of the implementation, I consider using a
third-party float-parser very, very risky. It's safer to use a
foreign function call to C than to assume you have sufficient
control over the representation in Common Lisp to piece together a
floating point number. And remember print-read consistency!
Tim Bradshaw
> I'm working with the notion that whatever is in the standard today
> is there because it existed outside the standard at some time and
> was deemed admissible into it, and I don't consider the concrete
> that standards are cast in to be completely solidified. The only
> way we can make something _become_ standard is to do the work and
> present it to the people who will hopefully measure the technical
> merits and accept it. Let's get agreement on useful, good stuff.
> (And let's all agree or encourage others to implement it that way.)
Well put.
--tim
Paul Foley
> instead of this repeated lamentation. Somewhere in every Common
> Lisp implementation, there's a function that takes a strings of
> characters and returns a floating point number. It may be hard to
> call, it may require other internals to be set up properly, or it
> may be embedded in something else that makes it hard to access right
> away, but it's there.
Of course, it's easily (and portably) called through READ-FROM-STRING;
so just check that you're looking at something the reader will parse
as a float, and call READ-FROM-STRING on it...
[The following has only been _very_ lightly tested...]
(defun parse-float (string &key (start 0) end)
(let ((point start)
(c nil))
(labels ((bad-float ()
(error "Failed to parse a float out of ~S."
(subseq string start end)))
(next-char (&optional (eof-error-p t))
(declare (type string string))
(cond ((= point (or end (length string)))
(setf c nil)
(if eof-error-p (bad-float)))
(t (setf c (char string point))
(incf point)))))
(next-char)
(tagbody
(when (or (char= c #\+) (char= c #\-))
(next-char))
(when (char= c #\.)
(next-char)
(go init-fraction))
(unless (digit-char-p c)
(bad-float))
(do ()
((not (digit-char-p c)))
(next-char))
(when (position c "sfdleSFDLE")
(go exponent))
(when (char= c #\.)
(next-char nil)
(go fraction))
(bad-float)
init-fraction
(unless (digit-char-p c)
(bad-float))
fraction
(do ()
((or (null c) (not (digit-char-p c))))
(next-char nil))
(unless (position c "sfdleSFDLE")
(go done))
exponent
(next-char)
(when (or (char= c #\+) (char= c #\-))
(next-char))
(unless (digit-char-p c)
(bad-float))
(do ()
((or (null c) (not (digit-char-p c))))
(next-char nil))
done
(when c (decf point))
(return-from parse-float (read-from-string string nil nil
:start start
:end point))))))
--
Whenever you find that you are on the side of the majority, it is time
to reform. -- Mark Twain
(setq reply-to
(concatenate 'string "Paul Foley " "<mycroft" '(#\@) "actrix.gen.nz>"))
Paolo Amoroso
> Well, laugh back: I hate "reuse" almost as much as I hate "API".
I would appreciate it if you could elaborate on this. Do you hate reuse
because of reasons related to what Gabriel tells about the topic in
"Patterns of Software"? What are the reasons why you hate APIs?
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
Erik Naggum
| Well, laugh back: I hate "reuse" almost as much as I hate "API".
* Paolo Amoroso
| I would appreciate it if you could elaborate on this. Do you hate
| reuse because of reasons related to what Gabriel tells about the
| topic in "Patterns of Software"? What are the reasons why you hate
| APIs?
I hate the words and their connotations, not the concepts they
_strictly_ denote. Reusability is a coincidental value of high
quality, but optimizing for it is really, really stupid, and so
talking about "reuse" as if it could exist without high quality
code just rubs me the wrong way. An "API" stresses that people
are not building languages and abstractions, but rather jumbled
heaps of disjoint functions that don't even make a _protocol_.
Marco Antoniotti
Russell Senior <sen...@aracnet.com> writes:
> I slapped an implementation together of a PARSE-FLOAT this
> afternoon.
How does it differ from the one available in the AI.Repository?
Cheers
--
Marco Antoniotti ===========================================
Erik Naggum
| How does it differ from the one available in the AI.Repository?
It probably makes a whole slew of subtly different mistakes.
#:Erik :)
Russell Senior
Marco> How does it differ from the one available in the AI.Repository?
Erik> It probably makes a whole slew of subtly different mistakes.
It would be interesting and educational for me if you could point a
few of them out. Given the `slew' it should not be difficult.
One shortcoming has already been patched. Replace the helper function
PARSE-EXPONENT with the following:
(defun parse-exponent (string start end radix)
"A helper function to PARSE-FLOAT, finds the exponent part of STRING
between indices START and END. The character at index START must be a
recognized exponent designator. Reads the subsequent digits assuming
radix RADIX, returning the exponent and the float-format. If
float-format cannot be determined, float-format is returned NIL."
(let ((ch (and (< start end)
(find (elt string start)
#(#\e #\E #\s #\S #\f #\F #\d #\D #\l #\L)))))
(if ch
(let ((exponent (parse-integer string
:start (+ start 1)
:radix radix
:junk-allowed t))
(float-format (case ch
((#\e #\E) nil)
((#\s #\S) 'short-float)
((#\f #\F) 'single-float)
((#\d #\D) 'double-float)
((#\l #\L) 'long-float))))
(values exponent float-format))
nil)))
--
Erik Naggum
| It would be interesting and educational for me if you could point a
| few of them out. Given the `slew' it should not be difficult.
The last sentence is completely bogus, and obviously so. Finding
which of 2^64 bit patterns are read and printed wrong is no small
task, even if there are 2^16 of them. I spent a few hundred CPU
hours on 2 × 600MHz Pentium III's and 2 × 400MHz Pentium II's
stress-testing Allegro CL's floating-point reader and printer and
came up with a few interesting cases where it missed, due to
accumulated rounding errors. It dook Duane Rettig of Franz Inc
several days to come up with a solution, and it was not trivial,
neither in terms of the code required or the time required to read
or print floating-point numbers correctly.
C's atof and printf were never meant to be print-read-consistent, so
it's fairly easy to find situations where a number printed and read
back yields a different bit pattern that is really hard to detect if
you don't look at the bits (except for the trivial unequality), but
for a Common Lisp system, that is basically unforgiveable.
If you're not an expert at numerical analysis, the simple dictum is:
don't presume to know how to read or print floating point numbers
but do show respect for those who are experts and do know how. (I
fall in the latter category, not the former of these, which is why I
keep stressing that a correct parse-float function that maintains
print-read consistency is heavily implementation-dependent and you
shoult not mix third-party readers with builtin printers, unless you
know what you're doing extremely well.)
#:Erik
Russell Senior
Russell> It would be interesting and educational for me if you could
Russell> point a few of them out. Given the `slew' it should not be
Russell> difficult.
Erik> The last sentence is completely bogus, and obviously so.
Yes, I agree. I apologize for my petulance.
Tim Bradshaw
> One shortcoming has already been patched. Replace the helper function
> PARSE-EXPONENT with the following:
It also doesn't deal with negative numbers right, try -0.x
In general reading and printing floats right (so you get print-read
consistency) is fairly non-trivial. JonL gave a talk about this at
LUGM99, but I don't know if notes from it available.
--tim
Russell Senior
Marco> Russell Senior <sen...@aracnet.com> writes:
Russell> I slapped an implementation together of a PARSE-FLOAT this
Russell> afternoon.
Marco> How does it differ from the one available in the AI.Repository?
From my brief examination of the `Thu Aug 25 00:56:39 1994 by Mark
Kantrowitz' version I see the following differences:
> My version coerces a rational to the indicated float-type, whereas
the MK version requires an exponent character of `e' and uses FLOAT
to do the coercion. I handle exponent characters of `s', `f', `d'
and `l' (I left out `f' in the version I posted) and coerce to the
corresponding float-type or to *READ-DEFAULT-FLOAT-FORMAT* if
ambiguous.
> My version doesn't include any error handling.
> There is a bug in my posted version dealing with float strings like
"-0.xxx", where my sign propagation logic is flawed. I have fixed
that in my local copy.
> I rely on the implementation's COERCE to cooperate with the print
functionality to achieve print-read consistency.
> My version doesn't provide the option of :junk-allowed nil.
Currently, junk is allowed.
> My version provides support for non-#\. decimal point characters,
such as #\,.
> I read the exponent in the extant radix, while it looks like the MK
version requires the radix to be 10 to even parse the exponent.
> I suspect that the MK version is vastly more thoroughly tested than
mine.
Those are the differences I am aware of.
Paolo Amoroso
> code just rubs me the wrong way. An "API" stresses that people
> are not building languages and abstractions, but rather jumbled
> heaps of disjoint functions that don't even make a _protocol_.
This is interesting. Now that I think about it, I don't remember seeing the
expression "API" much mentioned--if at all--in the Lisp literature I have
read. "Protocol" is much more frequent.
Paolo Amoroso
> In general reading and printing floats right (so you get print-read
> consistency) is fairly non-trivial. JonL gave a talk about this at
> LUGM99, but I don't know if notes from it available.
``A Historical Perspective on Numerics in Lisp: or, Numbers are Symbols,
Too''
Jon L. White
Proceedings of the Lisp User Group Meeting '99 (LUGM '99)
Abstract:
"For the first two decades after its introduction, Lisp had the reputation
of being two orders of magnitude slower on numerics than conventional
languages such as Fortran; in addition, limits on the magnitudes and
precision of numerical quantities stifled research in a number of areas,
for example Symbolic Algebra. This paper describes some of the motivational
forces behind the drive to improve Lisp's treatment of arithmetic
expressions, and numeric capabilities in general, as well as a few
techniques used to achieve these goals."
Here is an outline of the paper:
Introduction
Hardware Influences Language Design
Symbolic Algebra
Perfect Floating-Point Printout
Roots of Hardware Influence on Lisp Numerics
A Little Pre-History
Entering the MacLisp Era
Lowtag Pointer Schemes
Symbolic Algebra and Bignums
Fast Arithmetic and Ncomplr
Perfect Float Printout, or "Floats are Symbols Too!"
Which Truncation Matters?
The Notion of "Inexactness" Being Itself "Inexact"
Visualizing the Result
Visualizing the Early PDP10 MacLisp Algorithm
The LUGM '99 proceedings are available from Franz, Inc.
http://www.franz.com/.