Weird Math

[Andy Madigan]

I've been running some scripts through rhino, and one script kept on
getting an equation wrong, I finally found the problem, I ran a simple
test through the Rhino Shell, here's the output:

js> 100 - 69.1
30.900000000000006

Am I going nuts? Is this something about how javascript does math that
I don't know? (Note: 100 - 59.1 is correct)

P.S. Also:
js> 100 - 69.1 == 100 - 59.1 - 10
false

P.P.S In case this is something VM/System related, I am using Sun's
J2SDK 1.4.2_05 on Debian Sid (kernel 2.4.25)

=====

[Brendan Eich]

Andy Madigan wrote:
> I've been running some scripts through rhino, and one script kept on
> getting an equation wrong, I finally found the problem, I ran a simple
> test through the Rhino Shell, here's the output:
>
> js> 100 - 69.1
> 30.900000000000006

That's correct, per the IEEE-754 floating point standard, which is used
by JS, Java, and other "virtual machine" languages on all platforms, and
by C, C++, etc. ("real machine" languages) on almost all modern
architectures.

> Am I going nuts? Is this something about how javascript does math that
> I don't know? (Note: 100 - 59.1 is correct)

Any finite-precision binary number format will have rounding errors when
dealing with certain base-10 numbers. Notice that:

69.1 = 64 + 4 + 1 + 1/16 + 1/32 + 1/256 + 1/512 + 1/4096 + 1/8192 + ...

Use Number.prototype.toFixed to round appropriately:

js> (100 - 69.1).toFixed(1)
30.9
js> (100 - 69.1).toFixed(2)
30.90
js> (100 - 69.1).toFixed(10)
30.9000000000

/be

[Igor Bukanov]

Andy Madigan wrote:
> I've been running some scripts through rhino, and one script kept on
> getting an equation wrong, I finally found the problem, I ran a simple
> test through the Rhino Shell, here's the output:
>
> js> 100 - 69.1
> 30.900000000000006
>
> Am I going nuts? Is this something about how javascript does math that
> I don't know? (Note: 100 - 59.1 is correct)

69.1 can not be represented as exact Java double (binary representation
of 0.1 is infinite fraction) so it is rounded to the nearest
representable number. When the rounded number is subtracted from 100,
its closes decimal that when read back preserve the binary presentation
is 30.900000000000006.

You do not see that with 100 - 59.1 since 69.1. and 59.1 have rather
different binary presentations (starting from the fact that 69 > 64 and
59 < 64) so in this case the decimal 40.9 is the best fit.

Note that SpiderMonkey has exactly the same behavior which is not
surprising since Rhino code is just a translation of SM code into Java.

Regards, Igor

[Jens Thiele]

A nice paper with all the details about the topic:

http://docs.sun.com/source/806-3568/ncg_goldberg.html:

"What Every Computer Scientist Should Know About Floating-Point Arithmetic

Note – This appendix is an edited reprint of the paper What Every
Computer Scientist Should Know About Floating-Point Arithmetic, by David
Goldberg, published in the March, 1991 issue of Computing Surveys.
Copyright 1991, Association for Computing Machinery, Inc., reprinted by
permission."

Note there is a second section / comment on the paper:
http://docs.sun.com/source/806-3568/ncg_goldberg.html#3098:

"Differences Among IEEE 754 Implementations

Note – This section is not part of the published paper. It has been
added to clarify certain points and correct possible misconceptions
about the IEEE standard that the reader might infer from the paper. This
material was not written by David Goldberg, but it appears here with his
permission."

Jens

[saya...@gmail.com]

I am 99999999*99999999+2 in js it produces incorrect result
even 9999999700000002 * 3 produces incorrect result