Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

HP30S numerical anomaly

7 views
Skip to first unread message

Rodger Rosenbaum

unread,
Feb 22, 2006, 1:57:39 AM2/22/06
to
I posted about a peculiar HP30S behavior over on MoHP, and I thought I
should post it here too.

Recently while looking at the calculator forensics web site,

http://www.rskey.org/~mwsebastian/miscprj/forensics.htm

I noticed that the HP30S was shown to pass the

arcsin(arccos(arctan(tan(cos(sin(9)))))

test with no error.

I wondered if this was really true, so I decided to look at all the
intermediate results during the calculation. I discovered that the zero
error result is the result of a peculiarity of the HP30S arithmetic.

The HP30S is apparently doing binary arithmetic internally, with 80 bit
precision (about 24 digit decimal precision). It can be verified that
ordinary operations such as +, -, * and / are giving 24 digit results. Some
of the other mathematical functions don't do so well. Joe Horn discovered a
while ago that square root only gives about 12 digit precision. The trig
functions can give substantially less than 24 digits; sometimes as low as
14 digits.

The full precision of a particular result can be seen using the technique
described on the calculator forensics web site to see guard digits:
(http://www.rskey.org/~mwsebastian/miscprj/guard_digits.htm).

The idea is to make a calculation and then subtract some of the leading
digits of the result and then you will see the rest of the digits. It may
be necessary after the subtraction to multiply by some power of 10 to shift
the remaining digits left so they can be seen.

The last calculation in the calculator forensics test is an arcsin, and
while looking at the details, I discovered the following peculiar behavior
of the HP30S.

In degrees mode, type the following into the HP30S (followed by enter):

Type arcsin(.156434465031) and see the result: 8.999999999

Type (arcsin(.156434465031)-8.999999999)*100000000000 and see: 46.45174843

The digits from the second calculation should be appended to the digits
from the first calculation to give a multi-digit result of:
8.9999999994645174843; the exactly correct result should be:
8.99999999946451748615554. In this case the HP30S gets 18 correct digits.

But now do the same calculation with a slightly greater argument:

Type arcsin(.156434465032) and see the result: 9

Type (arcsin(.156434465032)-8.999999999)*100000000000 and see: 100

The exact result *should* be: 8.99999999952252746476729

What happened? After some playing around, it seems that whenever the result
of a calculation (of a trig function, at least) is an integer followed by
.9999999995+, that is, the digits after the decimal point are nine 9's and
a 5 plus just a few more non-zero digits, the HP30S just rounds up to the
next integer. So what should have been:

8.99999999952252746476729

turns into 9.000000000000000000000000.

And the same thing happens on the other side of an integer (slightly
greater than some integer). For example:

arcsin(.156434465049) returns 9.000000001:

(arsin(.156434465049)-9)*100000000000 returns 50.86970993

Realizing that the first result was actually 9.0000000005+some more digits
(it got rounded up to 9.000000001 in the display), we can construct the
full result of 9.0000000005086970993. The exact result *should* be:
9.00000000050869710116852--the HP30S gets about 16 correct digits.

But now try arcsin(.156434465048) and get 9

Furthermore, (arsin(.156434465048)-9)*100000000000 returns 0
, indicating that there are no more digits after the integer 9.

Again the digits after the decimal point have been dropped, and we don't
even get 12 digit accuracy.

It appears that whenever the digits after the decimal point are greater
than .9999999995 or less than .0000000005, they get dropped--at least as a
result of a trig calculation; I haven't checked all other caclulations, but
this may happen always.

Why do they do this? Probably to avoid some of the apparent small errors
that happen in simple calculations, which perplex naive calculator users.
For example, do (sqrt(2))^2 on a typical calculator and get
1.99999999999...(depending on how many digits the calculator carries, and
whether there are hidden guard digits) instead of exactly 2. Or do sin(pi)
and get something other than zero. Or do (6^-1)^-1 and get 5.99999999999...
instead of exactly 6. The HP30S gets the exact expected result, but at the
price of reduced accuracy when a result is near an integer.

So finally, this is why the HP30S gets exactly 9.00000000000000000 when
running the calculator forensics test. To see the real error, one should
subtract the input argument of the test from the final result. That is, do:

arcsin(arcos(arctan(tan(cos(sin(9))))))-9

On the HP30S, the result is 0, leading one to believe that the HP30S is
*really* accurate. But redo the test as:

arcsin(arcos(arctan(tan(cos(sin(9.1))))))-9.1

to foil its rounding near integers and see a result: 4.833288903E-11

Not all that great for a calculator that can do 24 digit calculations, eh?

John H Meyers

unread,
Feb 22, 2006, 2:47:04 AM2/22/06
to
On Wed, 22 Feb 2006 00:57:39 -0600, Rodger Rosenbaum wrote:

> It appears that whenever the digits after the decimal point are greater
> than .9999999995 or less than .0000000005, they get dropped

> Why do they do this? Probably to avoid some of the apparent small errors


> that happen in simple calculations, which perplex naive calculator users.

> The HP30S gets the exact expected result, but at the price


> of reduced accuracy when a result is near an integer.

Exactly as my old Casio models used to do
(and still do, since they've stayed in perfect condition
for decades, which seems to be a lost manufacturing art :)

Casio's FUDGE (posted Sep 4 1996):
http://groups.google.com/group/comp.sys.hp48/msg/1f5b0aec9c4918c2

Previous HP calculator series took an entirely different approach,
which put a high value on maintaining the best possible accuracy
that the constraints of fixed-size decimal mantissas would allow
(actually it's two-tier, with "long" internal intermediate values
and *rounded* "end user" values, plus very smart algorithms);
this is still achieved in the HP48/49 series,
as well as its many predecessors (including financial models).

Thanks, Rodger!

[r->] [OFF]

llevena

unread,
Mar 3, 2006, 2:47:02 AM3/3/06
to
John H Meyers wrote:
> On Wed, 22 Feb 2006 00:57:39 -0600, Rodger Rosenbaum wrote:
>
> > It appears that whenever the digits after the decimal point are greater
> > than .9999999995 or less than .0000000005, they get dropped
>
> > Why do they do this? Probably to avoid some of the apparent small errors
> > that happen in simple calculations, which perplex naive calculator users.
>
> > The HP30S gets the exact expected result, but at the price
> > of reduced accuracy when a result is near an integer.
>
> Exactly as my old Casio models used to do
> (and still do, since they've stayed in perfect condition
> for decades, which seems to be a lost manufacturing art :)
>
> Casio's FUDGE (posted Sep 4 1996):
> http://groups.google.com/group/comp.sys.hp48/msg/1f5b0aec9c4918c2

I tried the sqrt(24.999999) thing on the 49g+ and it returned
4.9999999, does this mean the 49g+ is as artfull as the old casio?

John H Meyers

unread,
Mar 3, 2006, 1:18:53 PM3/3/06
to
On Fri, 03 Mar 2006 01:47:02 -0600, llevena wrote:

>> Re Casio's FUDGE (posted Sep 4 1996):
>> http://groups.google.com/group/comp.sys.hp48/msg/1f5b0aec9c4918c2

> I tried the sqrt(24.999999) thing on the 49g+ and it returned

> 4.9999999, does this mean the 49g+ is as artful as the old casio?

[Casio calcs normally deliver truncated results,
while that particular result is somehow instead rounded]


According to "bc" on Unix: sc=50 ; sq(24.999999)
the square root of 24.999999, truncated to 50 decimal places,
is 4.99999989999999899999997999999949999998599999957999

*Rounded* to twelve significant digits, as HP calcs designed
between the HP31E and HP4x series all do, it is 4.99999990000

There is no "fudging" of correct answers in these HP calcs;
instead, there is great care taken, both with extended precision
"long" internal intermediate values and well thought out algorithms,
to get the best answer that can be returned in a "user" size
floating point value, which always has twelve significant digits.

The user is welcome to round that further, however,
in case (s)he would rather see "7." than "7.00000000001"
after 7. 1/x 1/x (note that the inverse of .142857142857,
again to 50 decimal places, is
7.00000000000700000000000700000000000700000000000700
and the HP4x once again rounds that truthfully to 12 places).

Also on HP4x:

1E24 - 500000000000 = 1E24
1E24 - 500000000001 = 9.99999999999E23
Note that the difference between these cases is 1 out of 1E24,
so that, in effect, a 25-digit wide result was considered.

It is now possible to achieve unlimited precision (or memory-limited
precision) results using add-on programs, but this is how
all built-in scientific calculations are done in HP4x.

Would you like to extend the normal 12-digit HP precision to 24 digits?

Rodger Rosenbaum, who just reported and analyzed
the HP30S anomaly for us, supplied DIGI24 (in UserRPL),
which can do that, only because of the precision and care
which the HP4x calculator already takes with its 12-digit results;
DIGI24 (by Rodger) is on http://www.hpcalc.org/details.php?id=240
http://www.hpcalc.org/hp48/compilations/horn/horn9.zip

[r->] [OFF]

Frère-Pierre

unread,
Mar 3, 2006, 2:11:07 PM3/3/06
to
"John H Meyers" <jhme...@miu.edu> wrote in message
news:op.s5unt...@news.cis.dfn.de...
X>

> Would you like to extend the normal 12-digit HP precision to 24 digits?
>
> Rodger Rosenbaum, who just reported and analyzed
> the HP30S anomaly for us, supplied DIGI24 (in UserRPL),
> which can do that, only because of the precision and care
> which the HP4x calculator already takes with its 12-digit results;
> DIGI24 (by Rodger) is on http://www.hpcalc.org/details.php?id=240
> http://www.hpcalc.org/hp48/compilations/horn/horn9.zip

I prefer the LongFloat library!
DIGITS set to 100
works very well for matrices
Hmmm...
Could that ARM CPU do it better in C ???
Anywayz I set the Clock to 120MHz before computing

0 new messages