[8.6.1] More floating point oddities (?)

[Richard Sargent]

(1023 to: 971 by: -1) inject: 0.0 into: [:sum :power | sum + (2.0 raisedTo: power)]
===> 16r7FEFFFFFFFFFFFFF (Float fmax)

(1023 to: 971 by: -1) inject: 0.0 into: [:sum :power | sum + (2.0 raisedTo: power)]
===> Floating point overflow, which probably shouldn't have, but resulting in a system crash!
(Unhandled exception at 0x62ddf4ae (esvm40.dll) in abt.exe: 0x80000026: a long jump has been executed.) [According to VisualStudio 2010]

(971 to: 1023) inject: 0.0 into: [:sum :power | sum + (2.0 raisedTo: power)]
===> 16r7FEFFFFFFFFFFFFF (Float fmax)

(970 to: 1023) inject: 0.0 into: [:sum :power | sum + (2.0 raisedTo: power)]
===> Floating point overflow, which probably shouldn't have.

[Richard Sargent]

The second example of (1023 to: 971 by: -1) should have been (1023 to: 970 by: -1).

The resulting last iteration should have been two small to affect the result, so should not have overflowed. (Of course, it might have affected extended bits in the math processor which would have wanted to round the resulting 64-bit value.)

--
You received this message because you are subscribed to a topic in the Google Groups "VA Smalltalk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/va-smalltalk/LDmrGw8Y8DM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to va-smalltalk...@googlegroups.com.
To post to this group, send email to va-sma...@googlegroups.com.
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

[Seth Berman]

Hi Richard,

Some other vm guys are looking into this...they were able to reproduce this but only on Win7 64-bit....is this what you are running?

-- Seth

[Richard Sargent]

On Thursday, May 28, 2015 at 8:32:59 AM UTC-7, Seth Berman wrote:

Hi Richard,

Some other vm guys are looking into this...they were able to reproduce this but only on Win7 64-bit....is this what you are running?

Oddly enough, Windows 7 Enterprise 64-bit. :-)
Why is this no surprise?

My hypothesis is that the computation that works can be expressed in 64 bits, but one more iteration requires 65 bits. I believe hardware FP uses 80 bits? So, I suspect a rounding scenario, triggering an overflow.

fmax: ........ 7F EF FF FF FF FF FF FF [00 00]
2.0^970: ..... 7F E0 00 00 00 00 00 00 [80 00] (guessing)
Sum: ......... 7F EF FF FF FF FF FF FF [80 00] (guessing)
Rounded: ..... 7F F0 00 00 00 00 00 00 (+Inf)

[Seth Berman]

Hi Richard,

Yes, we can look into the exact reason of the overflow...it may be as you suggest.  We see that the exception is triggered in EsScanner>>readNumber during a multiplication on fmax.
This is reproducible on all the windows platforms.  But the situation where the image hangs and crashes is only seen on Win7 64-bit at the moment...so we are working to understand that one.

Thanks!

-- Seth