[Arm32] float/NaN/int issue (fixed for me, posting for info/discussion/bug report?)

[simon place]

summary:

converting to int from NaN results in arch dependent values.

use case:

i'm using ints (scaled) for a type where this is the real underlying meaning.

but to avoid rounding issues in intermediate comp, i convert to float for these, and then back.

this seemed fine.

but....

for a particular use i increased the resolution of my scaled int from int32 to int64.

and the arm code broke! (the amd64 didn't)

what i find is this;

int32(NaN) returns math.MinInt32 on both amd64 and armv7(32bit)

int64(NaN) returns math.MinInt64 on amd64 but ZERO on armv7(32bit)

i see that NaN isn't really defined as an int, so any returned value isn't 'incorrect', but shouldn't the value returned be consistent across architectures? these end-of-range values play fine with my algorithm but 0 doesn't.

i've fixed by special-casing NaN values. (may end up using a build flag.)

extra info:

int32(NaN) and int64(NaN) both return 0 on armv5

[Kurtis Rader]

On Wed, Jul 8, 2020 at 10:09 AM 'simon place' via golang-nuts <golan...@googlegroups.com> wrote:

summary:

converting to int from NaN results in arch dependent values. 

... 

i see that NaN isn't really defined as an int, so any returned value isn't 'incorrect', but shouldn't the value returned be consistent across architectures? these end-of-range values play fine with my algorithm but 0 doesn't.

i've fixed by special-casing NaN values. (may end up using a build flag.)

The result of what you're doing is undefined. You might as well be executing the HCF instruction. See https://stackoverflow.com/questions/10366485/problems-casting-nan-floats-to-int for one reasonably good discussion. You absolutely have to special-case NaN if it can be an input to your code. You should not assume it will be converted to a particular int value.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[simon place]

i understand, but think that since Go particularly allows these conversations transparently, consistency should be covered by language specification. it potentially causes bugs that doesn't fail fast, didn't for me anyway.

how much code out there would break if you changed the existing returned value on x86?

[Kurtis Rader]

On Wed, Jul 8, 2020 at 10:50 AM 'simon place' via golang-nuts <golan...@googlegroups.com> wrote:

i understand, but think that since Go particularly allows these conversations transparently, consistency should be covered by language specification. it potentially causes bugs that doesn't fail fast, didn't for me anyway.

The spec is silent on this point: https://golang.org/ref/spec#Conversions. Which makes sense since we do not want to pay the cost of the additional logic for consistently handling NaN every time a float is converted to an int since almost none of those conversions will involve NaN.

 

how much code out there would break if you changed the existing returned value on x86?

Zero, or so close to it that it doesn't matter. Any code that is depending on int64(float64(NaN)) to result in a particular value is already broken since the language spec doesn't mandate any particular value and the behavior is therefore likely to depend on the compiler version, CPU architecture, and phase of the moon.

[Ian Lance Taylor]

On Wed, Jul 8, 2020 at 11:24 AM Kurtis Rader <kra...@skepticism.us> wrote:
>
> On Wed, Jul 8, 2020 at 10:50 AM 'simon place' via golang-nuts <golan...@googlegroups.com> wrote:
>>
>> i understand, but think that since Go particularly allows these conversations transparently, consistency should be covered by language specification. it potentially causes bugs that doesn't fail fast, didn't for me anyway.
>
>
> The spec is silent on this point: https://golang.org/ref/spec#Conversions. Which makes sense since we do not want to pay the cost of the additional logic for consistently handling NaN every time a float is converted to an int since almost none of those conversions will involve NaN.

The spec is not particularly helpful, but it is not entirely silent.
It says: "In all non-constant conversions involving floating-point or
complex values, if the result type cannot represent the value the
conversion succeeds but the result value is implementation-dependent."

Ian

[simon place]

i was just reading that!

does it mean (or just imply) that constant conversions are not implementation dependent?

i remember wondering, a while ago, why it was that NaN etc. weren't constants, after-all their encoding is unique and can't vary.

seems the reason for this is.....

"Numeric constants represent exact values of arbitrary precision and do not overflow. Consequently, there are no constants denoting the IEEE-754 negative zero, infinity, and not-a-number values."

not sure how negative zero, for example, fails here, its a value of a float type, so it is Numeric, an exact value, and i don't really see how "of arbitrary precision and do not overflow" apply?

but this is stymieing the whole point of the way i was doing it, simple/clear/short/std.lib leveraging,  because it seems you HAVE to check them all, (NaN/+Inf/-Inf/-0) because an "implementation-dependent" value could be anything and so potentially exactly what you were expecting right up until it isn't!

seems quite like map iteration randomisation. if it can be anything better make it random to cause earliest failure.

[Kurtis Rader]

On Wed, Jul 8, 2020 at 3:26 PM 'simon place' via golang-nuts <golan...@googlegroups.com> wrote:

i remember wondering, a while ago, why it was that NaN etc. weren't constants, after-all their encoding is unique and can't vary.

seems the reason for this is.....

"Numeric constants represent exact values of arbitrary precision and do not overflow. Consequently, there are no constants denoting the IEEE-754 negative zero, infinity, and not-a-number values."

not sure how negative zero, for example, fails here, its a value of a float type, so it is Numeric, an exact value, and i don't really see how "of arbitrary precision and do not overflow" apply?

If you're on a system with twos-complement representation for ints, rather than ones-complement or sign-and-magnitude, you can't represent negative zero in an int type. How would you convert float64(-0) to an int without losing information? It isn't overflow in the usual sense but the point is that those values cannot be represented by any of the int data types.

but this is stymieing the whole point of the way i was doing it, simple/clear/short/std.lib leveraging,  because it seems you HAVE to check them all, (NaN/+Inf/-Inf/-0) because an "implementation-dependent" value could be anything and so potentially exactly what you were expecting right up until it isn't! 

The way you were doing it relies on implementation dependent behavior. Which is risky even if you never intend to support anything but a single architecture (e.g., x86_64) and should have a prominent "HERE BE DRAGONS" comment in the code.

[simon place]

wait a minute, so this...  https://play.golang.org/p/x5SQVgSJsIs

could return anything!

[Ian Lance Taylor]

On Wed, Jul 8, 2020 at 3:26 PM 'simon place' via golang-nuts
<golan...@googlegroups.com> wrote:
>
> i was just reading that!
>
> does it mean (or just imply) that constant conversions are not implementation dependent?

Constant conversions are mostly not implementation dependent.
Constant conversions follow the rules of general constant expressions
described at https://golang.org/ref/spec#Constant_expressions. See
the "Implementation restriction" note at the end of that section.

> i remember wondering, a while ago, why it was that NaN etc. weren't constants, after-all their encoding is unique and can't vary.
>
> seems the reason for this is.....
>
> "Numeric constants represent exact values of arbitrary precision and do not overflow. Consequently, there are no constants denoting the IEEE-754 negative zero, infinity, and not-a-number values."
>
> not sure how negative zero, for example, fails here, its a value of a float type, so it is Numeric, an exact value, and i don't really see how "of arbitrary precision and do not overflow" apply?

Constants in Go are intended to act more like mathematical
representations than like computer types. Hence values have no
precision limits, and concepts like negative zero and infinity and NaN
do not exist.

> but this is stymieing the whole point of the way i was doing it, simple/clear/short/std.lib leveraging, because it seems you HAVE to check them all, (NaN/+Inf/-Inf/-0) because an "implementation-dependent" value could be anything and so potentially exactly what you were expecting right up until it isn't!
>
> seems quite like map iteration randomisation. if it can be anything better make it random to cause earliest failure.

Unfortunately I think using that approach for conversions would have
significant execution time costs. For map iteration it's almost free.

Ian

[Kurtis Rader]

On Wed, Jul 8, 2020 at 4:50 PM 'simon place' via golang-nuts <golan...@googlegroups.com> wrote:

wait a minute, so this...  https://play.golang.org/p/x5SQVgSJsIs

could return anything!

In theory, yes, because negative zero can't be represented as an int or uint. In practice, I would expect zero. At least on any platform using the IEEE 754 standard due to how the sign is encoded by that standard and the instructions for converting such a value to an int behave. Of the five magic values that is the only one you can (probably) safely handle using the straightforward conversion.

[Ian Lance Taylor]

On Wed, Jul 8, 2020 at 4:50 PM 'simon place' via golang-nuts
<golan...@googlegroups.com> wrote:
>
> wait a minute, so this... https://play.golang.org/p/x5SQVgSJsIs
>
> could return anything!

On different implementations, including different processors, yes. On
the same implementation on the same processor, the value should be
consistent.

Ian

> On Wednesday, 8 July 2020 19:57:30 UTC+1, Ian Lance Taylor wrote:
>>
>> The spec is not particularly helpful, but it is not entirely silent.
>> It says: "In all non-constant conversions involving floating-point or
>> complex values, if the result type cannot represent the value the
>> conversion succeeds but the result value is implementation-dependent."
>>
>> Ian
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/40aa201e-884a-4ca3-8d3d-4a5a503934bbo%40googlegroups.com.

[keith....@gmail.com]

On Wednesday, July 8, 2020 at 4:50:30 PM UTC-7, simon place wrote:

wait a minute, so this...  https://play.golang.org/p/x5SQVgSJsIs

could return anything!

I think float -0 should be guaranteed to convert to integer 0.

Just like -0.25 and 0.25. The spec says "fraction discarded" but I interpret that as throwing away the - in front of the zero (being an infinitesimal negative fraction).

Only if *after* rounding towards zero, if the value doesn't fit then the result is implementation dependent.

[simon place]

yes, agreed, just pushing to the extrema.