Reflect DeepEqual and NaN

[Brendan Tracey]

The current implementation of reflect.DeepEqual returns false if there are any NaN values in the structure. For example

http://play.golang.org/p/_hagR4R_y0
a := math.NaN()
b := math.NaN()
reflect.DeepEqual(a,b)  // returns false

It seems to me like the above should return true given the point of the test. Should I file an issue, or am I missing something? If I am missing something, is there any way to perform a deep equal where NaN is considered equal?

I do not feel changing the behavior would be breaking Go 1 compatibility. The documentation says "uses == where possible", but it is not possible to use "==" to compare if NaNs are equal.

[Ian Lance Taylor]

NaN values are never equal, by definition. That is, if a is a NaN
value, then a == a is false. See http://play.golang.org/p/YuE1sB_Jwf .

So reflect.DeepEqual is just reflecting the definition of the
language.

Ian

[Brendan Tracey]

I understand == for NaNs returns false. The question is if two NaN values should be "Deep Equals" each other.

Use case: I am testing a function which returns a result structure. With the same inputs, the function should return the same result structure. I was trying to use DeepEquals to compare if the results structures are the same (which I believe is one of the points of DeepEquals). The return values are:

Result: &opt.Result{Location:opt.Location{X:[]float64{1.0000000000001437, 1.0000000000002878}, F:2.0658738689734433e-26, Gradient:[]float64{NaN, NaN}}, Stats:opt.Stats{NumMajorIterations:58548, NumFunEvals:641043, NumGradEvals:0, NumFunGradEvals:58548, Runtime:0, GradNorm:9.98010841470715e-14}, Status:3},

Result2: &opt.Result{Location:opt.Location{X:[]float64{1.0000000000001437, 1.0000000000002878}, F:2.0658738689734433e-26, Gradient:[]float64{NaN, NaN}}, Stats:opt.Stats{NumMajorIterations:58548, NumFunEvals:641043, NumGradEvals:0, NumFunGradEvals:58548, Runtime:0, GradNorm:9.98010841470715e-14}, Status:3}

As far as testing for idempotency is concerned, these two structures are equal. However, DeepEquals returns false because the Gradient field contains NaN values. How can I perform a test for deep equality where NaN values are considered equal?

[Dan Kortschak]

Test over the fields individually and if you want to confirm NaN identity, write an equality checker for the types that may hold NaN such that NaN == NaN.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ian Lance Taylor]

On Thu, Oct 9, 2014 at 2:36 PM, Brendan Tracey <tracey....@gmail.com> wrote:
>
> I understand == for NaNs returns false. The question is if two NaN values
> should be "Deep Equals" each other.

As the DeepEqual doc says, "[i]t uses normal == equality where
possible." For values of type float32 or float64 normal == equality
is possible, and that is exactly what DeepEqual does.

> As far as testing for idempotency is concerned, these two structures are
> equal. However, DeepEquals returns false because the Gradient field contains
> NaN values. How can I perform a test for deep equality where NaN values are
> considered equal?

If you want something that is mostly like == but for which NaN's are
==, you'll have to write your own function. You can't use
reflect.DeepEqual.

Look at it this way: for DeepEqual, we have to decide how to handle
NaNs. There are two choices: like ==, or always treat NaN as equal.
Different people have different needs for DeepEqual, and there is no
obvious reason to make either choice. So we make it like ==, because
that is simpler to understand and easy to justify.

Ian

[Bakul Shah]

Scheme has EQUAL? which checks structural equality sorta like
DeepEqual:

[using Gambit Scheme]
> (define x (/ 0.0 0.0))
> x
+nan.0
> (= x x)
#f
> (equal? x x)
#t
> (equal?
> (equal? (list x x) (list x (/ 0.0 0.0)))
#t

Like Brendan I don't find the current definition of DeepEqual
to be useful (its use case is different than what one might
need in math/numerical analysis code). At the very least it
may be worth providing an alternative version with a different
name.

[Ian Lance Taylor]

On Thu, Oct 9, 2014 at 3:24 PM, Bakul Shah <ba...@bitblocks.com> wrote:
>
> Like Brendan I don't find the current definition of DeepEqual
> to be useful (its use case is different than what one might
> need in math/numerical analysis code). At the very least it
> may be worth providing an alternative version with a different
> name.

There are many different possible variants of DeepEqual. I don't
think it's useful to provide them all. Most people don't need the
full generality of DeepEqual for their specific application. However,
if you do, the whole function is less than 150 lines, and it doesn't
use any features not available to users of the reflect package in
general. Copy it and modify it to your taste.

Ian

[mkob...@gmail.com]

http://stackoverflow.com/questions/1565164/what-is-the-rationale-for-all-comparisons-returning-false-for-ieee754-nan-values

HTH,

Martin

"Brendan Tracey"<tracey....@gmail.com> wrote:
> Date: October 9, 2014 3:54:22 PM
> From: Brendan Tracey <tracey....@gmail.com>
> To: golan...@googlegroups.com
> Subject: [golang-dev] Reflect DeepEqual and NaN

>
> The current implementation of reflect.DeepEqual returns false if there are
> any NaN values in the structure. For example
>
> http://play.golang.org/p/_hagR4R_y0
> a := math.NaN()
> b := math.NaN()
> reflect.DeepEqual(a,b) // returns false
>
> It seems to me like the above should return true given the point of the
> test. Should I file an issue, or am I missing something? If I am missing
> something, is there any way to perform a deep equal where NaN is considered
> equal?
>
> I do not feel changing the behavior would be breaking Go 1 compatibility.
> The documentation says "uses == where possible", but it is not possible to
> use "==" to compare if NaNs are equal.
>

[Egon Elbre]

It seems you are using NaN where you should be using NA. NaN means there was a computation error. NA is for missing values, of course, Go doesn't have built-in notion for NA.

There are few ways to handle this, replace the value with a "stub" e.g. NaN/Inf/-Inf or some other unrepresentable value, replace float64 with *float64, custom type that tracks NA-ness.

If you want to check equalness of NaN you can just do:

func EqualAlternate(a, b float64) bool {

    if math.IsNaN(a) == math.IsNaN(b) {

        return true

    }

    return a == b

}

+ egon

 

[Ian Davis]

I second this, it's very easy to do, and it scales to every use case :)

I did it recently to modify DeepEqual to provide a hint of where the
equality test failed:

https://github.com/iand/deepequal

Ian

[saint....@gmail.com]

On Friday, 10 October 2014 08:53:08 UTC+1, Egon Elbre wrote:

It seems you are using NaN where you should be using NA. NaN means there was a computation error. NA is for missing values, of course, Go doesn't have built-in notion for NA.

To add to this, Go does of course have nil: a value that both represents unset and compares as equal (for same types).

[roger peppe]

On 10 October 2014 11:06, Ian Davis <m...@iandavis.com> wrote:
> I did it recently to modify DeepEqual to provide a hint of where the
> equality test failed:
>
> https://github.com/iand/deepequal

I've also done this in the past. You might find it useful actually,
as it provides a somewhat better hint (it actually tells
you *where* the comparison fails).

http://godoc.org/github.com/juju/testing/checkers#DeepEqual

It has saved me lots and lots of time over the last year or so.

[Ian Davis]

Yes, this is better than my version which admittedly was created simply
to debug a specific problem I had at the time. Just shows how hackable
the Go standard library is :)

Ian