--
You received this message because you are subscribed to the Google Groups "Macaulay2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to macaulay2+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/macaulay2/c8fbbb5a-1679-4f4a-b0ca-9dd7ae9d453fn%40googlegroups.com.
Hi Irena,
Often these spurious missing digits are correct according to the IEEE standard for floating point numbers: https://en.wikipedia.org/wiki/IEEE_754
If I do this in python (which is 100% IEEE correct) I get:
10594.48 - 10514.44
80.03999999999905
(which is different from your first M2 result).
Now, the classical example that everyone tries is this one here:
Python:
0.1+0.2
0.30000000000000004
M2:
i1 : 0.1+0.2
o1 = .3
o1 : RR (of precision 53)
And now for the fun part, your first example gives yet another result on my machine:
(Macaulay2, version 1.24.05 on MacOS)
i2 : d = 10594.48
o2 = 10594.48
o2 : RR (of precision 53)
i3 : e = 10514.44
o3 = 10514.44
o3 : RR (of precision 53)
i4 : d-e
o4 = 80.0399999999991
While the second is the same:
i7 : d = 10594.48, e = 10514.44, d-e
o7 = (10594.5, 10514.4, 80.04)
Of course I have not answered your question why these two ways to call the C++ library give different results. That I don’t know. I just wanted to say that sometimes these strange results can be correct according to a standard even if not mathematically correct. ¯_(ツ)_/¯
Best,
Thomas
--
You received this message because you are subscribed to the Google Groups "Macaulay2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to macaulay2+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/macaulay2/742cbdc3-ed1b-4ee0-914e-8bd49c45301en%40googlegroups.com.
Thomas Kahle
https://www.thomas-kahle.de
To view this discussion on the web visit https://groups.google.com/d/msgid/macaulay2/29EC409F-93FF-4BE4-8AB4-187C5CBD8E9B%40jpberlin.de.