int16 and float64 Multiplication

[ngju...@gmail.com]

I am trying to multiply an int16 and float64 in Golang 1.8 but the result it returns is incorrect. I need this number to be exact for some hash verification. Any ideas why this might be happening?

int16(10) // 10
float64(int16(10)) // 10

float64(3.99) // 3.99

float64(int16(10)) * float64(3.99) // 39.900000000000006

[Ian Lance Taylor]

That is how floating point numbers work. The number 3.99 can not be
represented precisely in the 64-bit floating point format, so the
result of the multiplication has errors. If you store 3.99 in a
float64 variable and print it however you are printing the result of
the multiplication, so you will see that it too has extraneous every
low precision bits.

Please see https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html .

Ian

[ngju...@gmail.com]

Thank you for your help. I managed to get the result I needed by using 32 bit floats.

[Michael Jones]

This feels unlikely. 

You might want to verify that the calculation matches your expectations exactly at many values. Ian's representation argument applies at every word size it may be that you just hit a "lucky" happenstance. 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Michael T. Jones
michae...@gmail.com

[ngju...@gmail.com]

After some testing I just decided to instead round the numbers to 2 decimals instead. Thank you for the heads up though.

[Michael Jones]

good.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[simon place]

if using fixed precision then don't use floats, just use int's with a fixed multiplier.