Inconsistent rounding with float printf ?

[Christophe Meessen]

I have noticed that printf performs an apparently inconsistent rounding of floating point values.

I divide a big number by 1000 and printf the resulting value with "%.1f".

Here is the code: https://play.golang.org/p/e7dD3c6IHq2

I would expect the rounding rule to be "round away from zero" as defined here: https://math.stackexchange.com/a/2252888/33796

In this case 0.5 is rounded to 1 (or 0.05 to 0.1) and -0.5 to -1 (or -0.05 to -0.1).

Here is what I see with golang 1.13.4:

999000/1000 = 999.0
999050/1000 = 999.0 // expected 999.1
999100/1000 = 999.1
999150/1000 = 999.1 // expected 999.2
999200/1000 = 999.2
999250/1000 = 999.2 // expected 999.3
999300/1000 = 999.3
999350/1000 = 999.4 
999400/1000 = 999.4
999450/1000 = 999.5 
999500/1000 = 999.5
999550/1000 = 999.5 // expected 999.6
999600/1000 = 999.6
999650/1000 = 999.6 // expected 999.7
999700/1000 = 999.7
999750/1000 = 999.8 
999800/1000 = 999.8
999850/1000 = 999.9 
999900/1000 = 999.9
999950/1000 = 1000.0 
-999950/1000 = -1000.0 
-999900/1000 = -999.9
-999850/1000 = -999.9  
-999800/1000 = -999.8
-999750/1000 = -999.8
-999700/1000 = -999.7
-999650/1000 = -999.6 // expected -999.7
-999600/1000 = -999.6
-999550/1000 = -999.5 // expected -999.6
-999500/1000 = -999.5
-999450/1000 = -999.5
-999400/1000 = -999.4
-999350/1000 = -999.4
-999300/1000 = -999.3
-999250/1000 = -999.2 // expected -999.3
-999200/1000 = -999.2
-999150/1000 = -999.1 // expected -999.2
-999100/1000 = -999.1
-999050/1000 = -999.0 // expected -999.1

Is this actual rounding the result of a rule or is it pseudo random ?

[Ian Davis]

On Fri, 6 Dec 2019, at 9:25 AM, Christophe Meessen wrote:

I have noticed that printf performs an apparently inconsistent rounding of floating point values.

I divide a big number by 1000 and printf the resulting value with "%.1f".

Here is the code: https://play.golang.org/p/e7dD3c6IHq2

I think you are just seeing the usual problems of floating point representation.

You may wonder why 999450/1000=999.45, 999500/1000=999.50 and 999550/1000=999.55 all format as 999.5. The answer is that the internal representation of the three results cannot correspond to the mathematical result you expect.

This link shows the internal representation of each answer: https://play.golang.org/p/bBTNCdsAttR

You can see that 999550/1000 = 999.549999999999954525264911353588104248046875 which is printed as 999.5

I would expect the rounding rule to be "round away from zero" as defined here: https://math.stackexchange.com/a/2252888/33796

In this case 0.5 is rounded to 1 (or 0.05 to 0.1) and -0.5 to -1 (or -0.05 to -0.1).

The strconv and fmt packages use round to even as a rule. Use math.Round to round away from zero.

-- Ian

[Michael Jones]

Agree with Ian. 

Solutions are: change expectations, use decimal floating point, or use a base-independent decimal representation. The latter implies scaled integers.

Quick, ugly, and typed on one hand from bed, but here it is: https://play.golang.org/p/fBztRY6qHP0

999000/1000 = 999.0
999050/1000 = 999.1
999100/1000 = 999.1
999150/1000 = 999.2
999200/1000 = 999.2
999250/1000 = 999.3
999300/1000 = 999.3
999350/1000 = 999.4
999400/1000 = 999.4
999450/1000 = 999.5
999500/1000 = 999.5
999550/1000 = 999.6
999600/1000 = 999.6
999650/1000 = 999.7
999700/1000 = 999.7
999750/1000 = 999.8
999800/1000 = 999.8
999850/1000 = 999.9
999900/1000 = 999.9
999950/1000 = 1000.0
-999950/1000 = -1000.0
-999900/1000 = -999.9
-999850/1000 = -999.9
-999800/1000 = -999.8
-999750/1000 = -999.8
-999700/1000 = -999.7
-999650/1000 = -999.7
-999600/1000 = -999.6
-999550/1000 = -999.6
-999500/1000 = -999.5
-999450/1000 = -999.5
-999400/1000 = -999.4
-999350/1000 = -999.4
-999300/1000 = -999.3
-999250/1000 = -999.3
-999200/1000 = -999.2
-999150/1000 = -999.2
-999100/1000 = -999.1
-999050/1000 = -999.1

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ee94624c-5485-4daf-98ad-8e59055056dd%40www.fastmail.com.

--

Michael T. Jones
michae...@gmail.com

[Robert Engels]

You can use github.com/robaho/fixed

On Dec 6, 2019, at 10:19 AM, Michael Jones <michae...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CALoEmQy%3DMCWgb69GgsZwAv2GT4ar%3DWrdvn212sFK1PfGES1ijw%40mail.gmail.com.

[Christophe Meessen]

I can't change expectations. It is to convert a byte count into a human readable byte count ( with kB, MB, ... units). 

I found out that I can produce the expected result by using math.Round. See here https://play.golang.org/p/UorDwbKlLj5

For my use case, I ended up converting "manually" the integer into a .1f float by using an array. The resulting code is 12 time faster than the one using the printf and float64 operations. It also need only one allocation to convert the byte array into a string. 

[Jan Mercl]

On Fri, Dec 6, 2019 at 6:04 PM Christophe Meessen
<christoph...@gmail.com> wrote:

> I can't change expectations. It is to convert a byte count into a human readable byte count ( with kB, MB, ... units).

So it was an XY problem?

No floating point operations are necessary to do that. Also, check
several existing libraries, for example this one:
https://github.com/dustin/go-humanize

[crate...@gmail.com]

https://play.golang.org/p/j5HKxitS-Z6

See  https://0.30000000000000004.com/