On 3/25/21 4:56 PM, Albrecht Schlosser wrote:
f = 9.0/8.0 = 1.125 = 1.12 = ....well I didn't check them all, but I certainly didn't expect /that/ :^D
If you've ever seen the movie "Office Space", written and
directed by the guy that brought us
Beavis and Butthead, I believe this type of thing was one of
the core plot lines.
I was brought up with a single rounding rule, that anything greater or equal to .5 rounded to the nearest whole away from zero, i.e. rounding 1.5 gives 2, and rounding -1.5 gives -2. In my programming life, I've never consciously noticed anything the contradicted this, but maybe I never really noticed.
--
You received this message because you are subscribed to the Google Groups "fltk.coredev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkcoredev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fltkcoredev/9c9892ba-d6a9-95dc-9c71-04d8ea858169%40online.de.
Maybe my expectation is "wrong" but we see that others had the same expectation.
On 3/25/21 7:52 PM, Greg Ercolano wrote:
I'm not sure, but I think the printf() mechanism's "rounding mode"
might be controllable in some cases, e.g. awk/gawk:
https://www.gnu.org/software/gawk/manual/html_node/Setting-the-rounding-mode.html
It's funny, that page cites the 0.125 case exactly. Quoting
that gawk manual page:
The default mode roundTiesToEven is the most preferred, but the least intuitive. This method does the obvious thing for most values, by rounding them up or down to the nearest digit. [..]
However, when it comes to rounding a value that is exactly halfway between, things do not work the way you probably learned in school. In this case, the number is rounded to the nearest even digit. So rounding 0.125 to two digits rounds down to 0.12, but rounding 0.6875 to three digits rounds up to 0.688.