How to deal with floating point rounding error

[Johnny Oshika]

How is everyone dealing with the floating point rounding error that your users may encounter when using Blockly? For example, if you use the Blockly math block and try 0.1 + 0.2, you will get 0.30000000000000004.

This isn't a Blockly problem as it's the expected behavior of all modern programming languages (https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html), but it's an unexpected behavior to my users.

Curious how everyone else is dealing with this.

[Johnny Oshika]

Here's a screenshot of my example:

[Coda Highland]

You deal with it the same way you deal with it anywhere else: you choose an output precision and round in the rendering code. Math.round(n * 1e8) / 1e8 or similar will probably do the trick.

/s/ Adam

--
You received this message because you are subscribed to the Google Groups "Blockly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blockly+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/bf078a51-2bad-4e0a-9a21-32adef1f8169n%40googlegroups.com.

[Johnny Oshika]

Thank you Adam

[Johnny Oshika]

For anyone else who comes across this problem, I corrected it like this. I started with Adam's suggestion, but accounted for those cases where rounding would be inappropriate:

const correctRoundingError = (num: number) => {

  if (num === 0) return num;

  const rounded = Math.round(num * 1e8) / 1e8;

  // Set a 1e-7 difference ratio where changing the number by more than that amount is inappropriate

  if (Math.abs(1 - rounded / num) > 1e-7) return num;

  else return rounded;

};

Examples:

correctRoundingError(0.1+0.2) // correctly rounds to 0.3

correctRoundingError(1.123456e-4) // correctly does not round, otherwise it would become 1.1235e-4

correctRoundingError(1e-9) // correctly does not round, otherwise it would become 0

[feni...@google.com]

The number field has optional constraints that you can set, including precision.  See the documentation for details.

Cheers,

Rachel

[Johnny Oshika]

Thanks Rachel, that's a nice option for custom blocks.

For anyone who's interested, I've updated my correctRoundingError function using JavaScript's Number.prototype.toPrecision(): https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toPrecision

const correctRoundingError = (num: number) =>

  parseFloat(num.toPrecision(14));

I chose a precision of 14 because JavaScript number's precision limit is about 16 digits and 14 is just below that. Math.js also uses 14 by default: https://github.com/josdejong/mathjs/blob/develop/docs/datatypes/numbers.md#round-off-errors

correctRoundingError(0.1+0.2) // 0.3

correctRoundingError(2 / 3) // 0.66666666666667

correctRoundingError(123.45678901234) // 123.45678901234 (does not round)