Format as decimal number? i think it doesn't work well

[Marco Modica]

Hi .. I have noticed that there is a problem with a "Format as decimal number" function. I am doing various tests I noticed that if I write anything larger than 15 digits, the digits more than 15 stoned, an example can be put 10/3 that I want to know what are the first 24 digits it will calculate 3.33 333 333 333 333 165 052 935. One can immediately see what the problem is.

Now let me see if I'm wrong to use it or if it's some sort of bug or does not work well

[Abraham Getzler]

From your experiment and from the various flavors of floating point number described at

https://en.wikipedia.org/wiki/IEEE_floating_point

it looks like AI2's Format as Decimal block sends your value

through a floating point number to generate its digits.

It's probably one of the 15 or 16 digit flavors.

ABG

[Evan Patton]

App Inventor stores decimal numbers as Java double floating point numbers, which is typically IEEE 754 (binary64 from the link Abraham provided). In the general case, rendering these values are not accurate beyond 15 digits. There's not really much we can do about this because it is a function of the floating point hardware of the device. Theoretically, if Google released a phone tomorrow where the Android runtime assumed a double that was 128 bits and the hardware supported that, then you would get a number that is accurate to 34 digits, but that just simply won't happen as it would break compatibility with all of the existing hardware and software.

Evan

[Abraham Getzler]

If you want infinite precision, be aware that AI2 has a sub-layer that knows rational numbers, where

division of integers is deferred as long as possible and both the numerator and denominator are kept internally.

See the attached example for proof.

A floating point intermediate result would be incapable of supporting 50 digits after multiplication.

Mind blown yet?

ABG