Loss of precision when passing large integers from Lua to Redis

[Simeon Simeonov]

I'm posting this to hopefully save others the many hours of debugging it took me to discover why a complex Lua script occasionally misbehaved. The summary is that the default conversion of numbers from Lua to Redis does not use a sufficient number of digits during stringification. I've added this issue on GitHub.

Best,

Sim

[Josiah Carlson]

I left the same comment on the bug, but for those readers that don't feel like clicking the link:

Lua's numbers are IEE 754 floating point doubles. Which means only exact precision for integers up to +/- 2**53. This is expected behavior.

 - Josiah

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Javier Guerra Giraldez]

On Mon, May 20, 2013 at 11:34 PM, Josiah Carlson
<josiah....@gmail.com> wrote:
> Lua's numbers are IEE 754 floating point doubles. Which means only exact
> precision for integers up to +/- 2**53. This is expected behavior.

The example shows how 2^53-1 is not correctly preserved. in theory,
that should be the last exact integer, but it seems to be rounded to
the nearest thousand.

--
Javier

[Josiah Carlson]

Okay. Then a format string should be updated, Simeon was right. That said, I wouldn't do any operations on large integers in Lua. There was discussion about adding a big number library, I can't remember if that happened.

 - Josiah

[Simeon Simeonov]

Javier, this is not the largest exact integer. There is a huge number of larger exact integers that IEEE 754 can represent precisely, however, it cannot do so continuously. In other words, for x > (2^53) - 1, if x were to be represented in IEEE 754 without loss of precision, it is guaranteed that x + 1 cannot be represented without loss of precision.

[Simeon Simeonov]

Why not do operations on large numbers? 

In my case, I'm packing multiple values into 52 bits and doing parallel HINCRBYs halving the number of Redis operations I have to do and saving a massive amount of RAM & protocol traffic. I can't think of any other way to get this type of performance out of Redis...

[Javier Guerra Giraldez]

On Tue, May 21, 2013 at 1:08 AM, Simeon Simeonov <s...@swoop.com> wrote:
> Javier, this is not the largest exact integer.

right. more correct would be to say that it's an "IEEE754
representable integer"

--
Javier

[Simeon Simeonov]

+1