Hexadecimal constants not checked for overflow in lexer [lua 5.4].

[dlroweht]

Any reason why hexadecimal constant values are not checked for overflow? Decimal constants are checked and work as expected one would expect the same for hexadecimal constants (lua_Integer).

After numeral is read by lexer it gets checked by 'luaO_str2num' this in turn this calls 'l_str2int' and the relevant part of this function:

for (; lisxdigit(cast_uchar(*s)); s++) {
    a = a * 16 + luaO_hexavalue(*s);
    empty = 0;

}

as you can see no overflow checks (hexadecimal).

In case constant value is not hexadecimal then it is assumed it is decimal:

for (; lisdigit(cast_uchar(*s)); s++) {
    int d = *s - '0';
    if (a >= MAXBY10 && (a > MAXBY10 || d > MAXLASTD + neg))  /* overflow? */
        return NULL;  /* do not accept it (as integer) */
    a = a * 10 + d;
    empty = 0;

}

so decimal constant values are checked for overflow.

[Sainan]

I'm not sure if I would quite call this behaviour an overflow, although it is weird that there's no parser warning or error:

print(string.format("%X", 0x123456789ABCDEFDEADCAFE)) --> 89ABCDEFDEADCAFE

[dlroweht]

So after browsing the source a bit more I found that even if you type out huge

constant decimal value lexer won't report the error (or your LSP).

And the reason is that even if overflow happens when lexer tries to convert

that string to actual integer value the function will just return NULL.

And Lua will automatically try to convert this value to 'lua_Number' (floating point)

instead of reporting overflow.

I think this is just plain wrong, it is fine to try to convert the same value to floating

point but only in case there is no overflow.

I would recommend that functions 'l_str2int' and 'luaO_str2num' start accepting additional

integer pointer as overflow flag.

This way the lexer can detect overflows in integer constants.

Instead, right now the constant value gets converted to 'lua_Number' and you silently

overflow by losing precision.

We can see in 'luaO_str2num':

size_t luaO_str2num (const char *s, TValue *o) {
  lua_Integer i; lua_Number n;
  const char *e;
  if ((e = l_str2int(s, &i)) != NULL) {  /* try as an integer */
    setivalue(o, i);
  }
  else if ((e = l_str2d(s, &n)) != NULL) {  /* else try as a float */
    setfltvalue(o, n);
  }
  else
    return 0;  /* conversion failed */
  return (e - s) + 1;  /* success; return string size */
}

If 'l_str2int' would accept additional integer pointer acting as overflow flag we

could easily do something like:

  else if (!overflow && (e = l_str2d(s, &n)) != NULL) {  /* else try as a float */
    setfltvalue(o, n);
  }

this would then correctly return '0' and lexer would correctly report all constant

decimal numbers that overflow.

Additionally I see no reason for not checking hexadecimal constant values for overflow

wish any of the devs could address this.

I get it that nobody in their right mind would just write constant number values that

straight up overflow, but this seems like a trivial fix.

[dlroweht]

Something like this:

size_t luaO_str2num (const char *s, TValue *o, int *of) {

        lua_Integer i; lua_Number n;
        const char *e;

int iof;

if (of) *of = iof = 0;
if ((e = l_str2int(s, &i, &iof)) != NULL) {
setivalue(o, i);
} else if ((e = l_str2d(s, &n, of)) != NULL) {
setfltvalue(o, n);
} else { /* both conversions failed */
if (of && !*of) *of = iof;
return 0;

}
return (e - s) + 1;

}

and then have static functions 'l_str2*' check for overflow in any way that is portable for C89.