Parsing decimal value of 0xFFFFFFFF (4294967295) renders strange error

[Per Franck]

I have a string of text, as follows.

{"command":"RegWr", "register": 12, "value":
4294967295,"function":"Spartan"}

Which the json_loads function chokes on. The error occurs when reading
the last '}' and the error text is:

'[' or '{' expected near '"Spartan"'

using the string

{"command":"RegWr", "register": 12, "value":
429496729,"function":"Spartan"}

removed the trailing 5 on the number, works fine

Anyone else seeing this? I'm running MinGW with GCC.

- Per

[Deron Meranda]

> I have a string of text, as follows.
>
> {"command":"RegWr", "register": 12, "value":
> 4294967295,"function":"Spartan"}
>
> Which the json_loads function chokes on.

Are you compiling a 32-bit executable (with 32-bit "long"), or 64-bit?
If you're 32-bit then you should get an error about a numeric
overflow, because 0xFFFFFFFF to too big to fit into a *signed* 32-bit
integer. The error you should see is:

too big integer near '4294967295'

What version of Jansson are you using?

And do you still get the same error if you simplify the JSON input
into the minimum necessary:
[4294967295]

--
Deron Meranda
http://deron.meranda.us/

[Per Franck]

Hello, Deron.

The version of Jansson is 2.3.1, and it's compiled as a 32-bit
executable. Looking in jansson.h, i see that it's using

#define JSON_INTEGER_FORMAT "I64d"
...
typedef long long json_int_t;

Looking over my error log i actually do see the error 'too big integer
near 4294967295' It was just buried in a slew of other messages. Now,
is this good behavior for a JSON lexer? I don't argue that flagging an
integer overflow has occured is a bad idea, but isn't this kind of
like defining a specific flavor of JSON? If i send perfectly valid
JSON on a 64-bit machine to a 32-bit machine, I don't think I would
expect the other machine to choke like that... It makes JSON machine
dependent all of a sudden. Anyway, just a thought.

[Deron Meranda]

> The version of Jansson is 2.3.1, and it's compiled as a 32-bit
> executable. Looking in jansson.h, i see that it's using
>
> #define JSON_INTEGER_FORMAT "I64d"
> ...
> typedef long long json_int_t;

So evidently the 'long long' type in your environment is also 32-bit.
Many 32-bit environments support an emulated 64-bit long long type.
Are there any compiler flags or such that you could use to get a
64-bit long long even though you're 32-bit?

> Looking over my error log i actually do see the error 'too big integer
> near 4294967295' It was just buried in a slew of other messages. Now,
> is this good behavior for a JSON lexer? I don't argue that flagging an
> integer overflow has occured is a bad idea, but isn't this kind of
> like defining a specific flavor of JSON? If i send perfectly valid
> JSON on a 64-bit machine to a 32-bit machine, I don't think I would
> expect the other machine to choke like that... It makes JSON machine
> dependent all of a sudden. Anyway, just a thought.

Well, that's the case for just about any implementation of JSON in any
language (perhaps with very few exceptions). The JSON spec defines a
syntax that is completely abstract, but eventually you've got to map
it to some specific machine or implementation. You'll even run into a
very similar restriction when using JSON inside of JavaScript.

As an aside, I had been working on an extension to Jansson which would
allow it to use "infinite precision" numeric packages, so that you
would not have to be limited to whatever size your "long long" was. If
you scan this mailing list archives you'll find my "prototype"
patches, which work, but aren't official yet (and will require a few
minor changes before they are accepted .... which I need to get back
to doing).

Anyway, Jansson is for the most part very good at implementing the
complete JSON spec, but there are a few limitations which are
thoroughly documented.

See http://www.digip.org/jansson/doc/2.3/conformance.html

[Per Franck]

Makes sense. Curiously, just trying out code on my end ( i commented
out the code block that gives the overflow error in jansson
\load.c ) ...

long long ll = 2147483648; // 0x8000 0000
json_t* s = json_loads( "{\"V\":2147483648}", 0, &err );

printf("value (json): %lld\n", json_integer_value(json_object_get(s,
"V")) );
printf("value (sys): %lld\n", ll );
printf("value (sys): %09X\n", ll );

--------------------------
value (json): 2147483647
value (sys): 2147483648
value (sys): 080000000

if json_int_t is defined as long long... then shouldn't the jansson
source be able to handle that value?

[Per Franck]

ok, it seems json_strtoint is defined as strtol... let me
investigate...

[Per Franck]

changed to this:

1. #define json_strtoint(x,y,z) atoll(x)
2. commented out line: //assert(end == saved_text + lex-
>saved_text.length);

now it works fine.

[Deron Meranda]

> changed to this:
>
> 1. #define json_strtoint(x,y,z) atoll(x)
> 2. commented out line: //assert(end == saved_text + lex-
>>saved_text.length);
>
> now it works fine.

So is it the case that your environment does have 64-bit 'long long'
integers, but that it doesn't support the corresponding C standard
functions, such as strtoll()? Or do you have to link in an additional
library or provide a CFLAGS argument or something?

Generally, using atoll() rather than strtoll() is less robust;
especially in error detection. If you can possibly get your
environment to use strtoll() (if it exists), it would be better. And
better yet would be to see if this is a problem with the configure
process and if it's possible to fix that instead.

The long long support is detected by the configure scripts, which then
will create a custom jansson_config.h header. Look for the macro
JSON_INTEGER_IS_LONG_LONG in jansson_config.h.

[Per Franck]

strtoll exists and works great.

[Petri Lehtinen]

Per Franck wrote:
> strtoll exists and works great.

So you have long long and strtoll(). Are you using the ./configure
script? Can you check src/jansson_config.h to see whether
JSON_INTEGER_IS_LONG_LONG is defined to 1 or 0?