Encoding NaN in JSON

[Miki Tebeka]

Greetings,

I'm trying to find a way to have json emit float('NaN') as 'N/A'.
I can't seem to find a way since NaN is a float, which means overriding "default" won't help.

Any simple way to do this?

Thanks,
--
Miki

[Tim Roberts]

Miki Tebeka <miki....@gmail.com> wrote:
>
>I'm trying to find a way to have json emit float('NaN') as 'N/A'.
>I can't seem to find a way since NaN is a float, which means overriding "default" won't help.
>
>Any simple way to do this?

No. There is no way to represent NaN in JSON. It's simply not part of the
specification. From RFC 4627 section 2.4:

Numeric values that cannot be represented as sequences of digits
(such as Infinity and NaN) are not permitted.
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.

[Miki Tebeka]

>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.

> No. There is no way to represent NaN in JSON. It's simply not part of the
> specification.

I know that. I'm trying to emit the *string* 'N/A' for every NaN.

[John Gordon]

import math

x = possibly_NaN()

if math.isnan(x):
x = 'N/A'

--
John Gordon A is for Amy, who fell down the stairs
gor...@panix.com B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

[Johann Hibschman]

Easiest way is probably to transform your object before you try to write
it, e.g.

def transform(x):
if isinstance(x, dict):
return dict((k, transform(v)) for k, v in x.items())
elif isinstance(x, list) or isinstance(x, tuple):
return [transform(v) for v in x]
elif isinstance(x, float) and x != x:
return 'N/A'
else:
return x

Then just use

json.dumps(transform(x))

rather than just

json.dumps(x)

[Miki Tebeka]

> >>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.

> Easiest way is probably to transform your object before you try to write

Yeah, that's what I ended up doing. Wondered if there's a better way ...

Thanks,
--
Miki

[Dave Angel]

On 04/17/2013 03:05 PM, Johann Hibschman wrote:
> Miki Tebeka <miki....@gmail.com> writes:
>
>>>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.
>>> No. There is no way to represent NaN in JSON. It's simply not part of the
>>> specification.
>> I know that. I'm trying to emit the *string* 'N/A' for every NaN.
>
> Easiest way is probably to transform your object before you try to write
> it, e.g.
>
> def transform(x):
> if isinstance(x, dict):
> return dict((k, transform(v)) for k, v in x.items())
> elif isinstance(x, list) or isinstance(x, tuple):
> return [transform(v) for v in x]
> elif isinstance(x, float) and x != x:
> return 'N/A'
> else:
> return x
>

Note that for a self-referencing object, this function might run
"forever," or until it runs out of stack. The programmer is likely to
know about the possibility, but just in case ...

> Then just use
>
> json.dumps(transform(x))
>
> rather than just
>
> json.dumps(x)
>

--
DaveA

[Roland Koebler]

Hi,

> > Easiest way is probably to transform your object before you try to write
> Yeah, that's what I ended up doing. Wondered if there's a better way ...

yes, there is: subclass+extend the JSON-encoder, see pydoc json.

e.g.:
class JsonNanEncoder(json.JSONEncoder):
def default(self, obj):
if some-check-if-obj-is-NaN:
return 'NaN'
return json.JSONEncoder.default(self, obj)

Roland

[Miki Tebeka]

[Roland]

> yes, there is: subclass+extend the JSON-encoder, see pydoc json.

Please read the original post before answering. What you suggested does not work since NaN is of float type.

[Roland Koebler]

Hi,

> > yes, there is: subclass+extend the JSON-encoder, see pydoc json.
> Please read the original post before answering. What you suggested does not work since NaN is of float type.

ok, right, default does not work this way.
But I would still suggest to extend the JSON-encoder, since that is
quite simple (see sourcecode of JSON module); as a quickhack, you
could even monkey patch json.encoder.floatstr with a wrapper which
returns "N/A" for NaN. (I've tested it: It works.)

But: If you only need NaN and inf, and are ok with 'NaN' instead of 'N/A',
you can simply use the json module. See pydoc json:

If allow_nan is True, then NaN, Infinity, and -Infinity will be
encoded as such. This behavior is not JSON specification compliant,
but is consistent with most JavaScript based encoders and decoders.
Otherwise, it will be a ValueError to encode such floats.

>>> import json
>>> json.dumps(float('NaN'))
'NaN'
>>> json.dumps(float('inf'))
'Infinity'


Roland

[Chris Angelico]

You may be able to override a bit more of the code, though. Check out
Lib/json/encoder.py for the implementation, and have a look at the
floatstr() internal function; unfortunately you can't simply subclass
and override that, but perhaps overriding iterencode (which is where
floatstr is defined) would do the job.

ChrisA

[Chris Angelico]

On Thu, Apr 18, 2013 at 11:39 AM, Roland Koebler <r.ko...@yahoo.de> wrote:
> as a quickhack, you
> could even monkey patch json.encoder.floatstr with a wrapper which
> returns "N/A" for NaN. (I've tested it: It works.)

Wait... you can do that? It's internal to iterencode, at least in
Python 3.3 and 2.7 that I'm looking at here. Can you share your code
please? I'd like to try that! When I first looked at the docstring, I
was thinking "Ah, can I override the bit that emits NaN to return
\"N/A\" instead?", but the code made me think that's not possible.

ChrisA

[Roland Koebler]

On Thu, Apr 18, 2013 at 11:46:37AM +1000, Chris Angelico wrote:
> Wait... you can do that? It's internal to iterencode, at least in
> Python 3.3 and 2.7 that I'm looking at here.

In Python 2.6 it wasn't internal to iterencode; in Python 2.7 and 3.x
you probably would have to monkey-patch iterencode. (In addition, patching
floatstr alone wouldn't be enough in 3.x and probably 2.7, since you also
have to make sure that the C-extension is not used here.)

BUT: Keep in mind that monkey-patches are problematic, and should be
avoided (or used very carefully) in production code. So, better
replace the complete encoder.py or use your own patched version
of the complete json-module.

Roland

[Wayne Werner]

Why not use `null` instead? It seems to be semantically more similar...

-W

[Tim Roberts]

You understand that this will result in a chunk of text that is not JSON?
Other JSON readers won't be able to read it.

[Robert Kern]

On 2013-04-19 10:34, Tim Roberts wrote:
> Miki Tebeka <miki....@gmail.com> wrote:
>>
>>>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.
>>> No. There is no way to represent NaN in JSON. It's simply not part of the
>>> specification.
>>
>> I know that. I'm trying to emit the *string* 'N/A' for every NaN.
>
> You understand that this will result in a chunk of text that is not JSON?
> Other JSON readers won't be able to read it.

I think he means something like this:

>>> json.dumps([float('nan')])
'["N/A"]'

Not

'[N/A]'

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Grant Edwards]

Why not use 'NaN' instead? It seems to be even more semantically
similar...

--
Grant Edwards grant.b.edwards Yow! I want a VEGETARIAN
at BURRITO to go ... with
gmail.com EXTRA MSG!!

[Chris “Kwpolska” Warrick]

On Fri, Apr 19, 2013 at 4:54 PM, Grant Edwards <inv...@invalid.invalid> wrote:
> On 2013-04-18, Wayne Werner <wa...@waynewerner.com> wrote:
>> On Wed, 17 Apr 2013, Miki Tebeka wrote:
>>
>>>>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.
>>>> No. There is no way to represent NaN in JSON. It's simply not part of the
>>>> specification.
>>> I know that. I'm trying to emit the *string* 'N/A' for every NaN.
>>
>> Why not use `null` instead? It seems to be semantically more similar...
>
> Why not use 'NaN' instead? It seems to be even more semantically
> similar...

Because there is no NaN in JSON? Unless you mean a string, which
makes no semantical sense and is human-oriented and not
machine-oriented.

--
Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16
stop html mail | always bottom-post
http://asciiribbon.org | http://caliburn.nl/topposting.html

[Grant Edwards]

On 2013-04-19, Chris ???Kwpolska??? Warrick <kwpo...@gmail.com> wrote:
> On Fri, Apr 19, 2013 at 4:54 PM, Grant Edwards <inv...@invalid.invalid> wrote:
>> On 2013-04-18, Wayne Werner <wa...@waynewerner.com> wrote:
>>> On Wed, 17 Apr 2013, Miki Tebeka wrote:
>>>
>>>>>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.
>>>>> No. There is no way to represent NaN in JSON. It's simply not part of the
>>>>> specification.
>>>> I know that. I'm trying to emit the *string* 'N/A' for every NaN.
>>>
>>> Why not use `null` instead? It seems to be semantically more similar...
>>
>> Why not use 'NaN' instead? It seems to be even more semantically
>> similar...
>
> Because there is no NaN in JSON? Unless you mean a string, which
> makes no semantical sense and is human-oriented and not
> machine-oriented.

The OP asked for a string, and I thought you were proposing the string
'null'. If one is to use a string, then 'NaN' makes the most sense,
since it can be converted back into a floating point NaN object.

I infer that you were proposing a JSON null value and not the string
'null'?

--
Grant Edwards grant.b.edwards Yow! I'm receiving a coded
at message from EUBIE BLAKE!!
gmail.com

[Miki Tebeka]

> > You understand that this will result in a chunk of text that is not JSON?

> I think he means something like this:
> >>> json.dumps([float('nan')])
> '["N/A"]'

That's exactly what I mean :)

[Chris “Kwpolska” Warrick]

On Fri, Apr 19, 2013 at 9:42 PM, Grant Edwards <inv...@invalid.invalid> wrote:
> The OP asked for a string, and I thought you were proposing the string
> 'null'. If one is to use a string, then 'NaN' makes the most sense,
> since it can be converted back into a floating point NaN object.
>
> I infer that you were proposing a JSON null value and not the string
> 'null'?

Not me, Wayne Werner proposed to use the JSON null value. I parsed
the backticks (`) used by him as a way to delimit it from text and not
as a string.

PS.

> On 2013-04-19, Chris ???Kwpolska??? Warrick <kwpo...@gmail.com> wrote:

Is Unicode support so hard, especially in the 21st century?

[Wayne Werner]

On Sat, 20 Apr 2013, Chris “Kwpolska” Warrick wrote:

> On Fri, Apr 19, 2013 at 9:42 PM, Grant Edwards <inv...@invalid.invalid> wrote:
>> The OP asked for a string, and I thought you were proposing the string
>> 'null'. If one is to use a string, then 'NaN' makes the most sense,
>> since it can be converted back into a floating point NaN object.
>>
>> I infer that you were proposing a JSON null value and not the string
>> 'null'?
>

> Not me, Wayne Werner proposed to use the JSON null value. I parsed
> the backticks (`) used by him as a way to delimit it from text and not
> as a string.

That was, in fact, my intention. Though it seems to me that you'll have to
suffer between some sort of ambiguity - in Chrome, at least,
`Number(null)` evaluates to `0` instead of NaN. But `Number('Whatever')`
evaluates to NaN. However, a JSON parser obviously wouldn't be able to
make the semantic distinction, so I think you'll be left with whichever
API makes the most sense to you:

NaN maps to null

or

NaN maps to "NaN" (or any other string, really)


Obviously you're not limited to these particular choices, but they're
probably the easiest to implement and communicate.

HTH,
-W