errors with json.loads

[john polo]

Greetings,

I am using IPython 6.1.0 with Python 3.6.2 on a Windows 7 machine. I am
not a programmer. I am using a book called Python Data Analytics to try
to learn some of Python. I am at a section for reading and writing JSON
data. The example JSON file is:


Listing 5-13.  books.json
[{"writer": "Mark Ross",
 "nationality": "USA",
 "books": [
         {"title": "XML Cookbook", "price": 23.56},
         {"title": "Python Fundamentals", "price": 50.70},
         {"title": "The NumPy library", "price": 12.30}
             ]
},
{"writer": "Barbara Bracket",
 "nationality": "UK",
 "books": [
         {"title": "Java Enterprise", "price": 28.60},
         {"title": "HTML5", "price": 31.35},
         {"title": "Python for Dummies", "price": 28.00}
             ]
}]

and the example code for reading the file is:

>>> file = open('books.json','r')
>>> text = file.read()
>>> text = json.loads(text)

When I use those 3 lines, I get the following:

JSONDecodeError                           Traceback (most recent call last)
<ipython-input-22-7aafd41f326e> in <module>()
----> 1 text = json.loads(text)

c:\users\..\python\python36\lib\json\__init__.py in loads(s, encoding,
cls, object_hook, parse_float, parse_int, parse_constant,
object_pairs_hook, **kw)
    352             parse_int is None and parse_float is None and
    353             parse_constant is None and object_pairs_hook is
None and not kw):
--> 354         return _default_decoder.decode(s)
    355     if cls is None:
    356         cls = JSONDecoder

c:\users\..\python\python36\lib\json\decoder.py in decode(self, s, _w)
    337
    338         """
--> 339         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    340         end = _w(s, end).end()
    341         if end != len(s):

c:\users\..\python\python36\lib\json\decoder.py in raw_decode(self, s, idx)
    353         """
    354         try:
--> 355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
    357             raise JSONDecodeError("Expecting value", s,
err.value) from None

JSONDecodeError: Expecting ':' delimiter: line 5 column 50 (char 161)

?json.loads says that the method is for deserializing "s", with "s"
being a string, bytes, or bytearray.

In [24]: type(text)
Out[24]: str

So "text" seems to be a string. Why does json.loads return an error?


John

[John Gordon]

In <mailman.101.1505945...@python.org> john polo <jp...@mail.usf.edu> writes:

> JSONDecodeError: Expecting ':' delimiter: line 5 column 50 (char 161)

> ?json.loads says that the method is for deserializing "s", with "s"
> being a string, bytes, or bytearray.

> In [24]: type(text)
> Out[24]: str

> So "text" seems to be a string. Why does json.loads return an error?

Because, although text is the right type, it does not contain a
valid json string.

--
John Gordon A is for Amy, who fell down the stairs
gor...@panix.com B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

[Bill]

Interesting problem, John.

I have probably even less experience with json than you do, so I'm
taking this as an opportunity to learn with you.

Suggestions:

1. Try your example with Python 2 rather than Python 3.
2. Take your file and make it into a string literal in your program, and
try calling json.loads with that as an argument.

Share with us what happens!

Good luck,
Bill

[Bill]

john polo wrote:
> Greetings,
>
> I am using IPython 6.1.0 with Python 3.6.2 on a Windows 7 machine. I
> am not a programmer. I am using a book called Python Data Analytics to
> try to learn some of Python. I am at a section for reading and writing
> JSON data. The example JSON file is:
>
>
> Listing 5-13. books.json
> [{"writer": "Mark Ross",
> "nationality": "USA",
> "books": [
> {"title": "XML Cookbook", "price": 23.56},
> {"title": "Python Fundamentals", "price": 50.70},
> {"title": "The NumPy library", "price": 12.30}
> ]
> },
> {"writer": "Barbara Bracket",
> "nationality": "UK",
> "books": [
> {"title": "Java Enterprise", "price": 28.60},
> {"title": "HTML5", "price": 31.35},
> {"title": "Python for Dummies", "price": 28.00}
> ]
> }]
>
> and the example code for reading the file is:
>
> >>> file = open('books.json','r')
> >>> text = file.read()
> >>> text = json.loads(text)

Hmm... are you sure you want "text" on the left hand side. I'm not sure
whether this is okay or not. It looks suspicious (but, like I said, I'm
a Python newbie).

[Bill]

Stefan Ram wrote:
> Dennis Lee Bieber <wlf...@ix.netcom.com> writes:
>> After removing all the \xa0 bytes
>> and trying to decode it I get...

Apparenty an \xa0 byte corresponds to a "non-breaking space". What sort
of white space characters are allowed in a json file ( tabs and
newlines?)? Just curious.

Bill

> I did the same here, before I read your post.
> I got the same results, but did not post them.
>
> Someone has posted programs with \xA0 (NBSP IIRC)
> at the start of lines of the soure here before, in:
>
> From: Christopher Reimer <christoph...@yahoo.com>
> Newsgroups: comp.lang.python
> Subject: Setting property for current class from property in an different class...
> Date: Wed, 6 Sep 2017 19:24:40 -0700
> Message-ID: <mailman.144.1504751...@python.org>
>
> , and when I removed them, I was able to execute the
> source from his post from 2017-Sep-06 without errors.
>
> So I was feeling to lazy to bother this time.
>

[Ned Batchelder]

On 9/20/17 8:22 PM, Bill wrote:
> Apparenty an \xa0 byte corresponds to a "non-breaking space". What
> sort of white space characters are allowed in a json file ( tabs and
> newlines?)?  Just curious.

These things can be looked up.  From RFC 7159
(https://tools.ietf.org/html/rfc7159):

Insignificant whitespace is allowed before or after any of the six
structural characters.

ws = *(
%x20 / ; Space
%x09 / ; Horizontal tab
%x0A / ; Line feed or New line
%x0D ) ; Carriage return

--Ned.

[Christopher Reimer]

> On Sep 20, 2017, at 5:03 PM, Stefan Ram <r...@zedat.fu-berlin.de> wrote:
>
> Dennis Lee Bieber <wlf...@ix.netcom.com> writes:
>> After removing all the \xa0 bytes
>> and trying to decode it I get...
>

> I did the same here, before I read your post.
> I got the same results, but did not post them.
>
> Someone has posted programs with \xA0 (NBSP IIRC)
> at the start of lines of the soure here before, in:
>
> From: Christopher Reimer <christoph...@yahoo.com>
> Newsgroups: comp.lang.python
> Subject: Setting property for current class from property in an different class...
> Date: Wed, 6 Sep 2017 19:24:40 -0700
> Message-ID: <mailman.144.1504751...@python.org>
>
> , and when I removed them, I was able to execute the
> source from his post from 2017-Sep-06 without errors.
>
> So I was feeling to lazy to bother this time.
>

> --
> https://mail.python.org/mailman/listinfo/python-list

I had a NBSP in my code?! News to me...

Chris R.

[Bill]

Ned Batchelder wrote:
>
> On 9/20/17 8:22 PM, Bill wrote:
>> Apparenty an \xa0 byte corresponds to a "non-breaking space". What
>> sort of white space characters are allowed in a json file ( tabs and
>> newlines?)? Just curious.
>
> These things can be looked up. From RFC 7159
> (https://tools.ietf.org/html/rfc7159):
>

Thank you. So what is most likely the root cause of the original
poster's problem (assuming he typed out the text with a text editor)?

[Thomas Jollans]

It looks to me like the root cause of the problem was that they copied
the code from a web page, and the web page contained invalid JSON.


--
Thomas Jollans

[Ned Batchelder]

I can only assume that the actual data being read is different than the
data they put into the message here.

--Ned.

[john polo]

On 9/20/2017 5:56 PM, John Gordon wrote:
> In <mailman.101.1505945...@python.org> john polo <jp...@mail.usf.edu> writes:
>
>> JSONDecodeError: Expecting ':' delimiter: line 5 column 50 (char 161)
>> ?json.loads says that the method is for deserializing "s", with "s"
>> being a string, bytes, or bytearray.
>> In [24]: type(text)
>> Out[24]: str
>> So "text" seems to be a string. Why does json.loads return an error?
> Because, although text is the right type, it does not contain a
> valid json string.

Thank you for the reply, John. I am not familiar with how to check
validity for json. I typed the example into notepad++ and saved as .json
and didn't take any further steps after that. I'll look into that.


John

[john polo]

On 9/20/2017 5:58 PM, Bill wrote:
> Interesting problem, John.
>
> I have probably even less experience with json than you do, so I'm
> taking this as an opportunity to learn with you.
>
> Suggestions:
>
> 1. Try your example with Python 2 rather than Python 3.

Bill,
Thanks for the reply. I wasn't sure how to get Python 2 through the cmd
or IPython, so I went through ArcGIS, but it's mostly the same result:

>>> file = open('books.json','r')
>>> text = file.read()
>>> text = json.loads(text)

Runtime error
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'json' is not defined
>>> import json   #whoops, forget to do that first in this go 'round.
>>> text = json.loads(text)
Runtime error
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\ArcGIS10.4\Lib\json\__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "C:\Python27\ArcGIS10.4\Lib\json\decoder.py", line 366, in decode

    obj, end = self.raw_decode(s, idx=_w(s, 0).end())

  File "C:\Python27\ArcGIS10.4\Lib\json\decoder.py", line 382, in
raw_decode

    obj, end = self.scan_once(s, idx)

ValueError: Expecting : delimiter: line 5 column 50 (char 161)

>>>

> 2. Take your file and make it into a string literal in your program,
> and try calling json.loads with that as an argument.

I am not sure I follow what you meant, but this was what I did:
In [26]: file2 = open('books.txt')

In [27]: text2 = file2.read()

In [28]: text2 = json.loads(text2)
---------------------------------------------------------------------------

JSONDecodeError                           Traceback (most recent call last)

<ipython-input-28-78a8123d71bd> in <module>()
----> 1 text2 = json.loads(text2)

c:\users\jpolo\appdata\local\programs\python\python36\lib\json\__init__.py

in lo
ads(s, encoding, cls, object_hook, parse_float, parse_int,
parse_constant, objec
t_pairs_hook, **kw)
    352             parse_int is None and parse_float is None and
    353             parse_constant is None and object_pairs_hook is
None and not
 kw):
--> 354         return _default_decoder.decode(s)
    355     if cls is None:
    356         cls = JSONDecoder

c:\users\jpolo\appdata\local\programs\python\python36\lib\json\decoder.py

in dec
ode(self, s, _w)
    337
    338         """
--> 339         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    340         end = _w(s, end).end()
    341         if end != len(s):

c:\users\jpolo\appdata\local\programs\python\python36\lib\json\decoder.py
in raw

_decode(self, s, idx)
    353         """
    354         try:
--> 355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
    357             raise JSONDecodeError("Expecting value", s,
err.value) from
None

JSONDecodeError: Expecting ':' delimiter: line 5 column 50 (char 161)

best,
John

[john polo]

On 9/20/2017 6:40 PM, Dennis Lee Bieber wrote:
> On Wed, 20 Sep 2017 17:13:41 -0500, john polo <jp...@mail.usf.edu>
> declaimed the following:

>
>
>> and the example code for reading the file is:
>>

>>>>> file = open('books.json','r')

> What encoding is the file? I did a cut&paste from your post into a
> file, and the contents (Python 2.7) failed with
Dennis,
I typed it into notepad++ and saved as .json. On the bottom of the
notepad++ window it says:
Windows (CR LF) UTF-8

I think that answers your question?

Thanks,
John

--
"Ask a man to be quiet,
and he'll be silent for a moment.
Feed a man to a red dragon
and he'll be silent for a lifetime."
-Anne Isabella Thackeray Ritchie

[john polo]

On 9/21/2017 4:24 AM, Thomas Jollans wrote:
>
> It looks to me like the root cause of the problem was that they copied
> the code from a web page, and the web page contained invalid JSON.

Thank you, Thomas.


John

[Ned Batchelder]

On 9/21/17 12:18 PM, john polo wrote:
> Bill,
> Thanks for the reply. I wasn't sure how to get Python 2 through the
> cmd or IPython, so I went through ArcGIS, but it's mostly the same
> result:
>

> >>> file = open('books.json','r')
> >>> text = file.read()
> >>> text = json.loads(text)

After the file.read() line, put this:  print(repr(text))  or
print(ascii(text)) if you are on Python 3.  Show us that output. That
will make clear what is wrong with the data.

--Ned.

[john polo]

On 9/21/2017 10:11 AM, Ned Batchelder wrote:
>
> I can only assume that the actual data being read is different than
> the data they put into the message here.
>
> --Ned.
>

There was a typo in the file that I had made and saved; an extra comma
before one of the ":". Apologies to the list for not catching that sooner.


John