how to access json data value(s) returned back to django view from 'GET' request to an api

[Henry Versemann]

I have a django view (django v. 1.7 ; python v. 2.7.8) which currently and successfully sends a request to an api, and receives a good response back. I'm sending and receiving using the 'requests' library (v. 2.4.3). Now in addition to sending the raw response data back in an HttpResponse I would also like to strip out some of the data and create a summary report of it, to also send back in my HttpResponse.

I can see the good response data from the request in my command prompt window, and it contains a list of items inside of the object to which I've assigned the response data. So my question is how do I get at the list of items that has been returned back to me as well as all of the data (key/value pairs) contained in each list item?

I'm sure there must be some kind of conversion process that I have to run the raw response data through. I just haven't out what it is yet.

I've never tried to do this before, from within a django view. If I were trying to do this on the client side I would use something like this:

                      mydata = jQuery.parseJSON(data);

to make the data accessible, but never having done anything like this before  on the server side I haven't found anything yet that's been useful or has worked.

I would appreciate any help anyone can offer, while I continue to search for an answer.

Thanks.

[Larry Martell]

See: https://docs.python.org/3.3/library/json.html

e.g.:

import json
jsonIn = json.loads(request.POST['json_content'])

[Scot Hacker]

On Thursday, February 5, 2015 at 11:10:32 AM UTC-8, Henry Versemann wrote:

I have a django view (django v. 1.7 ; python v. 2.7.8) which currently and successfully sends a request to an api, and receives a good response back. I'm sending and receiving using the 'requests' library (v. 2.4.3). Now in addition to sending the raw response data back in an HttpResponse I would also like to strip out some of the data and create a summary report of it, to also send back in my HttpResponse.

What Larry said - you need to convert the javascript data format into a python data format. json.loads() will do that for you. Here's a working example: 

import json

def get_nyt_news():

    # NYT API data

    response = requests.get(

        'http://api.nytimes.com/svc/mostpopular/v2/mostshared/all-sections/7.json?api-key={apikey}'.format(apikey=settings.NYT_API_KEY)

        )

    json_data = json.loads(response.text)

    data = json_data['results'][0:4]

    # Rewrite data to conform internal links

    for d in data:

        d['get_absolute_url'] = d['url']

    return {'object_list': data}

[Henry Versemann]

Scot, Thanks for the reply. OK I've gotten down to the point to where I have my version of your json_data in your code below, but I'm not sure I understand exactly what your code is doing, after you have the json_data variable contents. In my case the first object of response data that I'm trying to use is a list of two items, with each item containing multiple keys and internal or sub objects within them.

Can you please explain your code after you have the json_data variable formatted further? I'm guessing you're doing some kind of further required conversion of the data, and if so I'm not sure I understand why those conversions are necessary and being done.

My current piece of response data that I'm trying to work with looks like this(when I print it in my command prompt window):

[
    {
        u'status': u'complete',
        u'parameters':
        {
            u'enrollment_term_id': u'',
            u'extra_text': u'Term: All Terms;'
        },
        u'file_url': u'https://stchas.test.instructure.com/accounts/1/files/7787/download',
        u'attachment':
        {
            u'display_name': u'grade_export_csv_10_Feb_2014_40_18363-0.csv',
            u'unlock_at': None,
            u'url': u'https://stchas.test.instructure.com/files/7787/download?download_frd=1&verifier=nEU8FRMgfMaCFUK0NoIXiGMaQj0M
xWWJ4BM7PrMU',
            u'created_at': u'2014-02-10T15:26:24Z',
            u'updated_at': u'2014-02-10T15:26:24Z',
            u'filename': u'1392045984_664__grade_export_csv_10_Feb_2014_40_18363-0.csv',
            u'lock_at': None,
            u'thumbnail_url': None,
            u'hidden_for_user': False,
            u'locked': False,
            u'hidden': False,
            u'locked_for_user': False,
            u'content-type': u'text/csv',
            u'id': 7787,
            u'size': 4744
        },
        u'report': u'grade_export_csv',
        u'progress': 100,
        u'id': 40
    },
    {
        u'status': u'complete',
        u'parameters':
        {
            u'enrollment_term_id': u'',
            u'extra_text': u'Term: All Terms;'
        },
        u'file_url': u'https://stchas.test.instructure.com/accounts/1/files/1033/download',
        u'attachment':
        {
            u'display_name': u'grade_export_csv_22_Oct_2013_2_27551-0.csv',
            u'unlock_at': None,
            u'url': u'https://stchas.test.instructure.com/files/1033/download?download_frd=1&verifier=uhw0oyfb0f3m4GtgJAVvwQkRlDfK0kCbg200SIu4',
            u'created_at': u'2013-10-22T19:15:45Z',
            u'updated_at': u'2013-10-22T19:15:45Z',
            u'filename': u'grade_export_csv_22_Oct_2013_2_27551-0.csv',
            u'lock_at': None,
            u'thumbnail_url': None,
            u'hidden_for_user':False,
            u'locked': False,
            u'hidden': False,
            u'locked_for_user': False,
            u'content-type': u'text/csv',
            u'id': 1033,
            u'size': 395
        },
        u'report': u'grade_export_csv',
        u'progress': 100,
        u'id': 2
    }
]

So far I've tried to access the objects inside the list like I normally would in python using indexing for the list items like this:

lstobjct = rspnslist[index]

but so far all I've gotten from this is an "exceptions.AttributeError".

Thanks again for the help.  

[Henry Versemann]

Thanks Larry.

[Scot Hacker]

On Friday, February 6, 2015 at 9:39:00 AM UTC-8, Henry Versemann wrote:

Scot, Thanks for the reply. OK I've gotten down to the point to where I have my version of your json_data in your code below, but I'm not sure I understand exactly what your code is doing, after you have the json_data variable contents. In my case the first object of response data that I'm trying to use is a list of two items, with each item containing multiple keys and internal or sub objects within them.

Once your data comes back from the API and has been converted to python data structures, it's up to you to parse and extract from it what you need - every situation is different. All you need to know is that you access list items by their index, and dictionary items by quoted key name. So if I assign the response object you show here to a variable "data", I can do the following:

print(data[0]['parameters'])

print(data[0]['parameters']['extra_text'])

print(data[1]['attachment']['display_name'])

and the result is:

{u'enrollment_term_id': u'', u'extra_text': u'Term: All Terms;'}

Term: All Terms;

grade_export_csv_22_Oct_2013_2_27551-0.csv

Hope this helps!

./s

 

[JSON CSV]

On Friday, February 6, 2015 at 5:10:32 AM UTC+10, Henry Versemann wrote:

I have a django view (django v. 1.7 ; python v. 2.7.8) which currently and successfully sends a request to an api, and receives a good response back. I'm sending and receiving using the 'requests' library (v. 2.4.3). Now in addition to sending the raw response data back in an HttpResponse I would also like to strip out some of the data and create a summary report of it, to also send back in my HttpResponse.

In case you want to do ad-hoc reports based on JSON data - you may find it useful to upload the JSON data to our website https://json-csv.com.  A spreadsheet is produced from the API which you can use in Excel to chart on / summarize etc.

[Henry Versemann]

Scot, Thanks for the response. I will try it first thing Monday morning. If I have any other questions I may post them then, but this should get most of it for me. Thanks again. Henry