possible urllib.urlencode() problem with hookbox REST API payload parameter

[Dave B.]

Hi,

I'm currently working with hookbox 0.3.4dev, cherrypy 3.2.0, and
Python 2.7.1. I've encountered a problem when trying to publish from
the cherrypy web server to hookbox's REST API, and I thought I'd share
what I found in case anyone else hits it.

For this version of hookbox, the REST API wants a URL with a "payload"
list/array value encoded in the URL query string - to eventually be
parsed as JSON by hookbox. However, when I tried to do the following:

url = "http://localhost:8001/web/publish"
urlArgs = { "security_token": "sectoken",
"channel_name": "chname",
"payload": [] }
payload = "message"
urlArgs[ "payload" ].append( payload )

data = urllib.urlencode( urlArgs )
req = urllib2.Request( url, data )
resp = urllib2.urlopen( req )

hookbox complained that the payload data wasn't valid JSON.
json.dumps( urlArgs[ "payload"] ) worked and yielded the following
valid JSON string:

["message"]

However, after the call to urllib.urlencode(), the encoded "data"
string was:

channel_name=chname&payload=%5B%27message
%27%5D&security_token=sectoken

Now, %27 is a single quote, and I think it should be %22 double quote,
since a single quote is not valid JSON, right?

I'm not sure if I'm doing something wrong, or if I've found a genuine
problem. Should I not use urllib.urlencode()? Of course as a
workaround I can always change the %27 to %22 manually before creating
the Request.

Just wanted to ask and/or let people know in case they come across
this issue.

Still thinking hookbox is a great thing, and hoping it lives on. I
really don't want to have to switch to (or write!) something else...

Thanks,
Dave B.

[Dave B.]

A little more information/strangeness...

If the string in the payload contains double quotes, they do get
encoded as %22 - it's just the initial and trailing double quotes that
get encoded as %27. But then the double quotes in the string need to
be escaped (add a leading backslash = %5C). So the encoding currently
looks something like:

data = urllib.urlencode( urlArgs ).replace( "%22", "%5C
%22" ).replace( "%5B%27", "%5B%22" ).replace( "%27%5D", "%22%5D" )

Any suggestions welcome...

Dave B.

[Salman Haq]

Dave,

The payload value MUST be a json-formattied string. Not a python array
as in your example.

Try this snippet:

payload = [] # for your app, payload must be a list.
payload.append("message") # and you will append various elements to it
based on some logic.
urlArgs["payload"] = json.dumps(payload) # and finally dump it as a json
string.

# now encode urlArgs and http request...

Best,
Salman

[Dave B.]

Thanks very much Salman - that was the piece I was missing! (I guess
it shows that I just started with Python last week!) I had actually
been trying to follow the examples in the hookbox 0.3.4dev
distribution, and I think I saw your name there somewhere, so I
appreciate the extra guidance...

Thanks again,
Dave