Packet Formats (and HTTP/Socket proxy)
mcollins
favor of both socket-based and HTTP based protocols, though not much
objection to JSON as a data format. Simon Kaegi brought up the
possibility of using two transports, and I don't see why this
shouldn't be possible. I think we will need to put some stakes in the
ground though as to how the data will be encapsulated in both
transports, in order to quickly move on to discussions of the more
interesting aspects of Web debugging.
A somewhat related thought I had is that in remote debugging scenarios
some kind of a proxy is often used. If there were a common, open-
source implementation of a proxy that supported both the HTTP and
socket transports, it would free up individual tool developers to
choose to implement whichever transport was best for them, while still
allowing the possibility of connecting with other clients. It would
also remove the burden of forcing every tool which wishes to support a
common protocol from having to embed a web server.
So I would propose going ahead with both transports, and suggest the
following packet formats:
Socket: ascii message-length, null-byte, ascii message, null-byte
HTTP: POST request, message as x-www-form-urlencoded in request body.
I am interested in what others might think is sufficient particularly
for the socket format. There is the possiblity of sending additional
information with requests in the form of HTTP headers (e.g. character-
set), that might be difficult to add to another format after the fact.
Mike
Patrick Mueller
On Jul 1, 2009, at 3:57 PM, mcollins wrote:
> In the discussion of protocol layers, there were arguments made in
> favor of both socket-based and HTTP based protocols, though not much
> objection to JSON as a data format.
So JSON is the data format. +1
> So I would propose going ahead with both transports, and suggest the
> following packet formats:
>
> Socket: ascii message-length, null-byte, ascii message, null-byte
Why not make it more textual; the payload for content-encoding:
chunked seems kind of appropriate, which is basically uses whitespace
where you specify null bytes.
http://en.wikipedia.org/wiki/Chunked_transfer_encoding
> HTTP: POST request, message as x-www-form-urlencoded in request body.
Why x-www-form-urlencoded? Presumably JSON objects serialized to a
string would be sent to the server, and received back, in which case
use application/json for both.
Patrick Mueller - http://muellerware.org/
mcollins
>
> Why x-www-form-urlencoded? Presumably JSON objects serialized to a
> string would be sent to the server, and received back, in which case
> use application/json for both.
>
what turned me off from it now. Possibly the vague hunch that there
would be more web servers/frameworks that already know how to parse
out url-encoded, vs. application/json, although this is probably not
very relevant. It certainly would seem more modern to be using
application/json.
mc
Patrick Mueller
I'm not quite understanding what using form-encoded would mean. Would
you send a JSON payload as the value of a form key? So a request body
for a POST would look something like this:
json={"x":2}
which would actually need to get sent as this, to handle the form
encoding:
json=%7B%22x%22%3A2%7D
Mike Collins
I'm not quite understanding what using form-encoded would mean. Would
you send a JSON payload as the value of a form key? So a request body
for a POST would look something like this:
json={"x":2}
which would actually need to get sent as this, to handle the form
encoding:
json=%7B%22x%22%3A2%7D
mc
Patrick Mueller
My simple example could in theory be sent as
x=2
which you might imagine would get treated as:
{"x":2}
which has some nice properties; everyone can already easily deal with
form encoding, etc.
Problems are, you can't indicate a type, so what you'd really likely
have to translate that form body to is:
{"x": "2"}
And dealing with nested objects sends you back to the pits of hell.
So while it should never be used as a general encoding mechanism,
there might be some rational for actually allowing it for highly
constrained objects - "single level" objects whose properties are all
string values, specifically. Just seems hardly worth the hassle.
mcollins
On Jul 1, 9:22 pm, Patrick Mueller <pmue...@yahoo.com> wrote:
> On Jul 1, 2009, at 3:57 PM, mcollins wrote:
>
> > Socket: ascii message-length, null-byte, ascii message, null-byte
>
> Why not make it more textual; the payload for content-encoding:
> chunked seems kind of appropriate, which is basically uses whitespace
> where you specify null bytes.
>
> http://en.wikipedia.org/wiki/Chunked_transfer_encoding
>
use a chunked-encoding scheme over TCP socket, seems to be working
pretty well, I see no reason why it couldn't be used this way. But, I
am wondering if another reason for going this route is that a proxy
HTTP server that is also listening for raw socket connections, could
take the chunks from the raw stream and insert them into directly into
the body of an HTTP request? I don't know much about the nitty-gritty
of HTTP but it seems feasible to me on the surface.
mc
Patrick Mueller
I think you're suggesting that you could extend an existing web server
with something that read HTTP transfer-encoded chunks from a non-HTTP
socket and fed them to someone else as HTTP requests/responses. At
that point, your biggest problem is that you have to write an
extension to a web server; dealing with a different "chunk" ending
isn't an issue anymore, priority-wise. :-)
mcollins
as much as I was thinking of embedding an existing web server (e.g.
httpd.js in a firefox extension), although I like that idea as well,
maybe I'm running apache with some kind of mod_webdebug module, which
I connect to a socket with my IDE, and it proxies JSON debug messages
to the web page I am trying to debug (avoiding any cross-site XHR
issues)?
I was thinking that if the messages were already in chunked format on
the wire, then it would simplify/streamline writing such an
extension. The extension author would just grab messages off the wire
and place them directly into an HTTP response body, without having to
parse/reformat each message. But I don't really know what's involved
in writing something like an apache module, so maybe you are right
that any choice of wire format isn't going to provide a significant
advantage in that regard.
mc