Dear all
It is a period of changes for Se^H^HCommonJS and Narwhal (engines, etc). Perhaps it is the last chance to (re)-propose a change in JSGI. Please bear with me. I would like to propose the following change to the response object:
From
[200, {}, ["hello"]]
to
{ code: 200, headers: {}, body: [] }
Rack's array responses leverage Ruby's syntax features:
return 200, {}, ["hello"]
code, headers, body = app(env)
Arrays do not play as nicely with JavaScript. JavaScript's syntax is 'optimized' for Object literals so it makes sense to use them.
The code is cleaner and more elegant, consider:
response[0]
response[1]["Content-Type"]
vs
response.code
response.headers["Content-Type"]
An object literal *may* allow for some default values, ie:
return { body: ["hello"] } // status=200, headers={} by default
return { code: 404 } // headers={}, content=[] by default
return { code: 404, body: ["not found"] } // headers={} by default
Moreover, one can easily add extra fields. A useful example (in Nitro at least):
return { code: 200,
data: {
articles: Article.all().fetch(),
user: User.currentUser()
} // header={}, content=[] <- a template will be rendered downstream.
I can't help but notice the symmetry:
var app = function(env) {
return {..}
}
The function app receives an object literal (env) and returns an object literal ({..})
Finally, there is no need to call .finish() when using the Response helper, just return the response (it's an object already!)
It even allows code like this:
var response = new Response(app(env))
instead of
var ret = app(env);
var response = new Response(ret[0], ret[1], ret[2]);
One benefit of Arrays is they allow for easy porting of existing Rack code. But really, the conversion to a hash is not that difficult.
Please don't reject this proposal immediately 'because we have a standard'. We still have time for changes (if they are useful of course). IMHO the standardization process from Jack (a Rack port) to JSGI (A CommonJS standard) was too quick and not much discussed.
As of 1.7 javascript has a form of destructuring assignment [1]. While this isn't perfect, it would be suitable:
var [status, headers, body] = app(env);
But should JSGI depend on js 1.7 features? What is the absolute minimum CommonJS js version? I remember this was discussed before -- was it ever spec'd?
An object literal *may* allow for some default values, ie:
return { body: ["hello"] } // status=200, headers={} by default
return { code: 404 } // headers={}, content=[] by default
return { code: 404, body: ["not found"] } // headers={} by default
I would propose status rather than code as it's more common and less overloaded, but otherwise, I love it. I tend to prefer explicit over implicit, but I believe default values would be a godsend for readability.
But the symmetry doesn't stop there. You can tack extra values on the response just like you can on the request env. But are these values better suited to the response? Isn't this what the env is for in the first place?
Finally, there is no need to call .finish() when using the Response helper, just return the response (it's an object already!)
It even allows code like this:
var response = new Response(app(env))
instead of
var ret = app(env);
var response = new Response(ret[0], ret[1], ret[2]);
That's spectacular.
One benefit of Arrays is they allow for easy porting of existing Rack code. But really, the conversion to a hash is not that difficult.
Don't forget about wsgi code -- there's a metric shitton of it and it would port nicely to the current jsgi. I really like this idea but I'd be hesitant to +1 it until more folks weigh in. Are the symmetries of jsgi to rack, and rack to wsgi *only* due to ease of porting? Or is there more we're missing?
[1] https://developer.mozilla.org/en/New_in_JavaScript_1.7#Destructuring_assignmentI doubt there's that much JSGI production code floating around, so I agree now's the time to nail things down. In this thread [2] I pointed out some other problems (mostly culled from wsgi's mistakes and planned fixes [3]).
[2] http://groups.google.com/group/commonjs/browse_thread/thread/dbb8c27664a6c778/67bd904cbce005e7?lnk=gst&q=wsgi#67bd904cbce005e7
[3] http://www.wsgi.org/wsgi/WSGI_2.0
+1 for requests being an object (with status, headers, body) instead of
an array. I'd prefer status over code.
+1 from me too. Porting from Rack/WSGI will not really be any harder just because it's an object rather than an array :)
[0] -> .status[1] -> .headers[2] -> .bodyCan be acomplished with the text editor's find and replace function.
I have a var [status, headers, body] = res; in virtually all of my
Helma middleware, and I think it's perfect.
This means writing code that uses this feature for engines that
support it (Spidermonkey, Rhino) is future proof, and if you're
targeting JS 1.5 or 1.6 you can just fall back to something sightly
less verbose. And remember, Harmony is meant to be a small incremental
change to ES syntax, not the huge shift ES4 was meant to be. It
shouldn't take forever to materialize.
>>> An object literal *may* allow for some default values, ie:That's really a reason against this change. You can have default
values in your frameworks, but not on the low level JSGI protocol.
Again, this is not a good idea IMO. Once you add that extra level
beyond HTTP headers and status codes, middlewares are going to depend
on each other in all kind of weird ways.
>>> Finally, there is no need to call .finish() when using the ResponseThe Response object isn't part of JSGI, so it's really a detail of
>>> helper, just return the response (it's an object already!)
>>> It even allows code like this:
>>>
>>> var response = new Response(app(env))
>>>
>>> instead of
>>>
>>> var ret = app(env);
>>> var response = new Response(ret[0], ret[1], ret[2]);
>> That's spectacular.
your particular implementation. If a framework wanted to use some
different resoponse object - something JSGI should surely support -
it would still have to do some translation.
All in all, it's my belief that the array representation is the better
fit for representing an HTTP response in JavaScript.
>You're wrong, destructuring assignment is one of the planned Harmony
> javascript 1.7 is not a standard and I don't see destructuring assignment
> become a standard feature any time soon.
features agreed upon by the ES technical committee:
http://wiki.ecmascript.org/doku.php?id=harmony:proposals
This means writing code that uses this feature for engines that
support it (Spidermonkey, Rhino) is future proof, and if you're
targeting JS 1.5 or 1.6 you can just fall back to something sightly
less verbose.
I said it's a better fit for representing HTTP responses inJavaScript. And I actually did say why I think so, which is most
importantly because the array as a representation is more reduced and
concise and less inviting to extend the response (and thus introduce a
meta layer beyond HTTP).
Hannes
Still, I don't see how hanging additional off the response object could impact pure middleware (which should only peek at the three HTTP properties). Plus, the same argument could be made against extending the env object -- and certainly nobody would suggest freezing that.
{ status: 200, headers: {}, body: ["hello"] }which looks more JavaScriptish to you?
An argument could be made that it's a little verbose, but there's no doubt it's intent is clearer and self-describing.
ES5 should be adopted quickly as well -- after ratification -- although I would suggest thinking long and hard about making use of ES5 features in core libraries.
You're wrong, destructuring assignment is one of the planned Harmony
features agreed upon by the ES technical committee:
http://wiki.ecmascript.org/doku.php?id=harmony:proposals
This means writing code that uses this feature for engines that
support it (Spidermonkey, Rhino) is future proof, and if you're
targeting JS 1.5 or 1.6 you can just fall back to something sightly
less verbose. And remember, Harmony is meant to be a small incremental
change to ES syntax, not the huge shift ES4 was meant to be. It
shouldn't take forever to materialize.
On Thu, Aug 27, 2009 at 6:52 AM, Alexandre.Morgaut<mor...@hotmail.com> wrote:
>
> I expect some changes but not exactly like that.
> I'm happy you made the point on some of these "problems" (if we can
> talk about problems)
>
> The main point for me is default values (why specify 200 for each
> correct responses).
> We're implementing a some Server Side JavaScript on a Web Server and
> we're looking on how everybody does it (narwhal, jaxer, helmang, ...)
> We choose to support the JSGI API becaue of its simplicity and add a
> more complete API for more specific needs
>
> We very liked the simplicity of an hello world with jack :
>
> return [200, {Content-Type: 'text/html'}, ['hello world']];
>
> But still being compatible we thank about default values and
> unordered array
>
> As we can see, the response is sent as an array with 3 items of
> different types :
> - a number for the status code
> - an object for the HTTP headers
> - an array for the body(ies ?)
>
> If a request event handler return an array :
> - The position of the 3 elements of this array is not
> relevant.
> - Only the type of these elements must checked to know to
> which part of the response it belongs :
> - number -> status code
> - object -> header
> - string or array -> body
> - the default status code is "200" (OK)
> - if no body is specified, the default status code becomes
> "204" (No Content)
> - if no header is specified "Content-Type" and "Content-
> Length" fields are set from the nature of the body :
> - "text/plain" for strings or date objects and
> scalars embeded in the array (Number, Boolean, Null, Undefined)
> - "application/json" for other object (serialized as
> JSON)
> - "application/octet-stream" for images, blobs, or
> binary files if a more specific mediatype can't be recognized (if
> binary data supported)
> - xml and html mediatypes for DOM elements depending
> of their nature (text/html or text/xml may be acceptable in most
> cases) (if DOM supported)
> - if the body array element has several elements, this means
> that the response must be sent has multipart
> - if no body is sent and the status code match to an error
> (40x and 50x), a default content should be sent in the most supported
> format of the client (html, text, or json, and if you are really
> motivated, an audio version would be a very good point for
> accessibility and specific devices)
>
> At this point the user is quite helped but a response specified
> without status code and headers looks horrible
>
> return [['hellow world']];
> or
> return [[response]];
>
> So, any returned value which is not an array should be considered as a
> content :
>
> return 'hellow world';
>
> return 42;
>
> Access to "env" :
>
> on this page https://wiki.mozilla.org/ServerJS/System we can see a
> proposal to access environment variables
> I prefer to access it for system.env and not from an argument
> you still can start your handler by :
> var env = system.env
It is important that the application receive a mutable copy of the
environment variables, not use the environment directly. Request
routing frameworks must be able to give the application the impression
that every application is the main script for whatever part of the
(PATH_INFO) that remains to be dealt with.
> So what looks like such API in a use case :
>
> function () {
> var out,
> env = system.env;
> (...)
> out.push(201);
> out.push('employee created');
> return out;
> }
>
> or
>
> function () {
> // trace client request
> // system.env is an object so it is sent JSON encoded with JSON
> mediatype
> return system.env;
> }
>
>
> Optionally, a false returned value could have a 400 default status
> code
I have only minor trepidation about these use case optimizations. I
think we'll find that, for this kind of tuning, a "simplification" on
one side is a "complication" on the other, and since there will be a
lot of middleware standing on both sides of this equation, everyone is
likely to suffer. I do not consider it onerous to return an ordered
triple, although I am also very likely to extend that ordered triple
as George mentions, to provide additional optional metadata to
middleware as he does for template contexts. For that reason alone, I
think that an object would make a better conduit for a
*standards-managed* name space.
{
status: 200,
body: ["Hello, World"],
headers: {"Content-type": "text/plain"},
xNitroContext: {"title": "Hello", "author": "Foo Bar",
"breadcrumbs": …, "navigation": …}
}
Kris Kowal
This is the reason why Python uses "next" as its iterator primitive
method, not "forEach". "next" is a function that returns the next
value of an iteration or throws a StopIteration exception. The
"forEach" construct in turn is built on top of "next" and not meant to
be overridden:
forEach = function (block, context) {
while (true) {
try {
block.call(context, this.next());
} catch (exception) {
if (exception instanceof StopIteration) {
break;
} else {
throw exception;
}
}
}
};
Which could be extended at a price to do SkipIteration, for both break
and continue analogs.
Kris Kowal
-1
As a middleware developer I can see how it's not ideal, but since it's
much more common to write applications than middleware I think it's
better to optimize for that case. Typing [200, {}, []] is about 1/3 as
many characters as { code : 200, headers : {}, body [] } and less
error prone.
-tom
On Aug 27, 2009, at 12:59 AM, George Moschovitis wrote:
> Dear all
>
> It is a period of changes for Se^H^HCommonJS and Narwhal (engines,
> etc). Perhaps it is the last chance to (re)-propose a change in
> JSGI. Please bear with me. I would like to propose the following
> change to the response object:
>
> From
>
> [200, {}, ["hello"]]
>
> to
>
> { code: 200, headers: {}, body: [] }
>
> Rack's array responses leverage Ruby's syntax features:
>
> return 200, {}, ["hello"]
> code, headers, body = app(env)
>
> Arrays do not play as nicely with JavaScript. JavaScript's syntax is
> 'optimized' for Object literals so it makes sense to use them.
>
> The code is cleaner and more elegant, consider:
>
> response[0]
> response[1]["Content-Type"]
>
> vs
>
> response.code
> response.headers["Content-Type"]
>
> An object literal *may* allow for some default values, ie:
>
> return { body: ["hello"] } // status=200, headers={} by default
> return { code: 404 } // headers={}, content=[] by default
> return { code: 404, body: ["not found"] } // headers={} by default
>
> Moreover, one can easily add extra fields. A useful example (in
> Nitro at least):
>
> return { code: 200,
> data: {
> articles: Article.all().fetch(),
> user: User.currentUser()
> } // header={}, content=[] <- a template will be rendered
> downstream.
>
> I can't help but notice the symmetry:
>
> var app = function(env) {
> return {..}
> }
>
> The function app receives an object literal (env) and returns an
> object literal ({..})
>
> Finally, there is no need to call .finish() when using the Response
> helper, just return the response (it's an object already!)
> It even allows code like this:
>
> var response = new Response(app(env))
>
> instead of
>
> var ret = app(env);
> var response = new Response(ret[0], ret[1], ret[2]);
>
> One benefit of Arrays is they allow for easy porting of existing
> Rack code. But really, the conversion to a hash is not that difficult.
>
> Please don't reject this proposal immediately 'because we have a
> standard'. We still have time for changes (if they are useful of
> course). IMHO the standardization process from Jack (a Rack port) to
> JSGI (A CommonJS standard) was too quick and not much discussed.
>
> I would love to hear opinions from many people on this proposal.
>
>
>
> kind regards,
> George.
>
>
> --
> blog.gmosx.com
>
> >
Typing [200, {}, []] is about 1/3 as
many characters as { code : 200, headers : {}, body: [] } and less
error prone.
> > better to optimize for that case. Typing [200, {}, []] is about
> 1/3 as
> > many characters as { code : 200, headers : {}, body [] } and less
> > error prone.
>
> You know, we could always accept and codify both.
Having multiple response representations is a bad idea for the same
reason we specify that the body must respond to "forEach" and yield
objects that respond to "toByteString"... you don't want to have to
test for different objects types in every piece of middleware and
handler. A single API is really really key here.
> I honestly don't care which looks more like JavaScriptish. I care which is the more accurate and reliable representation of the data.--
I'm with you there.
So when are we going to discuss how to resolve CONTENT-TYPE vs. cOntent-Type: ?
My proposal is
- last one wins
- "last one" defined by iteration order
This has the side effect of relying on iteration order as implemented in browser engines, although it remains uncodified even in ES3 I think. I think Rhino finally gets this right as of about six months ago. (Right, in this case, being awfully subjective).
Having multiple response representations is a bad idea for the same
reason we specify that the body must respond to "forEach" and yield
objects that respond to "toByteString"... you don't want to have to
test for different objects types in every piece of middleware and
handler. A single API is really really key here.
I'm ready to concede to the object fanboys ;)
and presumably it's not a big undertaking for George's Nitro since he's the one spearheading this movement.
>
> On Aug 27, 2009, at 4:47 PM, Daniel Friesen wrote:
>
>>
>> Cept this is access, not setting or comparing:
>>
>> req["Content-Type".toLowerCase()] // Does NOT do what we are trying
>> to do
>>
>> We are trying to deal with the issue with headers like:
>> { 'Content-Type: "text/html", 'content-type': "text/plain",
>> 'CONTENT-TYPE', "image/png" }
>>
>> What do you grab, what can you set?
>
>
> AFAICT, the options are:
>
> 0. Use a plain Object for the headers object, then use helper
> functions in middleware to handle different cases. Current solution.
>
> In Jack middleware we just use "case-preserving" hash functions
> (called "HashP").
>
>
> 1. Require all headers be a certain case/format. I don't think we can
> do this. I believe HTTP says headers are case-insensitive, but it's
> good practice to use the "standard" format, which isn't always "Abc-
> Def", for example ETag.
>
>
> 2. Require something more complicated be the headers object. The API
> would have to be something like get(), set(), add(), has(). But then
> you have to do:
>
> return { status : 200, headers : SpecialHeaderObject({ "Content-
> Type" : "foo" }), body : ["bar"] }
>
>
> -tom
1) has my vote. We specify the case for the 'standard' HTTP headers
and say if you want it to work properly in middle ware, use this case:
(I stole that list from somewhere or other, I forget where now)
I'm ready to concede to the object fanboys ;)But we should definitely NOT support both in the spec, as I explained in another message:Having multiple response representations is a bad idea for the same
reason we specify that the body must respond to "forEach" and yield
objects that respond to "toByteString"... you don't want to have to
test for different objects types in every piece of middleware and
handler. A single API is really really key here.
Something I still don't understand is why sending the body in an
array ?
All the more if this array always have a single element and if this
body isn't multipart
What did I miss ?
> AFAICT, the options are:1) has my vote. We specify the case for the 'standard' HTTP headers
>
> 0. Use a plain Object for the headers object, then use helper
> functions in middleware to handle different cases. Current solution.
>
> In Jack middleware we just use "case-preserving" hash functions
> (called "HashP").
>
>
> 1. Require all headers be a certain case/format. I don't think we can
> do this. I believe HTTP says headers are case-insensitive, but it's
> good practice to use the "standard" format, which isn't always "Abc-
> Def", for example ETag.
>
>
> 2. Require something more complicated be the headers object. The API
> would have to be something like get(), set(), add(), has(). But then
> you have to do:
>
> return { status : 200, headers : SpecialHeaderObject({ "Content-
> Type" : "foo" }), body : ["bar"] }
>
>
> -tom
and say if you want it to work properly in middle ware, use this case:
http://github.com/ruediger/flusspferd/blob/e0948a3719bfb11b9a60090a4faa2571c803a846/src/js/http/headers.js#L36
(I stole that list from somewhere or other, I forget where now)
However, as Tom said, perhaps we should optimize for the application developer and not the middleware developer. Personally, I think your suggestion to use arrays only for multipart/chunked responses is reasonable. But I am not sure about the complications.
While I prefer the object return value to the array one, I do think that JSGI is generally *not* for the app developer. App developers will generally use a higher level abstraction, I believe...On Fri, Aug 28, 2009 at 5:49 AM, George Moschovitis <george.mo...@gmail.com> wrote:
However, as Tom said, perhaps we should optimize for the application developer and not the middleware developer. Personally, I think your suggestion to use arrays only for multipart/chunked responses is reasonable. But I am not sure about the complications.
multiple representations is bad... I think the body should stay with .forEach
I completely disagree with designing anything with the goal of forcing a coding principal (good or bad) upon the developer. Feel free to scream at the top of your lungs at stupid programming but don't design a API with controlling the developer in mind.
The middleware should provide methods for building the body. The body could be an Array that is joined together when the middleware hands the message over to the server but I don't like the idea of a body being an array. Arrays are for ordering like data not for chucking strings together or forcing paradigms on a developer.
However, as Tom said, perhaps we should optimize for the application developer and not the middleware developer. Personally, I think your suggestion to use arrays only for multipart/chunked responses is reasonable. But I am not sure about the complications.
Regarding: "I worry about the app developer compositing the body via the use of string concatenation"I completely disagree with designing anything with the goal of forcing a coding principal (good or bad) upon the developer. Feel free to scream at the top of your lungs at stupid programming but don't design a API with controlling the developer in mind.
The middleware should provide methods for building the body. The body could be an Array that is joined together when the middleware hands the message over to the server but I don't like the idea of a body being an array. Arrays are for ordering like data not for chucking strings together or forcing paradigms on a developer.I believe:A response should be an object, its body should be a string, its header should be an object and its return code should be in integer. The object's properties case should be standardized. I vote for camelCase seeing that it is pretty standard in JavaScript, but I am fine with underscores or even running the words together.
I don't understand any benefits of using an Array.Each packet needs its own header, etc... If you need to cache the body for a bit to wait before sending the packet, that should be handled in the middleware.The body of an html message is not always logically divisible and even when it is logically divisible, it is not logically divisible in the transport layer. As far as I can tell, the body should be a string... If you want to chuck data into a single html message, then chuck it at in the middleware where the logic behind building the response lays, then create the response object sending the completed body as a string.
It doesn't make since to me to have the body of a response respond to .forEach(). A body is a single thing... it should not be enumerable unless your suggesting that a single html message should have multiple bodies.
> Are you thinking:
> 1) That a packet is <= the MTU (~1500 bytes) and the body is a part of the
> packet? And a response is a packet.
> 2) Or are you thinking that a response is a whole document or data file and
> it will be broken down into response packets?
> I cannot think of any reason to iterate through a packets body. But if your
> thinking that a response is a web servers reply then it needs to be iterated
> through to break into packets. I guess I am not sure I understand what your
> referring to as response, body, & header
Yeah. I'll attempt to explain.
We're not talking about packets. You're right that we are not
responsible for breaking a layer 7 HTTP response into layer 2 TCP
packets or even layer 1 Ethernet packets. It should however be
possible for us to break a layer 7 HTTP response into layer 7 HTTP
*chunks* of arbitrary length, any length we choose. This permits us
to construct a sort of bucket brigade of discrete pieces down the OSI
stack. For example, instead of reading a 4GB file into memory at one
time (usually impossible) and returning it as a single object in an
HTTP response, we can, at layer 7, break the task down into smaller,
manageable chunks. These chunks get passed down to lower layers,
getting sliced up and buffered in different sizes (the MTU applies
only at layer 2).
Usually it is sufficient to send a single chunk from layer 7.
However, we need to be *able* to send multiple chunks, in order.
That is to say, the concept of streaming exists at *every* layer of
the OSI stack.
This is probably not the best venue for a discussion of how the loose
abstraction of OSI works. Perhaps you could join us on IRC, #commonjs
on freenode, or address those interested with an off list discussion.
Kris Kowal
I believe:A response should be an object, its body should be a string,
It appears that I have been confusing things tcp and http. I am sorry.
http_accept="a=foo,b=bar"
http_content_type="text/html"
There is no "the Library", and no standards for one.
We have the JSGI protocol which is used for handlers to talk to
apps/middleware. Which is basically summed up as an app function, env,
and the returned response object literal.
It's also what Rack does, FWIW.
WSGI uses an array of tuples for response headers, and the "env"
object containing HTTP_ properties for request headers. I'm not sure
how they (or we) handle multiple *request* headers values.
On Aug 31, 2009, at 4:40 PM, Daniel Friesen wrote:
>
> It's not my suggestion, the "\n" is what is already in the jsgi spec.
> Have multiple headers by the same name? A multiline header. Done.