Re: [narwhal] JSGI Response change proposal

34 views
Skip to first unread message

Dean Landolt

unread,
Aug 27, 2009, 7:55:17 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
I'm cc'ing commonjs as this is a JSGI spec issue...

On Thu, Aug 27, 2009 at 3:59 AM, George Moschovitis <george.mo...@gmail.com> wrote:
Dear all

It is a period of changes for Se^H^HCommonJS and Narwhal (engines, etc). Perhaps it is the last chance to (re)-propose a change in JSGI. Please bear with me. I would like to propose the following change to the response object:

From

[200, {}, ["hello"]]

to

{ code: 200, headers: {}, body: [] }

Rack's array responses leverage Ruby's syntax features:

return 200, {}, ["hello"]
code, headers, body = app(env)

As of 1.7 javascript has a form of destructuring assignment [1]. While this isn't perfect, it would be suitable:

var [status, headers, body] = app(env);

But should JSGI depend on js 1.7 features? What is the absolute minimum CommonJS js version? I remember this was discussed before -- was it ever spec'd?


Arrays do not play as nicely with JavaScript. JavaScript's syntax is 'optimized' for Object literals so it makes sense to use them.

The code is cleaner and more elegant, consider:

   response[0]
   response[1]["Content-Type"]
  
   vs
  
   response.code
   response.headers["Content-Type"]

In this context is this example to get or set? If get, is this even safe, unless response was a HashP? Could app return a HashP? Even better, should it be standardized that app MUST return a HashP? Sure this adds a small burden on the jsgi library developer to provide a HashP implementation, but I can imagine this sidestepping a pretty big gotcha to new jsgi devs.
 


An object literal *may* allow for some default values, ie:

    return { body: ["hello"] } // status=200, headers={} by default
    return { code: 404 } // headers={}, content=[] by default
    return { code: 404, body: ["not found"] } // headers={} by default

I would propose status rather than code as it's more common and less overloaded, but otherwise, I love it. I tend to prefer explicit over implicit, but I believe default values would be a godsend for readability.
 
  
Moreover, one can easily add extra fields. A useful example (in Nitro at least):

    return { code: 200,
             data: {
             articles: Article.all().fetch(),
             user: User.currentUser()
           } // header={}, content=[] <- a template will be rendered downstream.
          
I can't help but notice the symmetry:

    var app = function(env) {
        return {..}
    }          
   
The function app receives an object literal (env) and returns an object literal ({..})

But the symmetry doesn't stop there. You can tack extra values on the response just like you can on the request env. But are these values better suited to the response? Isn't this what the env is for in the first place?


Finally, there is no need to call .finish() when using the Response helper, just return the response (it's an object already!)
It even allows code like this:

var response = new Response(app(env))

instead of

var ret = app(env);
var response = new Response(ret[0], ret[1], ret[2]);          

That's spectacular.


One benefit of Arrays is they allow for easy porting of existing Rack code. But really, the conversion to a hash is not that difficult.

Don't forget about wsgi code -- there's a metric shitton of it and it would port nicely to the current jsgi. I really like this idea but I'd be hesitant to +1 it until more folks weigh in. Are the symmetries of jsgi to rack, and rack to wsgi *only* due to ease of porting? Or is there more we're missing?


Please don't reject this proposal immediately 'because we have a standard'. We still have time for changes (if they are useful of course). IMHO the standardization process from Jack (a Rack port) to JSGI (A CommonJS standard) was too quick and not much discussed.


I doubt there's that much JSGI production code floating around, so I agree now's the time to nail things down. In this thread [2] I pointed out some other problems (mostly culled from wsgi's mistakes and planned fixes [3]).
 
[1] https://developer.mozilla.org/en/New_in_JavaScript_1.7#Destructuring_assignment

[2] http://groups.google.com/group/commonjs/browse_thread/thread/dbb8c27664a6c778/67bd904cbce005e7?lnk=gst&q=wsgi#67bd904cbce005e7

[3] http://www.wsgi.org/wsgi/WSGI_2.0

George Moschovitis

unread,
Aug 27, 2009, 8:04:29 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
Thank you for the comments!

As of 1.7 javascript has a form of destructuring assignment [1]. While this isn't perfect, it would be suitable:
var [status, headers, body] = app(env);
But should JSGI depend on js 1.7 features? What is the absolute minimum CommonJS js version? I remember this was discussed before -- was it ever spec'd?

javascript 1.7 is not a standard and I don't see destructuring assignment become a standard feature any time soon.

An object literal *may* allow for some default values, ie:

    return { body: ["hello"] } // status=200, headers={} by default
    return { code: 404 } // headers={}, content=[] by default
    return { code: 404, body: ["not found"] } // headers={} by default

I would propose status rather than code as it's more common and less overloaded, but otherwise, I love it. I tend to prefer explicit over implicit, but I believe default values would be a godsend for readability.

yeah status is indeed better (more descriptive).

But the symmetry doesn't stop there. You can tack extra values on the response just like you can on the request env. But are these values better suited to the response? Isn't this what the env is for in the first place?

In my experience it is useful to tack values on the response.

 
Finally, there is no need to call .finish() when using the Response helper, just return the response (it's an object already!)
It even allows code like this:

var response = new Response(app(env))

instead of

var ret = app(env);
var response = new Response(ret[0], ret[1], ret[2]);          

That's spectacular.

indeed ;-)
 
One benefit of Arrays is they allow for easy porting of existing Rack code. But really, the conversion to a hash is not that difficult.

Don't forget about wsgi code -- there's a metric shitton of it and it would port nicely to the current jsgi. I really like this idea but I'd be hesitant to +1 it until more folks weigh in. Are the symmetries of jsgi to rack, and rack to wsgi *only* due to ease of porting? Or is there more we're missing?

I believe it is relatively easy and straightforward to convert from arrays to the object literal format. IMHO this is *not* a showstopper.

I doubt there's that much JSGI production code floating around, so I agree now's the time to nail things down. In this thread [2] I pointed out some other problems (mostly culled from wsgi's mistakes and planned fixes [3]).
 
[1] https://developer.mozilla.org/en/New_in_JavaScript_1.7#Destructuring_assignment

[2] http://groups.google.com/group/commonjs/browse_thread/thread/dbb8c27664a6c778/67bd904cbce005e7?lnk=gst&q=wsgi#67bd904cbce005e7

[3] http://www.wsgi.org/wsgi/WSGI_2.0


-g.

--
blog.gmosx.com

Kris Zyp

unread,
Aug 27, 2009, 8:05:35 AM8/27/09
to comm...@googlegroups.com, narw...@googlegroups.com
+1 for requests being an object (with status, headers, body) instead of
an array. I'd prefer status over code.

Dean Landolt wrote:
> I'm cc'ing commonjs as this is a JSGI spec issue...
>
> On Thu, Aug 27, 2009 at 3:59 AM, George Moschovitis
> <george.mo...@gmail.com <mailto:george.mo...@gmail.com>>

Miguel Coquet

unread,
Aug 27, 2009, 8:11:52 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
+ 1 Not just more flexible but way better to read.

Kevin Dangoor

unread,
Aug 27, 2009, 8:52:50 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
On Thu, Aug 27, 2009 at 8:05 AM, Kris Zyp <kri...@gmail.com> wrote:

+1 for requests being an object (with status, headers, body) instead of
an array. I'd prefer status over code.

+1 from me too. Porting from Rack/WSGI will not really be any harder just because it's an object rather than an array :)

--
Kevin Dangoor

work: http://labs.mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com

George Moschovitis

unread,
Aug 27, 2009, 8:55:08 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
+1 from me too. Porting from Rack/WSGI will not really be any harder just because it's an object rather than an array :)

It is a simple conversion:

[0] -> .status
[1] -> .headers
[2] -> .body

Can be acomplished with the text editor's find and replace function.

-g.

--
blog.gmosx.com

Dean Landolt

unread,
Aug 27, 2009, 8:57:12 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com

It is a simple conversion:

[0] -> .status
[1] -> .headers
[2] -> .body

Can be acomplished with the text editor's find and replace function.


So there's really no other benefits anyone can think of by mirroring the signatures of rack and wsgi? Works for me...

+1 

Wes Garland

unread,
Aug 27, 2009, 9:02:44 AM8/27/09
to comm...@googlegroups.com
I am seriously dubious as to whether JSGI should require JavaScript 1.7.  This is not an ECMA standard, and AFAIK is not implemented outside of Mozilla.

While I'm a huge Mozilla fan, I don't think making CommonJS specific to Mozilla implementations of JavaScript is wise.

It would be nice to be able to implement the standards in any ES3-compliant JavaScript engine, and there are LOT out there.   ES5 should be adopted quickly as well -- after ratification -- although I would suggest thinking long and hard about making use of ES5 features in core libraries.

Besides the "big three" (SpiderMonkey, Rhino, v8), we need to think about
  - Apple SquirrelFish
  - Tamairin (Adobe ActionScript)
  - Windows Scripting Host
  - KDE KJS / Apple JavascriptCore

and there are host of smaller/niche products as well
   - NJS javascript
   - That mobile one mob works on
   - Narcissus
   - plenty of others I'm sure are slipping my mind

Wes
--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

Hannes Wallnoefer

unread,
Aug 27, 2009, 9:14:01 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
-1 from me.

I have a var [status, headers, body] = res; in virtually all of my
Helma middleware, and I think it's perfect.

http://github.com/hns/helma-ng/tree/ef94a1e07dfcab08a7c4fee919dbbddb915fcbb4/modules/helma/middleware

(Note that Helma middleware is not JSGI middleware - it uses Helma's
request object, but it uses the response format defined by JSGI)

2009/8/27 George Moschovitis <george.mo...@gmail.com>:
> Thank you for the comments!
>
>> As of 1.7 javascript has a form of destructuring assignment [1]. While
>> this isn't perfect, it would be suitable:
>> var [status, headers, body] = app(env);
>> But should JSGI depend on js 1.7 features? What is the absolute minimum
>> CommonJS js version? I remember this was discussed before -- was it ever
>> spec'd?
>
> javascript 1.7 is not a standard and I don't see destructuring assignment
> become a standard feature any time soon.

You're wrong, destructuring assignment is one of the planned Harmony
features agreed upon by the ES technical committee:
http://wiki.ecmascript.org/doku.php?id=harmony:proposals

This means writing code that uses this feature for engines that
support it (Spidermonkey, Rhino) is future proof, and if you're
targeting JS 1.5 or 1.6 you can just fall back to something sightly
less verbose. And remember, Harmony is meant to be a small incremental
change to ES syntax, not the huge shift ES4 was meant to be. It
shouldn't take forever to materialize.

>>> An object literal *may* allow for some default values, ie:
>>>
>>>     return { body: ["hello"] } // status=200, headers={} by default
>>>     return { code: 404 } // headers={}, content=[] by default
>>>     return { code: 404, body: ["not found"] } // headers={} by default
>>
>> I would propose status rather than code as it's more common and less
>> overloaded, but otherwise, I love it. I tend to prefer explicit over
>> implicit, but I believe default values would be a godsend for readability.

That's really a reason against this change. You can have default
values in your frameworks, but not on the low level JSGI protocol.

> yeah status is indeed better (more descriptive).
>
>> But the symmetry doesn't stop there. You can tack extra values on the
>> response just like you can on the request env. But are these values better
>> suited to the response? Isn't this what the env is for in the first place?
>
> In my experience it is useful to tack values on the response.

Again, this is not a good idea IMO. Once you add that extra level
beyond HTTP headers and status codes, middlewares are going to depend
on each other in all kind of weird ways.

>>>
>>> Finally, there is no need to call .finish() when using the Response
>>> helper, just return the response (it's an object already!)
>>> It even allows code like this:
>>>
>>> var response = new Response(app(env))
>>>
>>> instead of
>>>
>>> var ret = app(env);
>>> var response = new Response(ret[0], ret[1], ret[2]);
>> That's spectacular.

The Response object isn't part of JSGI, so it's really a detail of
your particular implementation. If a framework wanted to use some
different resoponse object - something JSGI should surely support -
it would still have to do some translation.

All in all, it's my belief that the array representation is the better
fit for representing an HTTP response in JavaScript.

Hannes

George Moschovitis

unread,
Aug 27, 2009, 9:31:07 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
I have a var [status, headers, body] = res; in virtually all of my
Helma middleware, and I think it's perfect.

This is only supported in Mozilla at the moment.

This means writing code that uses this feature for engines that
support it (Spidermonkey, Rhino) is future proof, and if you're
targeting JS 1.5 or 1.6 you can just fall back to something sightly
less verbose. And remember, Harmony is meant to be a small incremental
change to ES syntax, not the huge shift ES4 was meant to be. It
shouldn't take forever to materialize.

it will take 1-2 years to materialize.

and still, var response = app(env) is even cleaner than var [status, headers, body] = app(env).

>>> An object literal *may* allow for some default values, ie:That's really a reason against this change. You can have default
values in your frameworks, but not on the low level JSGI protocol.

I said *may*. Ie this is not part of JSGI, but if a framework wants use default values can do so.

Again, this is not a good idea IMO. Once you add that extra level
beyond HTTP headers and status codes, middlewares are going to depend
on each other in all kind of weird ways.

Not really, extra values would be ignored by JSGI middleware, will only be handled by the application/framework.

>>> Finally, there is no need to call .finish() when using the Response
>>> helper, just return the response (it's an object already!)
>>> It even allows code like this:
>>>
>>> var response = new Response(app(env))
>>>
>>> instead of
>>>
>>> var ret = app(env);
>>> var response = new Response(ret[0], ret[1], ret[2]);
>> That's spectacular.

The Response object isn't part of JSGI, so it's really a detail of
your particular implementation. If a framework wanted to use some
different resoponse object - something JSGI should surely support -
it would still have to do some translation.

It is not my implementation. It is included in Jack. Remember, the original post was sent to the Jack mailing list. However, I think typical Response helpers would include status, headers, body fields and could be used in the place of the object literal.

All in all, it's my belief that the array representation is the better
fit for representing an HTTP response in JavaScript.

I respect your opinion, but I disagree. Can you provide a reason why an array representation is a better fit for JavaScript that will help illustrating your assessment? In your post you only discuss my perceived advantages for using objects. Excluding the benefit of compatibility with Rack/WSGI, I can't see any other benefit in arrays.

kind regards,
-g.


--
blog.gmosx.com

Dean Landolt

unread,
Aug 27, 2009, 9:31:50 AM8/27/09
to comm...@googlegroups.com
>
> javascript 1.7 is not a standard and I don't see destructuring assignment
> become a standard feature any time soon.

You're wrong, destructuring assignment is one of the planned Harmony
features agreed upon by the ES technical committee:
http://wiki.ecmascript.org/doku.php?id=harmony:proposals

This means writing code that uses this feature for engines that
support it (Spidermonkey, Rhino) is future proof, and if you're
targeting JS 1.5 or 1.6 you can just fall back to something sightly
less verbose.

If the whole point of JSGI is to be able to allow us to write reusable middleware (as you point out later), wouldn't every piece of middleware have to have this fallback? That seems like more trouble than it's worth.

Hannes Wallnoefer

unread,
Aug 27, 2009, 10:44:18 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
2009/8/27 George Moschovitis <george.mo...@gmail.com>:
>> I have a var [status, headers, body] = res; in virtually all of my
>> Helma middleware, and I think it's perfect.
>
> This is only supported in Mozilla at the moment.
>>
>> This means writing code that uses this feature for engines that
>> support it (Spidermonkey, Rhino) is future proof, and if you're
>> targeting JS 1.5 or 1.6 you can just fall back to something sightly
>> less verbose. And remember, Harmony is meant to be a small incremental
>> change to ES syntax, not the huge shift ES4 was meant to be. It
>> shouldn't take forever to materialize.
>
> it will take 1-2 years to materialize.
> and still, var response = app(env) is even cleaner than var [status,
> headers, body] = app(env).

Nope, because with destructuring assignment, you have the actual
pieces of the response in local variables, which is much more
convenient than working with the response object as a whole. There's
nothing preventing you from writing var response = app(env) if the
response is an array.

>> >>> An object literal *may* allow for some default values, ie:That's
>> >>> really a reason against this change. You can have default
>> values in your frameworks, but not on the low level JSGI protocol.
>
> I said *may*. Ie this is not part of JSGI, but if a framework wants use
> default values can do so.
>>
>> Again, this is not a good idea IMO. Once you add that extra level
>> beyond HTTP headers and status codes, middlewares are going to depend
>> on each other in all kind of weird ways.
>
> Not really, extra values would be ignored by JSGI middleware, will only be
> handled by the application/framework.

My point is that once you provide that extra level beyond the pure
HTTP response, middleware is going to start using that instead of (or
in addition to) HTTP headers, and other middleware will start to
depend on it. This will result in middleware with all kinds of
undocumented cross-dependencies, which will result in fragile code.

>> >>> Finally, there is no need to call .finish() when using the Response
>> >>> helper, just return the response (it's an object already!)
>> >>> It even allows code like this:
>> >>>
>> >>> var response = new Response(app(env))
>> >>>
>> >>> instead of
>> >>>
>> >>> var ret = app(env);
>> >>> var response = new Response(ret[0], ret[1], ret[2]);
>> >> That's spectacular.
>>
>> The Response object isn't part of JSGI, so it's really a detail of
>> your particular implementation. If a framework wanted to use some
>> different resoponse object - something JSGI should surely support -
>> it would still have to do some translation.
>
> It is not my implementation. It is included in Jack. Remember, the original
> post was sent to the Jack mailing list. However, I think typical Response
> helpers would include status, headers, body fields and could be used in the
> place of the object literal.
>>
>> All in all, it's my belief that the array representation is the better
>> fit for representing an HTTP response in JavaScript.
>
> I respect your opinion, but I disagree. Can you provide a reason why an
> array representation is a better fit for JavaScript that will help
> illustrating your assessment? In your post you only discuss my perceived
> advantages for using objects. Excluding the benefit of compatibility with
> Rack/WSGI, I can't see any other benefit in arrays.
> kind regards,

I said it's a better fit for representing HTTP responses in
JavaScript. And I actually did say why I think so, which is most
importantly because the array as a representation is more reduced and
concise and less inviting to extend the response (and thus introduce a
meta layer beyond HTTP).

Hannes

> -g.
>
> --
> blog.gmosx.com
>
> >
>

Hannes Wallnoefer

unread,
Aug 27, 2009, 10:53:23 AM8/27/09
to comm...@googlegroups.com
2009/8/27 Dean Landolt <de...@deanlandolt.com>:
Agreed, if you're going to write code that should work on any JS
engine, you'll use ES3/JS1.5. But the question is a slightly more
complex one. It's more like: should we design a standard so that it is
_slightly_ more convenient to use with today's syntax when we know
this problem will be gone in 1-2 years? Remember, standards have a
very long life time - they're basically there forever.

So if we change the JSGI response type, it should be for a reason
other than "it looks awkward in JS < 1.7" (or ES < 6).

Hannes

> >
>

George Moschovitis

unread,
Aug 27, 2009, 10:53:53 AM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
I said it's a better fit for representing HTTP responses in
JavaScript. And I actually did say why I think so, which is most
importantly because the array as a representation is more reduced and
concise and less inviting to extend the response (and thus introduce a
meta layer beyond HTTP).

I think that the object literal is the canonical JavaScript structure (much like a list is for lisp). I believe we should leverage JavaScript's excellent syntactical support for objects/hashes. You can enforce the same restrictions in an Object. And you can abuse (in you notion) the Array just like an object (in fact a JavaScript array *is* an object):

var response = [200, {}, ["hello"]];
response.meta1 = "meta1";
response.meta2 = "meta2";

But I think this is missing the point. My main argument is that:

response.status
response.headers
response.body

is much more readable (and feels more JavaScriptish) than:

response[0]
response[1]
response[2]

I think this is obvious. But I guess you see it from another perspective. As I said I fully respect your opinion.

Let's wait for more people to step in with opinions.

kind regards,
George





 


Hannes

> -g.
>
> --
> blog.gmosx.com
>
> >
>





--
blog.gmosx.com

George Moschovitis

unread,
Aug 27, 2009, 11:00:03 AM8/27/09
to comm...@googlegroups.com


---------- Forwarded message ----------
From: George Moschovitis <george.mo...@gmail.com>
Date: Thu, Aug 27, 2009 at 5:59 PM
Subject: Re: [narwhal] Re: JSGI Response change proposal
To: narw...@googlegroups.com



Still, I don't see how hanging additional off the response object could impact pure middleware (which should only peek at the three HTTP properties). Plus, the same argument could be made against extending the env object -- and certainly nobody would suggest freezing that.

This is not a valid argument, you can attach meta data to the array (remember, it is an object too). This is the hack that nitro currently uses to pass template data to the render middleware.

In JS an array coule be considered a hash with numeric keys. There is no benefit in using an array over an object. You just get strange syntax like:

[200, {}, ["hello"]]

instead of the clean and elegant:

{ status: 200, headers: {}, body: ["hello"] }

which looks more JavaScriptish to you?


-g.


--
blog.gmosx.com



--
blog.gmosx.com

George Moschovitis

unread,
Aug 27, 2009, 11:10:17 AM8/27/09
to comm...@googlegroups.com
---------- Forwarded message ----------
From: George Moschovitis <george.mo...@gmail.com>
Date: Thu, Aug 27, 2009 at 6:10 PM
Subject: Re: [narwhal] Re: JSGI Response change proposal
To: narw...@googlegroups.com


{ status: 200, headers: {}, body: ["hello"] }

which looks more JavaScriptish to you?

An argument could be made that it's a little verbose, but there's no doubt it's intent is clearer and self-describing.

I guess It is always a trade-off ;-)

-g.



--
blog.gmosx.com

Joshaven Potter

unread,
Aug 27, 2009, 12:12:26 PM8/27/09
to comm...@googlegroups.com
I don't know where the argument has come from or gone to but I don't think html messages should be represented as Arrays.  Arrays are for ordered data (the key is the index of the data not a string which can be more meaningful), html messages are not necessarily ordered so an object is a more accurate way to represent the html status + header + body in JavaScript.  

+1 for:   { status: 200, headers: {}, body: ["hello"] }

I honestly don't care which looks more like JavaScriptish.  I care which is the more accurate and reliable representation of the data.


 


--
Sincerely,
Joshaven Potter

"No man making a profession of faith ought to sin, nor one possessed of love to hate his brother. For He that said, “Thou shalt love the Lord thy God,”  said also, “and thy neighbor as thyself.”  Those that profess themselves to be Christ’s are known not only by what they say, but by what they practice. “For the tree is known by its fruit.”" -- Ignatius

Mark S. Miller

unread,
Aug 27, 2009, 12:23:15 PM8/27/09
to comm...@googlegroups.com
On Thu, Aug 27, 2009 at 6:02 AM, Wes Garland <w...@page.ca> wrote:
ES5 should be adopted quickly as well -- after ratification -- although I would suggest thinking long and hard about making use of ES5 features in core libraries.

ES5 unratified is already much closer to what we can expect engines to implement that ES3 is to what anyone implemented. ES3 conformance is a mess. The community is already investing heavily in to-be-official test suites for ES5 so we can avoid repeating this mess.

I recommend adopting ES5 now. The relevant impediment isn't ratification; it is deployment of ES5 implementations.


--
   Cheers,
   --MarkM

Mark S. Miller

unread,
Aug 27, 2009, 12:26:25 PM8/27/09
to comm...@googlegroups.com, narw...@googlegroups.com
On Thu, Aug 27, 2009 at 6:14 AM, Hannes Wallnoefer <han...@gmail.com> wrote:
You're wrong, destructuring assignment is one of the planned Harmony
features agreed upon by the ES technical committee:
http://wiki.ecmascript.org/doku.php?id=harmony:proposals

This means writing code that uses this feature for engines that
support it (Spidermonkey, Rhino) is future proof, and if you're
targeting JS 1.5 or 1.6 you can just fall back to something sightly
less verbose. And remember, Harmony is meant to be a small incremental
change to ES syntax, not the huge shift ES4 was meant to be. It
shouldn't take forever to materialize.

I agree. But we may be talking another 2-3 year delay after ES5. And the details of this are hardly settled.

That said, I do like this feature a lot.

--
   Cheers,
   --MarkM

Daniel Friesen

unread,
Aug 27, 2009, 12:33:47 PM8/27/09
to comm...@googlegroups.com
Note that I don't like the hack of adding meta data onto the array.

While Arrays are objects the fact you can add other data to them is a
side-product that shouldn't be relied on. The de-facto standard method
of copying an array is arr.slice(); however meta data is not preserved,
in fact no array methods preserve any data other than what's inside the
array itself.
I see no reason to disallow middleware the ability to copy the response
passed to it, thus I don't like arrays with meta data as copying an
array would destroy that meta data.

That kind of issue is not present with a good deep-copy technique, so +1
for object literals.

As a side note. Using an object literal means that response classes
themselves, if they conform with those three keys, can be passed up
through middleware verbatim cleanly without even converting back and
forth between the class and an array.


The one thing still racking my head is body. I understand how using
.forEach supports the chunking of data needed for streaming, but I don't
understand how that would work itself on a stream. How can .forEach be
handled async while waiting for the app to write data?

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
> blog.gmosx.com <http://blog.gmosx.com>
>
>
>
> --
> blog.gmosx.com <http://blog.gmosx.com>
>
> >

Kris Kowal

unread,
Aug 27, 2009, 12:35:33 PM8/27/09
to comm...@googlegroups.com, narw...@googlegroups.com
This message was only posted on Narwhal; I've included it in whole
here. Most of this suggests common case optimizations.

On Thu, Aug 27, 2009 at 6:52 AM, Alexandre.Morgaut<mor...@hotmail.com> wrote:
>
> I expect some changes but not exactly like that.
> I'm happy you made the point on some of these "problems" (if we can
> talk about problems)
>
> The main point for me is default values (why specify 200 for each
> correct responses).
> We're implementing a some Server Side JavaScript on a Web Server and
> we're looking on how everybody does it (narwhal, jaxer, helmang, ...)
> We choose to support the JSGI API becaue of its simplicity and add a
> more complete API for more specific needs
>
> We very liked the simplicity of an hello world with jack :
>
> return [200, {Content-Type: 'text/html'}, ['hello world']];
>
> But still being compatible we thank about default values  and
> unordered array
>
> As we can see, the response is sent as an array with 3 items of
> different types :
> - a number for the status code
> - an object for the HTTP headers
> - an array for the body(ies ?)
>
> If a request event handler return an array :
>         - The position of the 3 elements of this array is not
> relevant.
>         - Only the type of these elements must checked to know to
> which part of the response it belongs :
>                  - number -> status code
>                  - object -> header
>                  - string or array -> body
>         - the default status code is "200" (OK)
>         - if no body is specified, the default status code becomes
> "204" (No Content)
>         - if no header is specified "Content-Type" and "Content-
> Length" fields are set from the nature of the body :
>                  - "text/plain" for strings or date objects and
> scalars embeded in the array (Number, Boolean, Null, Undefined)
>                  - "application/json" for other object (serialized as
> JSON)
>                  - "application/octet-stream" for images, blobs, or
> binary files if a more specific mediatype can't be recognized (if
> binary data supported)
>                  - xml and html mediatypes for DOM elements depending
> of their nature (text/html or text/xml may be acceptable in most
> cases) (if DOM supported)
>         - if the body array element has several elements, this means
> that the response must be sent has multipart
>         - if no body is sent and the status code match to an error
> (40x and 50x), a default content should be sent in the most supported
> format of the client (html, text, or json, and if you are really
> motivated, an audio version would be a very good point for
> accessibility and specific devices)
>
> At this point the user is quite helped but a response specified
> without status code and headers looks horrible
>
> return [['hellow world']];
> or
> return [[response]];
>
> So, any returned value which is not an array should be considered as a
> content :
>
> return 'hellow world';
>
> return 42;
>
> Access to "env" :
>
> on this page https://wiki.mozilla.org/ServerJS/System we can see a
> proposal to access environment variables
> I prefer to access it for system.env and not from an argument
> you still can start your handler by :
> var env = system.env


It is important that the application receive a mutable copy of the
environment variables, not use the environment directly. Request
routing frameworks must be able to give the application the impression
that every application is the main script for whatever part of the
(PATH_INFO) that remains to be dealt with.

> So what looks like such API in a use case :
>
> function () {
>  var out,
>        env = system.env;
>  (...)
>  out.push(201);
>  out.push('employee created');
>  return out;
> }
>
> or
>
> function () {
>  // trace client request
>  // system.env is an object so it is sent JSON encoded with JSON
> mediatype
>  return system.env;
> }
>
>
> Optionally, a false returned value could have a 400 default status
> code


I have only minor trepidation about these use case optimizations. I
think we'll find that, for this kind of tuning, a "simplification" on
one side is a "complication" on the other, and since there will be a
lot of middleware standing on both sides of this equation, everyone is
likely to suffer. I do not consider it onerous to return an ordered
triple, although I am also very likely to extend that ordered triple
as George mentions, to provide additional optional metadata to
middleware as he does for template contexts. For that reason alone, I
think that an object would make a better conduit for a
*standards-managed* name space.

{
status: 200,
body: ["Hello, World"],
headers: {"Content-type": "text/plain"},
xNitroContext: {"title": "Hello", "author": "Foo Bar",
"breadcrumbs": …, "navigation": …}
}

Kris Kowal

Kris Kowal

unread,
Aug 27, 2009, 12:41:03 PM8/27/09
to comm...@googlegroups.com
On Thu, Aug 27, 2009 at 9:33 AM, Daniel
Friesen<nadir.s...@gmail.com> wrote:
> The one thing still racking my head is body. I understand how using
> .forEach supports the chunking of data needed for streaming, but I don't
> understand how that would work itself on a stream. How can .forEach be
> handled async while waiting for the app to write data?

This is the reason why Python uses "next" as its iterator primitive
method, not "forEach". "next" is a function that returns the next
value of an iteration or throws a StopIteration exception. The
"forEach" construct in turn is built on top of "next" and not meant to
be overridden:

forEach = function (block, context) {
while (true) {
try {
block.call(context, this.next());
} catch (exception) {
if (exception instanceof StopIteration) {
break;
} else {
throw exception;
}
}
}
};

Which could be extended at a price to do SkipIteration, for both break
and continue analogs.

Kris Kowal

Tom Robinson

unread,
Aug 27, 2009, 1:05:35 PM8/27/09
to comm...@googlegroups.com
-1

As a middleware developer I can see how it's not ideal, but since it's
much more common to write applications than middleware I think it's
better to optimize for that case. Typing [200, {}, []] is about 1/3 as
many characters as { code : 200, headers : {}, body [] } and less
error prone.

-tom

George Moschovitis

unread,
Aug 27, 2009, 1:36:45 PM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
The main benefit pf my proposal is that it makes the API actually useful to the application developer.

You cannot really expect an application developer to write code like this:

exports.GET = function(env) {
  ...
  return [200, {}, ["This is my response"]];
}

This looks *ugly*. It may be acceptable (but painful) in middleware but is unusable in the application domain. That is why you have to introduce an actual object (Response()).  Why not just use an object in the first place?

Consider this example:

exports.GET = function(env) {
  ...
  return { body: ["Hello there"] };
}

in conjunction with a Default middleware that sets default values for status, headers.


The proposal was made for the benefit of application developers.





On Thu, Aug 27, 2009 at 8:04 PM, Tom Robinson <tlrob...@gmail.com> wrote:

-1

As a middleware developer I can see how it's not ideal, but since it's
much more common to write applications than middleware I think it's
better to optimize for that case. Typing [200, {}, []] is about 1/3 as
many characters as { code : 200, headers : {}, body [] } and less
error prone.

-tom

On Aug 27, 2009, at 12:59 AM, George Moschovitis wrote:

> Dear all
>
> It is a period of changes for Se^H^HCommonJS and Narwhal (engines,
> etc). Perhaps it is the last chance to (re)-propose a change in
> JSGI. Please bear with me. I would like to propose the following
> change to the response object:
>
> From
>
> [200, {}, ["hello"]]
>
> to
>
> { code: 200, headers: {}, body: [] }
>
> Rack's array responses leverage Ruby's syntax features:
>
> return 200, {}, ["hello"]
> code, headers, body = app(env)
>
> Arrays do not play as nicely with JavaScript. JavaScript's syntax is
> 'optimized' for Object literals so it makes sense to use them.
>
> The code is cleaner and more elegant, consider:
>
>    response[0]
>    response[1]["Content-Type"]
>
>    vs
>
>    response.code
>    response.headers["Content-Type"]
>
> An object literal *may* allow for some default values, ie:
>
>     return { body: ["hello"] } // status=200, headers={} by default
>     return { code: 404 } // headers={}, content=[] by default
>     return { code: 404, body: ["not found"] } // headers={} by default
>
> Moreover, one can easily add extra fields. A useful example (in
> Nitro at least):
>
>     return { code: 200,
>              data: {
>              articles: Article.all().fetch(),
>              user: User.currentUser()
>            } // header={}, content=[] <- a template will be rendered
> downstream.
>
> I can't help but notice the symmetry:
>
>     var app = function(env) {
>         return {..}
>     }
>
> The function app receives an object literal (env) and returns an
> object literal ({..})
>
> Finally, there is no need to call .finish() when using the Response
> helper, just return the response (it's an object already!)
> It even allows code like this:
>
> var response = new Response(app(env))
>
> instead of
>
> var ret = app(env);
> var response = new Response(ret[0], ret[1], ret[2]);
>
> One benefit of Arrays is they allow for easy porting of existing
> Rack code. But really, the conversion to a hash is not that difficult.
>
> Please don't reject this proposal immediately 'because we have a
> standard'. We still have time for changes (if they are useful of
> course). IMHO the standardization process from Jack (a Rack port) to
> JSGI (A CommonJS standard) was too quick and not much discussed.
>
> I would love to hear opinions from many people on this proposal.
>
>
>
> kind regards,
> George.
>
>
> --
> blog.gmosx.com
>
> >






--
blog.gmosx.com

Mike Wilson

unread,
Aug 27, 2009, 1:49:39 PM8/27/09
to comm...@googlegroups.com, narw...@googlegroups.com
+1 for object literal. (And wrt error prone I think the array destructuring syntax risks getting the order wrong.)
 
A few other things that come to mind:
 
1) There's an asymmetry for input and output bodies; the former being an inputstream and the latter a forEach:able object. Is this good?
 
2) Compulsary Content-Length; other web stacks like f ex Java servlets do chunked transfer encoding when there is no Content-Length. Is this not in scope for JSGI?
 
3) I guess the spec page http://jackjs.org/jsgi-spec.html means to say:
"There must be a Content-Type header, except when..."
 
Thanks
Mike Wilson

George Moschovitis

unread,
Aug 27, 2009, 1:57:07 PM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
Typing [200, {}, []] is about 1/3 as
many characters as { code : 200, headers : {}, body: [] } and less
error prone.

Optimizing for chars is peculiar, in the extreme case:

{b: []}

is shorter than

[200, {}, []]

but of course it makes no sense to trade-of char count over readability. 

So, really, I cannot see the benefit.

-g.


--
blog.gmosx.com

Isaac Z. Schlueter

unread,
Aug 27, 2009, 3:00:31 PM8/27/09
to CommonJS, NarwhalJS
+1 to making the JSGI response an object.

On Aug 27, 10:04 am, Tom Robinson <tlrobin...@gmail.com> wrote:
> -1
>
> As a middleware developer I can see how it's not ideal, but since it's
> much more common to write applications than middleware I think it's
> better to optimize for that case. Typing [200, {}, []] is about 1/3 as
> many characters as { code : 200, headers : {}, body [] } and less
> error prone.

I'm not convinced it's less error prone, personally. While an array
literal is certainly fewer characters than an object literal, in
complex applications, it's more likely that you'll have disparate bits
of code touching different pieces of the response. I'd argue that
response.status = 200 is less error prone than response[0] = 200.

In fact, were I to write a program against JSGI as it exists today,
I'd be tempted to define something like: var STATUS_CODE = 0, just so
that I could do response[STATUS_CODE] to make my code more readable,
albeit with a slight performance hit. (Incidentally, this would
future-proof the app against this change, as you could easily change
that to STATUS_CODE="status" in one place.)

Let's not make developers jump through hoops to work around warts.
Let's use what Javascript is good at. There are plenty of warts
already ;)


On Aug 27, 9:35 am, Kris Kowal <cowbertvon...@gmail.com> wrote:
> likely to suffer. I do not consider it onerous to return an ordered
> triple, although I am also very likely to extend that ordered triple
> as George mentions, to provide additional optional metadata to
> middleware as he does for template contexts. For that reason alone, I
> think that an object would make a better conduit for a
> *standards-managed* name space.
>
> {
> status: 200,
> body: ["Hello, World"],
> headers: {"Content-type": "text/plain"},
> xNitroContext: {"title": "Hello", "author": "Foo Bar",
> "breadcrumbs": …, "navigation": …}
> }

My thoughts exactly. Yes. Kris, you nailed it.

While an ordered triple *does* still support this sort of extension,
it's less beautiful and less discoverable than this method.


Re: Now middleware will all depend on one anothers' nonstandard stuff

That was going to happen anyway. (Qv. Nitro) Javascript is not a
language that is built on a philosophy of doing much to protect the
programmer from himself. So long as the spec defines a very small
number of "important" fields, and a standard syntax for extending the
object, I think we'll be fine.

(Aside: towards that end, I propose starting non-standard fields with
a lower-case "x" as Kris has done above. My first instinct was to use
_, but this is already used in many contexts to mean "private", and
"xFooBar" follows a convention established for HTTP headers such as X-
Powered-By and X-Pingback.)


Re: destructuring assignment makes it ok

It seems to me that destructuring assignment vs object syntax is a
bikeshed, and it is unlikely to be supported in v8 or squirrelfish any
time soon. Array literal syntax is fine, but as I mentioned above,
not likely to be used everywhere, if different parts of your app
handle different things. It seems we can all agree, however, that
"response[0] = 200" is lamer than "response.status = 200".

Saving characters is kind of a null point wrt server-side code. What
really matters for bugs and readability is reducing the number of
tokens and the cognitive distance between tokens and their meaning.
What really matters for interoperability is making sure that any
extensions follow a consistent pattern, and make sure that developers
understand the tradeoffs involved in using those extensions.


Re: furthermore...

An array has a lot of stuff on it that the JSGI response doesn't
need. (push, splice, map, reduce, length, etc.) Javascript programs
tend to treat Arrays as ordered lists of like things. While this
isn't required as it is in C—since that would violate the "you are
allowed to shoot your foot" principle, and who are we to say what's
"like" in your app?—it's clearly implied by the various Array
methods. Thus, Javascript arrays are not the same as Ruby tuples or
Lisp lists.

The status code is not the same kind of thing as the headers hash,
which is not the same kind of thing as the body. Making it an Array
introduces cognitive dissonance with the rest of the language, and
offers no benefit. The fact is, the JSGI response actually *is* an
object with three meaningfully predefined fields, and we'd do well to
respect those fields by giving them proper names.

--i

Wes Garland

unread,
Aug 27, 2009, 4:13:07 PM8/27/09
to comm...@googlegroups.com
>  I honestly don't care which looks more like JavaScriptish.  I care which is the more accurate and reliable representation of the data.--

I'm with you there.

So when are we going to discuss how to resolve CONTENT-TYPE vs. cOntent-Type: ?

My proposal  is
  - last one wins
  - "last one" defined by iteration order

This has the side effect of relying on iteration order as implemented in browser engines, although it remains uncodified even in ES3 I think.   I think Rhino finally gets this right as of about six months ago.  (Right, in this case, being awfully subjective).

Wes

Wes Garland

unread,
Aug 27, 2009, 4:14:17 PM8/27/09
to comm...@googlegroups.com
> better to optimize for that case. Typing [200, {}, []] is about 1/3 as
> many characters as { code : 200, headers : {}, body [] } and less
> error prone.

You know, we could always accept and codify both.

Joshaven Potter

unread,
Aug 27, 2009, 4:26:28 PM8/27/09
to comm...@googlegroups.com
how often do you type out html packets anyway???

In my opinion, I am less likely to make a mistake with:
response.body = "Hello World";
then:
response[2] = "Hello World";

or even:
{code:"...", headers:"...", body:"Hello World"};
then:
['...','...', "Hello World"];

Also, I would be happier with the first way of accessing the HTML message then with specifying with the later... and the nice thing is the object literal, in my opinion is the more readable way of doing either method.



Daniel Friesen

unread,
Aug 27, 2009, 4:43:54 PM8/27/09
to comm...@googlegroups.com
And how many of those people writing apps are even going to be typing that?
I was under the impression that most people would prefer to use whatever
Request/Response object they have their hands on from Jack or whatever,
rather than playing with the protocol themselves with absolutely no
helpful layer in between.

The protocol is designed for communication, not for being written by
hand. If anything is going to do something by hand it's more likely
going to be middleware.

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Joshaven Potter

unread,
Aug 27, 2009, 5:08:31 PM8/27/09
to comm...@googlegroups.com
1) An Object does a better job of representing the data then an Array because the members of a http message is not about order and Arrays are about order.
2) As an application developer I do sometimes read or modify things that end up touching low level stuff so it makes more since to me if I can look down into my middleware and see what its doing... and frankly response as an Array has no benefits unless forcing people to reference documentation is the benefit your looking for.

1/3 less typing means fewer mistakes... is not a valid concern when the message encoding is done in middleware anyway.

If I were writing middle ware I would create an object that would handle this anyway so I guess if it were an Array then I would just access the array through methods in my object anyway.  You can save me the trouble by making it an object to begin with.

The only argument I could see for an Array would be conserving memory usage... which seems like a ridiculous idea then I am using a machine with 4GB of RAM... if I had only 4MB then I would probably be worried about 10 or so fewer bits in my RAM per response.  




Isaac Z. Schlueter

unread,
Aug 27, 2009, 5:49:09 PM8/27/09
to CommonJS
It seems like the aye's have it, roughly 10:2.

Anyone care to weigh in with the costs of making such a change or a
strategy/timeline for updating the documentation and code?

Should we support/codify both as Wes suggested?


Note for destructuring assignment afficianados: You can still do that
with an object. The var names have to match the keys for extra
goodness:

var {headers, body, status} = app(env);

but for just a few extra characters, you can name the vars whatever
you want:

var {body:x, headers:y, status:z} = app(env);

Of course, this is Mozilla-only as of today, which makes it
objectionable to some, but my point here is that using an object
doesn't prevent destructuring assignments in general.

Vote count:

+1
dean landolt
george moschovitis
kris zyp
miguel croquet
kevin dangoor
joshaveen potter
isaac schlueter
kris kowal
daniel friesen
mike wilson

-1
hannes wallnoefer
tom robinson

--i

Joshaven Potter

unread,
Aug 27, 2009, 5:55:25 PM8/27/09
to comm...@googlegroups.com
There is a good old string method that may come in handy called toLowerCase()... Your object method access case insensitive. :-)

Tom Robinson

unread,
Aug 27, 2009, 6:36:40 PM8/27/09
to comm...@googlegroups.com
On Aug 27, 2009, at 1:14 PM, Wes Garland wrote:

> > better to optimize for that case. Typing [200, {}, []] is about
> 1/3 as
> > many characters as { code : 200, headers : {}, body [] } and less
> > error prone.
>
> You know, we could always accept and codify both.

Having multiple response representations is a bad idea for the same
reason we specify that the body must respond to "forEach" and yield
objects that respond to "toByteString"... you don't want to have to
test for different objects types in every piece of middleware and
handler. A single API is really really key here.

Dean Landolt

unread,
Aug 27, 2009, 7:18:57 PM8/27/09
to comm...@googlegroups.com
On Thu, Aug 27, 2009 at 4:13 PM, Wes Garland <w...@page.ca> wrote:
>  I honestly don't care which looks more like JavaScriptish.  I care which is the more accurate and reliable representation of the data.--

I'm with you there.

So when are we going to discuss how to resolve CONTENT-TYPE vs. cOntent-Type: ?

My proposal  is
  - last one wins
  - "last one" defined by iteration order

This has the side effect of relying on iteration order as implemented in browser engines, although it remains uncodified even in ES3 I think.   I think Rhino finally gets this right as of about six months ago.  (Right, in this case, being awfully subjective).

When would you see two Content-Type headers?

Daniel Friesen

unread,
Aug 27, 2009, 7:47:16 PM8/27/09
to comm...@googlegroups.com
Cept this is access, not setting or comparing:

req["Content-Type".toLowerCase()] // Does NOT do what we are trying to do

We are trying to deal with the issue with headers like:
{ 'Content-Type: "text/html", 'content-type': "text/plain",
'CONTENT-TYPE', "image/png" }

What do you grab, what can you set?

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Tom Robinson

unread,
Aug 27, 2009, 8:29:55 PM8/27/09
to comm...@googlegroups.com, narw...@googlegroups.com
I'm ready to concede to the object fanboys ;)

But we should definitely NOT support both in the spec, as I explained in another message:

Having multiple response representations is a bad idea for the same   
reason we specify that the body must respond to "forEach" and yield   
objects that respond to "toByteString"... you don't want to have to   
test for different objects types in every piece of middleware and   
handler. A single API is really really key here. 

However it might be worth *temporarily* adding a shim to all existing middleware/handlers that will automatically convert the array to object, and print a deprecation warning.

    function jsgi_deprecated(res) {
        if (Array.isArray(res)) {
            print("WARNING ...");
            return { status : res[0], headers: res[1], body : res[2] };
        }
        return res;
    }


I can probably update jack, jack-servlet, jack-dav, etc, in a few hours, and presumably it's not a big undertaking for George's Nitro since he's the one spearheading this movement.

Anyone else have significant code bases following the current JSGI spec?

-tom

Tom Robinson

unread,
Aug 27, 2009, 8:53:41 PM8/27/09
to comm...@googlegroups.com
On Aug 27, 2009, at 4:47 PM, Daniel Friesen wrote:

>
> Cept this is access, not setting or comparing:
>
> req["Content-Type".toLowerCase()] // Does NOT do what we are trying
> to do
>
> We are trying to deal with the issue with headers like:
> { 'Content-Type: "text/html", 'content-type': "text/plain",
> 'CONTENT-TYPE', "image/png" }
>
> What do you grab, what can you set?


AFAICT, the options are:

0. Use a plain Object for the headers object, then use helper
functions in middleware to handle different cases. Current solution.

In Jack middleware we just use "case-preserving" hash functions
(called "HashP").


1. Require all headers be a certain case/format. I don't think we can
do this. I believe HTTP says headers are case-insensitive, but it's
good practice to use the "standard" format, which isn't always "Abc-
Def", for example ETag.


2. Require something more complicated be the headers object. The API
would have to be something like get(), set(), add(), has(). But then
you have to do:

return { status : 200, headers : SpecialHeaderObject({ "Content-
Type" : "foo" }), body : ["bar"] }


-tom

Hannes Wallnoefer

unread,
Aug 27, 2009, 8:55:17 PM8/27/09
to narw...@googlegroups.com, comm...@googlegroups.com
Same for me - not a big deal, I can probably update my code in minutes.

Hannes

2009/8/28 Tom Robinson <tlrob...@gmail.com>:

NevilleB

unread,
Aug 27, 2009, 10:03:00 PM8/27/09
to CommonJS
+1 here

Wes Garland

unread,
Aug 27, 2009, 11:06:21 PM8/27/09
to comm...@googlegroups.com
> When would you see two Content-Type headers?

When two different app developers working on the same request spell it differently (with respect to case).

Trust me, it happens. It's stupid and annoying, but that matters not.

George Moschovitis

unread,
Aug 28, 2009, 12:25:39 AM8/28/09
to narw...@googlegroups.com, comm...@googlegroups.com
I'm ready to concede to the object fanboys ;)

nice ;-)

 and presumably it's not a big undertaking for George's Nitro since he's the one spearheading this movement.

will take a few minutes.

thank you,
-g.

--
blog.gmosx.com

George Moschovitis

unread,
Aug 28, 2009, 12:29:04 AM8/28/09
to CommonJS
>     var {headers, body, status} = app(env);
>     var {body:x, headers:y, status:z} = app(env);

interesting ;-)

-g.

Hannes Wallnoefer

unread,
Aug 28, 2009, 3:52:41 AM8/28/09
to comm...@googlegroups.com
2009/8/28 Tom Robinson <tlrob...@gmail.com>:
>
> On Aug 27, 2009, at 4:47 PM, Daniel Friesen wrote:
>
>>
>> Cept this is access, not setting or comparing:
>>
>> req["Content-Type".toLowerCase()] // Does NOT do what we are trying
>> to do
>>
>> We are trying to deal with the issue with headers like:
>> { 'Content-Type: "text/html", 'content-type': "text/plain",
>> 'CONTENT-TYPE', "image/png" }
>>
>> What do you grab, what can you set?

It's an interesting problem. In Helma NG, middleware is invoked from a
request.process() closure that in turn calls each middleware and
finally the app (I think manually rolling middleware balls is
cumbersome, so middleware is defined as an array of module names).

So it would be easy to replace the headers object with a case
insensitive version after the app has been invoked, but it's a bit
dangerous to have developers grow accustomed to
headers['content-type'] being case insensitive. So at least for now I
think I'll stick to Narwhal's HashP. Hashing vs. iterating is not a
big issue for collections with ~5 elements, anyway.

Hannes

Ash Berlin

unread,
Aug 28, 2009, 4:38:59 AM8/28/09
to comm...@googlegroups.com

On 28 Aug 2009, at 01:53, Tom Robinson wrote:

>
> On Aug 27, 2009, at 4:47 PM, Daniel Friesen wrote:
>
>>
>> Cept this is access, not setting or comparing:
>>
>> req["Content-Type".toLowerCase()] // Does NOT do what we are trying
>> to do
>>
>> We are trying to deal with the issue with headers like:
>> { 'Content-Type: "text/html", 'content-type': "text/plain",
>> 'CONTENT-TYPE', "image/png" }
>>
>> What do you grab, what can you set?
>
>
> AFAICT, the options are:
>
> 0. Use a plain Object for the headers object, then use helper
> functions in middleware to handle different cases. Current solution.
>
> In Jack middleware we just use "case-preserving" hash functions
> (called "HashP").
>
>
> 1. Require all headers be a certain case/format. I don't think we can
> do this. I believe HTTP says headers are case-insensitive, but it's
> good practice to use the "standard" format, which isn't always "Abc-
> Def", for example ETag.
>
>
> 2. Require something more complicated be the headers object. The API
> would have to be something like get(), set(), add(), has(). But then
> you have to do:
>
> return { status : 200, headers : SpecialHeaderObject({ "Content-
> Type" : "foo" }), body : ["bar"] }
>
>
> -tom


1) has my vote. We specify the case for the 'standard' HTTP headers
and say if you want it to work properly in middle ware, use this case:

http://github.com/ruediger/flusspferd/blob/e0948a3719bfb11b9a60090a4faa2571c803a846/src/js/http/headers.js#L36

(I stole that list from somewhere or other, I forget where now)

Ash Berlin

unread,
Aug 28, 2009, 4:45:39 AM8/28/09
to narw...@googlegroups.com, comm...@googlegroups.com
On 28 Aug 2009, at 01:29, Tom Robinson wrote:

I'm ready to concede to the object fanboys ;)

But we should definitely NOT support both in the spec, as I explained in another message:

Having multiple response representations is a bad idea for the same   
reason we specify that the body must respond to "forEach" and yield   
objects that respond to "toByteString"... you don't want to have to   
test for different objects types in every piece of middleware and   
handler. A single API is really really key here. 


The other thing that was being discussed was optional members/defaults in the struct. I think the JSGI spec should require them. If some middleware wants to default it, thats fine )and can do so while still following the spec) but at the bottom level it should be required.

-ash

George Moschovitis

unread,
Aug 28, 2009, 5:49:12 AM8/28/09
to narw...@googlegroups.com, comm...@googlegroups.com
The idea was to ease the pain of the middleware developer. The developer can just call:

body.forEach(...)

without checking if body is a String or an object responding to .forEach();

Originally, Tom added a .forEach() method to String, but the consensus was to remove it. Another reason was compatibility with Rack.

However, as Tom said, perhaps we should optimize for the application developer and not the middleware developer. Personally, I think your suggestion to use arrays only for multipart/chunked responses is reasonable. But I am not sure about the complications.


kind regards,
George.



On Fri, Aug 28, 2009 at 12:01 PM, Alexandre.Morgaut <mor...@hotmail.com> wrote:


Something I still don't understand is why sending the body in an
array ?

All the more if this array always have a single element and if this
body isn't multipart

What did I miss ?





--
blog.gmosx.com

Wes Garland

unread,
Aug 28, 2009, 9:36:29 AM8/28/09
to comm...@googlegroups.com
> Personally, I think your suggestion to use arrays only for multipart/chunked responses
> is reasonable. But I am not sure about the complications.

I worry about the app developer compositing the body via the use of string concatenation rather the Array.push in this case.  Due to JS's immutable strings, the latter is significantly cheaper than the former in many common cases.

These same cases are littered all over the web with the innerHTML += idiom.  It's terrible. It would be nice if we could encourage better document generation without resorting to a shadow DOM or E4X.

Wes

Dean Landolt

unread,
Aug 28, 2009, 9:38:43 AM8/28/09
to comm...@googlegroups.com
> AFAICT, the options are:
>
> 0. Use a plain Object for the headers object, then use helper
> functions in middleware to handle different cases. Current solution.
>
> In Jack middleware we just use "case-preserving" hash functions
> (called "HashP").
>
>
> 1. Require all headers be a certain case/format. I don't think we can
> do this. I believe HTTP says headers are case-insensitive, but it's
> good practice to use the "standard" format, which isn't always "Abc-
> Def", for example ETag.
>
>
> 2. Require something more complicated be the headers object. The API
> would have to be something like get(), set(), add(), has(). But then
> you have to do:
>
>      return { status : 200, headers : SpecialHeaderObject({ "Content-
> Type" : "foo" }), body : ["bar"] }
>
>
> -tom


1) has my vote. We specify the case for the 'standard' HTTP headers
and say if you want it to work properly in middle ware, use this case:

http://github.com/ruediger/flusspferd/blob/e0948a3719bfb11b9a60090a4faa2571c803a846/src/js/http/headers.js#L36

(I stole that list from somewhere or other, I forget where now)

If you specify a header case as consistently all lower or upper case, you could cover the nonstandard headers too (and nobody would have to consult an authority list). I don't know how everyone would feel about having to write headers: { 'content-type' : 'foo' } -- but if you don't like that you could always fall back on something like a HashP that returns lowercase. In mixed case lies madness.

Kevin Dangoor

unread,
Aug 28, 2009, 9:47:40 AM8/28/09
to narw...@googlegroups.com, comm...@googlegroups.com
On Fri, Aug 28, 2009 at 5:49 AM, George Moschovitis <george.mo...@gmail.com> wrote:
However, as Tom said, perhaps we should optimize for the application developer and not the middleware developer. Personally, I think your suggestion to use arrays only for multipart/chunked responses is reasonable. But I am not sure about the complications.

While I prefer the object return value to the array one, I do think that JSGI is generally *not* for the app developer. App developers will generally use a higher level abstraction, I believe...

multiple representations is bad... I think the body should stay with .forEach

Kevin

--
Kevin Dangoor

work: http://labs.mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com

George Moschovitis

unread,
Aug 28, 2009, 11:30:18 AM8/28/09
to narw...@googlegroups.com, comm...@googlegroups.com
On Fri, Aug 28, 2009 at 4:47 PM, Kevin Dangoor <dan...@gmail.com> wrote:
On Fri, Aug 28, 2009 at 5:49 AM, George Moschovitis <george.mo...@gmail.com> wrote:
However, as Tom said, perhaps we should optimize for the application developer and not the middleware developer. Personally, I think your suggestion to use arrays only for multipart/chunked responses is reasonable. But I am not sure about the complications.

While I prefer the object return value to the array one, I do think that JSGI is generally *not* for the app developer. App developers will generally use a higher level abstraction, I believe...

multiple representations is bad... I think the body should stay with .forEach

I hate to admit that .forEach has indeed some benefits :(

-g.

--
blog.gmosx.com

Joshaven Potter

unread,
Aug 28, 2009, 11:48:10 AM8/28/09
to comm...@googlegroups.com
Regarding: "I worry about the app developer compositing the body via the use of string concatenation"

I completely disagree with designing anything with the goal of forcing a coding principal (good or bad) upon the developer.  Feel free to scream at the top of your lungs at stupid programming but don't design a API with controlling the developer in mind.

The middleware should provide methods for building the body.  The body could be an Array that is joined together when the middleware hands the message over to the server but I don't like the idea of a body being an array.  Arrays are for ordering like data not for chucking strings together or forcing paradigms on a developer.


I believe:
A response should be an object, its body should be a string, its header should be an object and its return code should be in integer.  The object's properties case should be standardized.  I vote for camelCase seeing that it is pretty standard in JavaScript, but I am fine with underscores or even running the words together.



Mike Wilson

unread,
Aug 28, 2009, 11:49:59 AM8/28/09
to comm...@googlegroups.com, narw...@googlegroups.com
George Moschovitis wrote:

However, as Tom said, perhaps we should optimize for the application developer and not the middleware developer. Personally, I think your suggestion to use arrays only for multipart/chunked responses is reasonable. But I am not sure about the complications. 
When saying "arrays" here, is that in a logical sense?
Or would serving a 1GB file imply loading the whole file into an array first?
 
Best regards
Mike Wilson

Kevin Dangoor

unread,
Aug 28, 2009, 11:54:07 AM8/28/09
to narw...@googlegroups.com, comm...@googlegroups.com
"arrays" means an object with a forEach method so you can certainly read the file bit-by-bit.

George Moschovitis

unread,
Aug 28, 2009, 12:05:34 PM8/28/09
to CommonJS

> When saying "arrays" here, is that in a logical sense?

Sorry for the confusion, I meant object that responds to .forEach();

-g.

Joshaven Potter

unread,
Aug 28, 2009, 1:28:19 PM8/28/09
to comm...@googlegroups.com
I don't understand any benefits of using an Array.

Each packet needs its own header, etc... If you need to cache the body for a bit to wait before sending the packet, that should be handled in the middleware.  

The body of an html message is not always logically divisible and even when it is logically divisible, it is not logically divisible in the transport layer.  As far as I can tell, the body should be a string... If you want to chuck data into a single html message, then chuck it at in the middleware where the logic behind building the response lays, then create the response object sending the completed body as a string.

It doesn't make since to me to have the body of a response respond to .forEach().  A body is a single thing... it should not be enumerable unless your suggesting that a single html message should have multiple bodies.

The body could have elements, (ie xml), but that is not something that I want my response object dealing with. Any other division of the body seems to me to be frivolous.

Also if there is an actual need to chunk the body into segments to fit in a MTU or something then the string can be split into appropriate sizes which really has nothing to do with a question of Array vs String.  I would assume that this type of chucking would be on the network layer anyway due to the MTU being relative to the network layer.

Tom Robinson

unread,
Aug 28, 2009, 3:00:01 PM8/28/09
to comm...@googlegroups.com
On Aug 28, 2009, at 8:48 AM, Joshaven Potter wrote:

Regarding: "I worry about the app developer compositing the body via the use of string concatenation"

I completely disagree with designing anything with the goal of forcing a coding principal (good or bad) upon the developer.  Feel free to scream at the top of your lungs at stupid programming but don't design a API with controlling the developer in mind.

The middleware should provide methods for building the body.  The body could be an Array that is joined together when the middleware hands the message over to the server but I don't like the idea of a body being an array.  Arrays are for ordering like data not for chucking strings together or forcing paradigms on a developer.


I believe:
A response should be an object, its body should be a string, its header should be an object and its return code should be in integer.  The object's properties case should be standardized.  I vote for camelCase seeing that it is pretty standard in JavaScript, but I am fine with underscores or even running the words together.

I agree there's no need to design JSGI to encourage certain practices, in particular since it's primary purpose isn't an application API, but rather an API for higher level frameworks.

However, there are other good reasons for specifying that the body responds to "forEach" rather than a String, which have been explained in various posts.

Joshaven Potter

unread,
Aug 28, 2009, 3:12:58 PM8/28/09
to comm...@googlegroups.com
I've been reading the various posts and I don't agree that there is any benefit to the body being an un-natural data type.

I don't think you should ever do response.body.push('...'); nor should you do response.body += '...';  you should do response.body = myContent.toString();

The [].push() should be done in a process of building a body (in the middleware) not in assigning the body to the a response.

Joshaven Potter

unread,
Aug 28, 2009, 4:48:49 PM8/28/09
to comm...@googlegroups.com
I don't really get the whole object the responds to .forEach() thing...

Anything can respond to .forEach() if it is a method of that object.

An Array is an established data type.  So is String.

You could modify String so that it responds to .forEach().

I don't think that the http message body should be enumerable because it is a single entity not a set of entities... if you need to iterate through the content of the body then iterate before you package it in a http message... don't .push() to the http message, generate the message and transmit it.

I'm all for generating a http body in chunks... partials etc are great but that is for building a page that has elements not for assembling a packet.  It makes perfect sense to iterate through a page's elements so they can be enumerable but the content of a http message is not enumerable nor should it be.


Joshaven Potter

unread,
Aug 28, 2009, 5:02:54 PM8/28/09
to comm...@googlegroups.com
The body content of the http reply packet should be specified by the packet header...

The Content-Type declares the bodies content type.... Here is a list of the common content types, note that Array is not there!


You will however find a lot of non-enumerable types like:
image/gif, text/css, text/javascript, text/plain, video/quicktime... etc

Why do you want to have your body be an Array when it needs to end up being encoded as text/plain or some such type?  If it is an Array then it will need to respond to toString() properly so it can be encoded.  I'd would rather send a string then define toString when the standard join() doesn't give me the proper whitespace etc... 

Tom Robinson

unread,
Aug 28, 2009, 5:06:51 PM8/28/09
to comm...@googlegroups.com
On Aug 28, 2009, at 10:28 AM, Joshaven Potter wrote:

I don't understand any benefits of using an Array.

Each packet needs its own header, etc... If you need to cache the body for a bit to wait before sending the packet, that should be handled in the middleware.  

The body of an html message is not always logically divisible and even when it is logically divisible, it is not logically divisible in the transport layer.  As far as I can tell, the body should be a string... If you want to chuck data into a single html message, then chuck it at in the middleware where the logic behind building the response lays, then create the response object sending the completed body as a string.

You are free to use an array with a string as it's only element. This is probably the most common case, but not the only case.

Part of the elegance of WSGI/Rack/JSGI is that middleware and applications have the same interface. An application wrapped in middleware is just another application.

If you want to return a string as the body you can:

a) implement forEach on String (which we used to do in Jack, but took out because we consider it a hack)

String.prototype.forEach = function() { return this; }

b) have a higher level framework, or even just a simple piece of middleware

function StringBodyMiddleware(app) {
    return function(env) {
        var result = app(env);
        if (typeof result.body === "string")
            result.body = [result.body];
        return result;
    }
}


It doesn't make since to me to have the body of a response respond to .forEach().  A body is a single thing... it should not be enumerable unless your suggesting that a single html message should have multiple bodies.

HTML and XML are two of many possible HTTP payloads.

For things like comet you do have multiple bodies.

For large files it's inefficient to buffer the entire file in memory before starting to send it.

Tom Robinson

unread,
Aug 28, 2009, 5:08:19 PM8/28/09
to comm...@googlegroups.com
note that Javascript String object is not there either!

Joshaven Potter

unread,
Aug 28, 2009, 5:47:49 PM8/28/09
to comm...@googlegroups.com
I must be thinking about something different then you are... because your arguments are not making any since to me.

Are you thinking:
1) That a packet is <= the MTU (~1500 bytes) and the body is a part of the packet?  And a response is a packet.  
2) Or are you thinking that a response is a whole document or data file and it will be broken down into response packets?

I cannot think of any reason to iterate through a packets body.  But if your thinking that a response is a web servers reply then it needs to be iterated through to break into packets.  I guess I am not sure I understand what your referring to as response, body, & header

Kris Kowal

unread,
Aug 28, 2009, 5:58:52 PM8/28/09
to comm...@googlegroups.com
On Fri, Aug 28, 2009 at 2:47 PM, Joshaven Potter<your...@gmail.com> wrote:
> I must be thinking about something different then you are... because your
> arguments are not making any since to me.

> Are you thinking:
> 1) That a packet is <= the MTU (~1500 bytes) and the body is a part of the
> packet?  And a response is a packet.
> 2) Or are you thinking that a response is a whole document or data file and
> it will be broken down into response packets?
> I cannot think of any reason to iterate through a packets body.  But if your
> thinking that a response is a web servers reply then it needs to be iterated
> through to break into packets.  I guess I am not sure I understand what your
> referring to as response, body, & header

Yeah. I'll attempt to explain.

We're not talking about packets. You're right that we are not
responsible for breaking a layer 7 HTTP response into layer 2 TCP
packets or even layer 1 Ethernet packets. It should however be
possible for us to break a layer 7 HTTP response into layer 7 HTTP
*chunks* of arbitrary length, any length we choose. This permits us
to construct a sort of bucket brigade of discrete pieces down the OSI
stack. For example, instead of reading a 4GB file into memory at one
time (usually impossible) and returning it as a single object in an
HTTP response, we can, at layer 7, break the task down into smaller,
manageable chunks. These chunks get passed down to lower layers,
getting sliced up and buffered in different sizes (the MTU applies
only at layer 2).

Usually it is sufficient to send a single chunk from layer 7.
However, we need to be *able* to send multiple chunks, in order.

That is to say, the concept of streaming exists at *every* layer of
the OSI stack.

This is probably not the best venue for a discussion of how the loose
abstraction of OSI works. Perhaps you could join us on IRC, #commonjs
on freenode, or address those interested with an off list discussion.

Kris Kowal

Tim Schaub

unread,
Aug 28, 2009, 6:03:34 PM8/28/09
to comm...@googlegroups.com
Joshaven Potter wrote:
> I must be thinking about something different then you are... because
> your arguments are not making any since to me.
>
> Are you thinking:
> 1) That a packet is <= the MTU (~1500 bytes) and the body is a part of
> the packet? And a response is a packet.

http://en.wikipedia.org/wiki/IPv4 perhaps?

> 2) Or are you thinking that a response is a whole document or data file
> and it will be broken down into response packets?

http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol maybe?

JSGI is about HTTP.

Maybe I don't get what you're asking and this is irrelevant.

Ash Berlin

unread,
Aug 28, 2009, 6:19:36 PM8/28/09
to comm...@googlegroups.com
On 28 Aug 2009, at 16:48, Joshaven Potter wrote:

I believe:
A response should be an object, its body should be a string,

I would just like to clarify this point: the response to an HTTP request should not be a *string* (a string is a sequence of characters/code points), its a sequence of bytes + a content-type header which specifies the encoding.

(No comments on how JSGI handles this, I'm fine with the way it behaves now, just felt that this was an important point to make)

Tom Robinson

unread,
Aug 28, 2009, 6:29:52 PM8/28/09
to comm...@googlegroups.com
The response's forEach method yields objects which must have a toByteString() method.

Strings have toByteString. But I'm not sure how to handle picking the encoding to use. Right now Jack uses the default (UTF-8?) in the conversion. Perhaps it should try extracting the encoding from the Content-Type, and fall back to the default if it's unsuccessful?

Tom Robinson

unread,
Aug 28, 2009, 6:44:19 PM8/28/09
to comm...@googlegroups.com
Well, http://rack.rubyforge.org/ and http://wsgi.org/ also disagree with you. I suggest you read up on those.

Joshaven Potter

unread,
Aug 28, 2009, 7:51:27 PM8/28/09
to comm...@googlegroups.com
It appears that I have been confusing things tcp and http.  I am sorry. 

Daniel Friesen

unread,
Aug 28, 2009, 10:02:06 PM8/28/09
to comm...@googlegroups.com
.next is the same as .forEach as what I'm talking about goes, they're
both called by the jsgi layer in a blocking way.

This one is probably a tough one to explain. Most apps either just build
a html body and return, or they simply take chunks from an existing
stream (a file or whatnot). Both fit fine in the .forEach logic.

While the more common issue, they aren't the whole picture. Long polling
Comet requests, the new x-dom-event-stream, how do we handle a case
where rather than building data, then returning, or puling data out of
another stream, instead we ourselves are waiting for something else to
finish, or rather waiting for something to even happen.
With .forEach as I understand it, to do that kind of streaming we would
have to block. Block and wait for something to happen, then return a chunk.

I guess I'm trying to say. Why are we using a method that forces
blocking, instead of a system that lets us write chunks as they come in,
and lets the jsgi layer use those chunks if available, but if not
available allows it to go and handle other pending business.

function app(env) {
var bodyStream = new DomEventStreamHelper(); // just a protocol helper
that takes an object and writes it in the proper format
var calcWorker = new Worker('calc.js');
calcWorker.onmessage = function(msg) {
if ( msg == "finished" ) bodyStream.close();
else bodyStream.writeEvent({ event: "calculated", data: msg });
};
// messages are queued, these get handled one by one.
myworker.postMessage(complexRequestA);
myworker.postMessage(complexRequestB);
myworker.postMessage(complexRequestC);
myworker.postMessage(complexRequestD);
myworker.postMessage("close");
return { status: 200, headers: { 'Content-Type': 'x-dom-event-stream' },
bodyStream };
}

In this case, we would probably have to put a forEach on
DomEventStreamHelper which would poll for messages basically block and
wait for the next message then return it.
If we already have a stream which we can ask "do you have any data ready
for me to send?" why can we only block? Why can't we ask "do you have
the data?" get a response "nope, not yet." and instead of waiting, go
see if someone else has data they need to send?

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Kris Kowal wrote:
> On Thu, Aug 27, 2009 at 9:33 AM, Daniel
> Friesen<nadir.s...@gmail.com> wrote:
>
>> The one thing still racking my head is body. I understand how using
>> .forEach supports the chunking of data needed for streaming, but I don't
>> understand how that would work itself on a stream. How can .forEach be
>> handled async while waiting for the app to write data?
>>
>
> This is the reason why Python uses "next" as its iterator primitive
> method, not "forEach". "next" is a function that returns the next
> value of an iteration or throws a StopIteration exception. The
> "forEach" construct in turn is built on top of "next" and not meant to
> be overridden:
>
> forEach = function (block, context) {
> while (true) {
> try {
> block.call(context, this.next());
> } catch (exception) {
> if (exception instanceof StopIteration) {
> break;
> } else {
> throw exception;
> }
> }
> }
> };
>
> Which could be extended at a price to do SkipIteration, for both break
> and continue analogs.
>
> Kris Kowal
>
> >
>

Kevin Dangoor

unread,
Aug 28, 2009, 11:34:49 PM8/28/09
to comm...@googlegroups.com
On Fri, Aug 28, 2009 at 7:51 PM, Joshaven Potter <your...@gmail.com> wrote:
It appears that I have been confusing things tcp and http.  I am sorry. 


Ahh, missed this message before I sent that reply on the other thread. Glad that's cleared up :)

Alexandre.Morgaut

unread,
Aug 31, 2009, 5:35:14 AM8/31/09
to CommonJS
About recurrent HTTP header field names :

HTTP specification says that header fields can be specified more than
once if these multiple occurence can be joined with a coma separator

This rule is easily applicable for the "Accept" request header field
or the "Allow" response header field.

Thats mean :
- several "Accept" header fields can be joined in the env.HTTP_ACCEPT
JSGI property
- several "Allow" header fields can be joined in a single
response.header['Content-Type'] property


Specific cases :


This rule can't apply to header fields like 'Content-Type' which only
accept a single value.
The solution I most frequently saw was that new value set to an
already existing header field replaced the previous one (which is
considered as a default value)


This rule is also difficult to apply to the "Warning" header field.
The HTTP 1.1 specification says that an HTTP message can have several
"Warning" header fields
It looks like that :

Warning: 110 mydomain.org:8080 Response is staled "Mon, 31 Aug 2009
09:29:15 GMT"

As you can see, when Warning header field specify the date, a comma
appear,
hopefully the date is enclosed with double-quotes

Alexandre.

Daniel Friesen

unread,
Aug 31, 2009, 12:58:57 PM8/31/09
to comm...@googlegroups.com
"consisting of lines (for multiple header values) separated by “\n”"

ie:
GET / HTTP/1.1
Cookie: a=foo
Cookie: b=bar
Cookie: c=baz

Becomes:
env.HTTP_ACCEPT = "a=foo\nb=bar\nc=baz";

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Alexandre.Morgaut

unread,
Aug 31, 2009, 1:35:36 PM8/31/09
to CommonJS


On Aug 31, 6:58 pm, Daniel Friesen <nadir.seen.f...@gmail.com> wrote:
> "consisting of lines (for multiple header values) separated by “\n”"
>
> ie:
> GET / HTTP/1.1
> Cookie: a=foo
> Cookie: b=bar
> Cookie: c=baz
>
> Becomes:
> env.HTTP_ACCEPT = "a=foo\nb=bar\nc=baz";
>

"Multiple message-header fields with the same field-name MAY be
present in a message if and only if the entire field-value for that
header field is defined as a comma-separated list [i.e., #(values)].
It MUST be possible to combine the multiple header fields into one
"field-name: field-value" pair, without changing the semantics of the
message, by appending each subsequent field-value to the first, each
separated by a comma. The order in which header fields with the same
field-name are received is therefore significant to the interpretation
of the combined field value, and thus a proxy MUST NOT change the
order of these field values when a message is forwarded. "

http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2

Well, ideally, it would be better if it became:

env.HTTP_COOKIE = "a=foo,b=bar,c=baz";

;-)

But I didn't talk about the cookie's specificity, and the '\n' hack
might help (even if I prefer the use of arrays which are easier to use
and more scalable)

Cookie and Set-Cookie header field most often don't respect this HTTP
1.1 rule

Cookie separates, instead, multiple values with semicolons

ex:
Cookie: a=foo; b=bar; c=baz

While Set-Cookie use semicolon to seperate cookie's parameters and
don't enclose the expire value into double-quotes

ex:
Set-Cookie: name=Mike; expires=Sat, 03 May 2025 17:44:22 GMT

That's why node.js prefer to set HTTP header field values into a
JavaScript array

Joshaven Potter

unread,
Aug 31, 2009, 2:45:36 PM8/31/09
to comm...@googlegroups.com
I prefer using an Array!
Then the array can be joined with .join(','); to meet the http message requirements of controlled order and comma separated.


Daniel Friesen

unread,
Aug 31, 2009, 2:53:37 PM8/31/09
to comm...@googlegroups.com
The ugly part about using an array is then you're going to have to all
the time use code like:

env.HTTP_CONTENT_TYPE[0]

Converting \n into , inside of a string is no harder than joining an
array together (heck, for the micro-optimization folks it's probably
more efficient).

env.HTTP_ACCEPT.replace("\n", ','); instead of env.HTTP_ACCEPT.join(',');

(And no, we will not be adding logic into jsgi where we maintain lists
of what headers to combine and what headers not to... That's request
object gruntwork.)

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Joshaven Potter wrote:
> I prefer using an Array!
> Then the array can be joined with .join(','); to meet the http message
> requirements of controlled order and comma separated.
>
>
>
> On Mon, Aug 31, 2009 at 1:35 PM, Alexandre.Morgaut
> <mor...@hotmail.com <mailto:mor...@hotmail.com>> wrote:
>
>
>
>
> On Aug 31, 6:58 pm, Daniel Friesen <nadir.seen.f...@gmail.com

Joshaven Potter

unread,
Aug 31, 2009, 3:47:59 PM8/31/09
to comm...@googlegroups.com
Maybe I assumed too much.  I would like to see an object containing an array or a string.

So the message would have a header object and each of that headers properties should support toString().

An example is worth a lot of words:


var responce = {
  headers:{
    toString: function(){
      var a = [], k, self = responce.headers;
      for ( k in self )
        if ( self.hasOwnProperty(k) && k != 'toString' ) a.push(k + '="' + self[k].toString() + '"');
      return a.join('\n');
    }
  }
};

var h = responce.headers;
h.http_content_type = 'text/html';
h.http_accept = [];
h.http_accept.push('a=foo');
h.http_accept.push('b=bar');

alert(responce.headers.toString());

http_accept="a=foo,b=bar"
http_content_type="text/html"



This would allow the header to be anything that will respond to the method toString() which must join any ordered data with a comma.  If you need to use a custom datatype then just write an appropriate toString() method.

Daniel Friesen

unread,
Aug 31, 2009, 5:45:53 PM8/31/09
to comm...@googlegroups.com
Except this is a protocol, NOT a library.

If you want those kind of stuff, just use a Request class.
> <mailto:mor...@hotmail.com <mailto:mor...@hotmail.com>>> wrote:
> >
> >
> >
> >
> > On Aug 31, 6:58 pm, Daniel Friesen
> <nadir.seen.f...@gmail.com <mailto:nadir.seen.f...@gmail.com>
> > <mailto:nadir.seen.f...@gmail.com

Joshaven Potter

unread,
Aug 31, 2009, 6:16:56 PM8/31/09
to comm...@googlegroups.com
I thought the protocol was already well defined.  The http headers are supposed to be comma seporated.  And, like you said, I was talking about how the library should implement the standards. 

Daniel Friesen

unread,
Aug 31, 2009, 6:42:21 PM8/31/09
to comm...@googlegroups.com
There is no "the Library", and no standards for one.

We have the JSGI protocol which is used for handlers to talk to
apps/middleware. Which is basically summed up as an app function, env,
and the returned response object literal.

JSGI and this group has nothing to do with anything beyond that. That
layer is for Jack or whatever library with Request/Response classes
people feel like creating. Heck, I'm probably going to have a
MonkeyScript specific one myself to nicely integrate with my URL class.
> <mailto:nadir.s...@gmail.com

Dean Landolt

unread,
Aug 31, 2009, 7:09:44 PM8/31/09
to comm...@googlegroups.com
On Mon, Aug 31, 2009 at 6:42 PM, Daniel Friesen <nadir.s...@gmail.com> wrote:

There is no "the Library", and no standards for one.

We have the JSGI protocol which is used for handlers to talk to
apps/middleware. Which is basically summed up as an app function, env,
and the returned response object literal.

Exactly. So back to the issue at hand, how should jsgi specify concatenating multiple headers? As Alexandre pointed out, HTTP sorta does this for us (specifying the ",") but there are some problems associated with it. Daniel suggested concatenating with \n, which may not be the _least surprising_ alternative, but should be relatively safe (though headers can be extended over multiple lines when preceding the next line with a space or tab, so perhaps not completely safe).

I can't find the list of reserved chars for headers, but something in it might make a good split char. Either way, it should be spec'd so that any library implementing a Request obj can do something sane. In the interest of sticking with the letter of the spec, I'm +1 on comma concatenation. Then the library authors sort it out :)

Daniel Friesen

unread,
Aug 31, 2009, 7:40:37 PM8/31/09
to comm...@googlegroups.com
It's not my suggestion, the "\n" is what is already in the jsgi spec.
Have multiple headers by the same name? A multiline header. Done.

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Dean Landolt wrote:
>
>
> On Mon, Aug 31, 2009 at 6:42 PM, Daniel Friesen
> <nadir.s...@gmail.com <mailto:nadir.s...@gmail.com>> wrote:
>
>
> There is no "the Library", and no standards for one.
>
> We have the JSGI protocol which is used for handlers to talk to
> apps/middleware. Which is basically summed up as an app function, env,
> and the returned response object literal.
>
>
> Exactly. So back to the issue at hand, how should jsgi specify
> concatenating multiple headers? As Alexandre pointed out, HTTP sorta
> does this for us (specifying the ",") but there are some problems
> associated with it. Daniel suggested concatenating with \n, which may
> not be the _least surprising_ alternative, but should be relatively
> safe (though headers can be extended over multiple lines when
> preceding the next line with a space or tab, so perhaps not
> /completely /safe).

Tom Robinson

unread,
Aug 31, 2009, 8:06:13 PM8/31/09
to comm...@googlegroups.com
It's also what Rack does, FWIW.

WSGI uses an array of tuples for response headers, and the "env"
object containing HTTP_ properties for request headers. I'm not sure
how they (or we) handle multiple *request* headers values.

Dean Landolt

unread,
Aug 31, 2009, 8:38:46 PM8/31/09
to comm...@googlegroups.com
On Mon, Aug 31, 2009 at 8:06 PM, Tom Robinson <tlrob...@gmail.com> wrote:

It's also what Rack does, FWIW.

WSGI uses an array of tuples for response headers, and the "env"
object containing HTTP_ properties for request headers. I'm not sure
how they (or we) handle multiple *request* headers values.

On Aug 31, 2009, at 4:40 PM, Daniel Friesen wrote:

>
> It's not my suggestion, the "\n" is what is already in the jsgi spec.
> Have multiple headers by the same name? A multiline header. Done.

Fair enough -- I thought I'd combed through the jsgi spec pretty carefully but I was never really concerned about this until it was brought up. But when you say "Have multiple headers by the same name? A multiline header. Done." -- what do you mean? Can you differentiate the two (assuming the leading SP and HT are stripped per the HTTP spec)? Do you even need to?

I can't figure out what WSGI does -- it looks like it defers to the SERVER_PROTOCOL for this (which would imply comma-separated for HTTP/1.1). It's nice to hear that Rack does it this way -- it's been around long enough to have exposed any problems with this approach, so what the hell -- it ought to work. I've certainly never seen a request that would break it.
Reply all
Reply to author
Forward
0 new messages