Webserver module, Transfer-Encoding chunked and Content-Length

[Ivan De Marino]

Hi all,

it seems like we have a non-standard-compliant Webserver module at the moment.

The issue is with "Content-Length" header and Response object: we don't send this header appropriately.

We implement a "faulty" version of "Transfer-Encoding: chunked", and this is starting to get in the way (of GhostDriver :) ).

So, I'm going to fix this. But before I do, I want to ask if you guys would like a more "node-like" API for this: currently we don't really implement the same methods as node (see  the "end()" method for example), and nor the same behaviour.

Details about the Node HTTP Response object: http://nodejs.org/api/http.html#http_class_http_serverresponse

Personally I'd like to have the same behaviour as Node:

- first call to "write()" is buffered

- IF followed by "end", it calculates the total Content-Length, writes the body and closes the connection

- IF followed by a "write()", it sends the header Transfer-Encoding: chunked and than writes the data in the appropriate form (see http://en.wikipedia.org/wiki/Chunked_transfer_encoding for details)

I'd love to hear your opinion.

Ivan

[Ariya Hidayat]

In principle, I agree with the proposed changes to make us behaves
more like Node.js.

If compatibility is important (and I think it is), we can have a flag
to switch back to the old behaviour, maybe something like unbuffered.


Thanks!

Regards,


--
Ariya Hidayat, http://ariya.ofilabs.com
http://twitter.com/ariyahidayat

[Ivan De Marino]

Would be interesting to know how many people are currently using the Webserver module.

[Milian Wolff]

Hey guys,

as the one who contributed most of the code, let me first say that I'm "sorry" to not follow the Node.JS API from the start. Maybe in the future, things like that should be made more clear for new contributions. Only afterwards I got to know the CommonJS effort e.g. Just saying.

Now regarding the existing API, I know that at least our customer is already heavily (and happily) using the current API. So I hope backwards compatibility will be upheld, or at least a *simple* (automated?) porting mechanism proposed.

Lastly, regarding the current behavior: I wrote the WebServer (and it's API) assuming - well - nothing from the users. It should let the user do everything he wants to. This shows in e.g. the lack of API simplifications like the one for a buffered write. I see though how this is probably the most often required usecase hence making that less painful to use a good idea.

So from my side: Please go ahead and improve the code, but try to keep backwards compatibility.

Thanks!

[Ivan De Marino]

No need to apologize: sometimes I can sound very "grumpy" in writing, but I don't actually mean to.

Your perspective on this issue is valid: users are left with the "raw" functionality - it becomes their responsibility to make sure they implement the proper HTTP protocol.

The issues for me boils down to the fact that the User can call "write" more than once, and we won't take care at all of "Content-Length" and "Transfer-Encoding".

For a "simple" user, this will make us look like "broken" when they try to access our server with "picky HTTP clients" (i.e. clients that follow the HTTP standard by-the-book).

What do you guys suggest to do?

I propose Node because it offers an example solution: we don't need to.

Regarding retro-compatibility, we could:

- have this new behaviour disabled by a flag

- add "res.end()" next to "res.close()"

I think this should be enough to ensure we don't brake stuff.

--

Ivan De Marino

Coder, Technologist, Cook, Italian

blog.ivandemarino.me | www.linkedin.com/in/ivandemarino | twitter.com/detronizator

[Trevor]

I'll be relying on the Webserver module heavily as a means of controlling and triggering rendering for svg charts.