Hijack for comet?

[Robert Smart]

I presume Hijack in http package is to allow comet style server
behaviour. Has anybody tried it and care to share.

[Ambel]

Not necessarily.

Comet is typically and HTTP layer connection in which the client
forces the server to maintain an open connection. Go's Hijack
function, however, operates at a level lower than the HTTP protocol.
Because HTTP cannot be assumed to maintain state, a browser should not
be expected to utilize this functionality.

If you code your own client, however, only then can you expect your
client to manage and close a connection itself. This may be
applicable in web services circumstances when persistent XML
communication over HTTP might be necessary.

~A

On Nov 26, 5:58 pm, Robert Smart <robert.kenneth.sm...@gmail.com>
wrote:

[Mike D]

I have written a comet-like server. It is very basic and only uses
event-source and Mozilla's mixed/x-multipart-replace method. I am not
storing the events on the server so it would need to be modified to
use a hacky long-polling technique.

You can have a look at my code here, it works for my needs. It needs
some work to handle reconnecting clients.

http://go.anykeysoftware.co.uk/evtserver-0.1.tar.gz

As you can see, I only need the Hijack command so that I can close
connections when clients disconnect. One go routine is launched for
each connected client which just tries to read from the connection.

I have tested it with WebKit nightly and Mozilla and both of them can
hold an open connection for hours. I don't think it is necessary to
keep the connection alive manually.

It has a few comments but if you have any questions (or patches),
please let me know. I am using 6g so if you use 8g then modify the
Makefile.

Events are sent in the format name:data to port 9057 (UDP by default)
and clients connect to TCP 9058 with the url /listen/type. The first
connection by a client will give a client_id which they must then use
to register for events by making a get request to /register/....

Regards
Mike

[Ben Tilly]

On Sat, Nov 28, 2009 at 5:44 AM, Mike D <mikedra...@gmail.com> wrote:
> I have written a comet-like server.  It is very basic and only uses
> event-source and Mozilla's mixed/x-multipart-replace method.  I am not
> storing the events on the server so it would need to be modified to
> use a hacky long-polling technique.
>
> You can have a look at my code here, it works for my needs.  It needs
> some work to handle reconnecting clients.
>
> http://go.anykeysoftware.co.uk/evtserver-0.1.tar.gz
>
> As you can see, I only need the Hijack command so that I can close
> connections when clients disconnect.  One go routine is launched for
> each connected client which just tries to read from the connection.
>
> I have tested it with WebKit nightly and Mozilla and both of them can
> hold an open connection for hours.  I don't think it is necessary to
> keep the connection alive manually.

[...]

Sorry, but it is necessary to keep a connection alive manually in the
real world.

Many corporate firewalls maintain state per connection, and so will
automatically close any connection that has had no traffic after a
certain time. Therefore you need to periodically ping the connection
so that the firewall doesn't close the connection on its end. The
timeout time depends on the corporate firewall.

Cheers,
Ben

[Mike Dransfield]

Thanks for the info.  The problem with always keeping the connection alive is that you will constantly wake up mobile clients which will reduce their battery life.

Event source will automatically reconnect when disconnected so I do not think it is really necessary to add something to forcefully keep connections alive.

If it were needed then it could be added on a client by client basis, just by sending a ping event at a certain interval.  Clients could also request the interval to be whatever they need, it could even calculate automatically based on the average time that connections last.

Regards

Mike

[Russ Cox]

Hijack is for protocols that start out as HTTP but then
hijack the connection and start executing a completely
different protocol. The native Go rpc does this--clients
connect using a CONNECT request and the handler
takes over the connection--and there is a websocket
implementation in progress that also does this.

If you're calling Hijack, you're not using HTTP anymore.

If you just want to dribble response data back to the
client slowly, you don't need Hijack: just write to the
connection and call Flush as appropriate.

Russ

[Mike Dransfield]

Writing the connection was not a problem, I was having problems disconnecting the connections after clients disconnected.  The server side connections were being left with the state CLOSE_WAIT.

To test if they were disconnected I was reading from the connection, and closing the connection when EOF was read.

Is there a better way to do this without using Hijack()?  I needed Hijack so that I could access rwc and buff manually.

Regards
Mike

[Russ Cox]

> Writing the connection was not a problem, I was having problems
> disconnecting the connections after clients disconnected.  The server side
> connections were being left with the state CLOSE_WAIT.

If that's true, then there's a bug in the http server.
Better to find the bug than hack around it.
The source is in $GOROOT/src/pkg/http/server.go. ;-)

Russ

[Mike Dransfield]

I am not sure it is a bug in the http package.

Normally it works like this.

Read request -> Write headers -> Write body (Handler function exits) -> Connection is closed.

But in my server it works like this.

Read request -> Write headers -> Write body (Handler is blocking waiting for events) -> Write headers -> Write body.

It would be possible to do it without using Hijack but I would need to check that all the clients are connected when the event is sent out.  This is not going to scale very well and clients are likely to be kept open longer than necessary (which will cause problems on reconnect).  The Hijack method actually seems very clean compared to the alternatives.

If I were to fix the http package then all it would do is add a go routine to check for EOF and then close connections, it would only be needed in the very rare circumstance that a handler did not exit and the client closed.  Under normal circumstances my extra code would never run just leading to overhead for 99% of users.

I haven't checked the HTTP protocol in much detail but I don't think it allows headers after body and even if it did, what is sent after the response headers is just raw data so is not really HTTP anymore.

Regards
Mike