Roadmap

138 views
Skip to first unread message

Benjamin van der Veen

unread,
Aug 7, 2011, 5:49:23 PM8/7/11
to kayak...@googlegroups.com
Hey all,

As you know, I've been working hard to make the OWIN stack a reality. Last week, I released Gate 0.1.0 and posted some example code demonstrating how to run Nancy on Kayak via Gate—a tremendous milestone. Now, I'm taking some time to evaluate how to these projects can each forward. With that in mind, I put together a high-level roadmap for Kayak.

0.7.2
 - HTTP client implementation
 
0.8.0
 - optional libuv network driver (this might change some APIs, hence version bump)
  - filesystem access/events
  - timers
  - signal events
  - UDP/datagram support (if available in libuv)
 - Async DNS API - introduced to wrap the libuv DNS capabilities, will be implemented in terms of System.Net.Dns for the non-uv implementation

0.8.1
- SSL/TLS/HTTPS

I would feedback on this roadmap. Are there features you would like to see sooner or later on the roadmap? Are there important features I have missed? What else would you like to see?

Cheers,
Benjamin

panesofglass

unread,
Aug 8, 2011, 11:35:28 AM8/8/11
to kayak...@googlegroups.com
Any plans to put ASP.NET on Kayak? Maybe start with IHttpHandler and IHttpModule? I would bet that getting the basic foundation in place would motivate someone to finish the job for you. Haven't gotten that far myself in my own work, but it would be a cool milestone and might allow some people to start using it in their work environments.

Benjamin van der Veen

unread,
Aug 8, 2011, 12:20:34 PM8/8/11
to kayak...@googlegroups.com


On Mon, Aug 8, 2011 at 8:35 AM, panesofglass <ryan....@panesofglass.org> wrote:
Any plans to put ASP.NET on Kayak? Maybe start with IHttpHandler and IHttpModule? I would bet that getting the basic foundation in place would motivate someone to finish the job for you. Haven't gotten that far myself in my own work, but it would be a cool milestone and might allow some people to start using it in their work environments.

It be definitely be cool to see a server-agnostic implementation of ASP.NET, but such a project would be squarely outside the scope of Kayak itself. I haven't taken a look at the Mono implementation, but it might be a good place to start.

daniel carli

unread,
Aug 8, 2011, 1:27:05 PM8/8/11
to kayak...@googlegroups.com
Maybe just use something of razor template to better serve html content.

2011/8/8 Benjamin van der Veen <b...@bvanderveen.com>

Paul Batum

unread,
Aug 10, 2011, 3:03:21 AM8/10/11
to kayak...@googlegroups.com
Can you elaborate on what you plan to build for the HTTP client?

Benjamin van der Veen

unread,
Aug 10, 2011, 12:59:10 PM8/10/11
to kayak...@googlegroups.com


On Aug 8, 2011, at 10:27, daniel carli <dansa...@gmail.com> wrote:

Maybe just use something of razor template to better serve html content.

This is also out-of-scope for Kayak itself as a IO library and web server—definitely going to leave that up to a web framework. 

Benjamin van der Veen

unread,
Aug 10, 2011, 1:06:39 PM8/10/11
to kayak...@googlegroups.com

On Aug 10, 2011, at 0:03, Paul Batum <paul....@gmail.com> wrote:

> Can you elaborate on what you plan to build for the HTTP client?

The HTTP client will reuse the IHttpRequestDelegate and IHttpResponseDelegate interfaces, but the user implements IHttpResponseDelegate and calls into IHttpRequestDelegate—the opposite of the HTTP server interface.

The client implementation will internally /transparently handle DNS resolution, manage a connection pool, and deal with request pipelining/keep-alive.

You will be able to test client/server code directly together without any sockets or IPC.

Does that give you a better picture? Let me know if I can elaborate or if I'm missing anything. :)

Massimiliano Mantione

unread,
Aug 10, 2011, 4:22:46 PM8/10/11
to kayak...@googlegroups.com
On Wed, Aug 10, 2011 at 7:06 PM, Benjamin van der Veen
<b...@bvanderveen.com> wrote:
> The client implementation will internally /transparently handle DNS resolution, manage a connection pool, and deal with request pipelining/keep-alive.

This is interesting, but I wonder if each of those things should be
"optional", in the sense that there should be a way to use the client
library so that it does not attempt to provide those services.

Skipping DNS resolution should be straightforward if the user provides
an IP endpoint, but pooling and keep-alive... my gut feeling is that
there should be a way to disable them.
I would even say that they should be implemented "later", after a
stable (and already useful) HTTP client release.

This in the "minimalist and barebone" spirit of Kayak, which for me is
a framework that strives to be a good foundation that integrates well
with whatever the user already has...

Or maybe I am not getting what "connection pooling" means in this context.
I think of a connection as an open and connected socket, and I don't
see how a connected socket can be recycled in a pool.

> You will be able to test client/server code directly together without any sockets or IPC.

Now *this* is a really cute idea!

Cheers,
Massi

Benjamin van der Veen

unread,
Aug 10, 2011, 4:51:15 PM8/10/11
to kayak...@googlegroups.com
On Wed, Aug 10, 2011 at 1:22 PM, Massimiliano Mantione <massimilia...@gmail.com> wrote:
On Wed, Aug 10, 2011 at 7:06 PM, Benjamin van der Veen
<b...@bvanderveen.com> wrote:
> The client implementation will internally /transparently handle DNS resolution, manage a connection pool, and deal with request pipelining/keep-alive.

This is interesting, but I wonder if each of those things should be
"optional", in the sense that there should be a way to use the client
library so that it does not attempt to provide those services.

Skipping DNS resolution should be straightforward if the user provides
an IP endpoint, but pooling and keep-alive... my gut feeling is that
there should be a way to disable them.

Good point, the keep-alive behavior should be optional. Keep-Alive is only relevant to HTTP/1.1, so perhaps it is off by default and only enabled if the user specifies an HTTP/1.1 request without the Connection: close header.
 
I would even say that they should be implemented "later", after a
stable (and already useful) HTTP client release.

Also a good point. The HTTP client implementation should be split into two phases. HTTP/1.0 first (with no keep-alive/connection pooling) and then HTTP/1.1 later.


Or maybe I am not getting what "connection pooling" means in this context.
I think of a connection as an open and connected socket, and I don't
see how a connected socket can be recycled in a pool.

Yeah, that's what I mean. For example, if the user makes two requests to the same server on one iteration of the event loop, those requests could happen in series over one TCP connection. If the user makes another request to the same server later, it would be nice to reuse that already-open connection if possible. I haven't fully sussed out the details of the behavior, but it should be possible to come up with a decent algorithm for re-using connections.
 
Thanks Massi!

Massimiliano Mantione

unread,
Aug 10, 2011, 6:11:41 PM8/10/11
to kayak...@googlegroups.com
On Wed, Aug 10, 2011 at 10:51 PM, Benjamin van der Veen
<b...@bvanderveen.com> wrote:
>[...]

>> Or maybe I am not getting what "connection pooling" means in this context.
>> I think of a connection as an open and connected socket, and I don't
>> see how a connected socket can be recycled in a pool.
>
> Yeah, that's what I mean. For example, if the user makes two requests to the
> same server on one iteration of the event loop, those requests could happen
> in series over one TCP connection. If the user makes another request to the
> same server later, it would be nice to reuse that already-open connection if
> possible.
> [...]

Ok, now I think I get it: pooling is HTTP 1.1 specific, it happens
when the connection stays open and can be reused for other requests.

Maybe you could do like this... 1st of all model a sort of "HTTP end
point", which is an object that "represents the server".
You use it to make requests passing a response handler and the handler
will be called appropriately when the server answers (the usual Kayak
idiom, but as you said in reverse).

Creating an "HTTP end point" can involve DNS requests, but does not
involve creating sockets.

Then, when a request is issued, the end point creates a socket as
needed, eventually reusing it for HTTP 1.1.

Of course (and this is what I was wondering about thinking of
"pooling") the socket can be reused only inside the *same* HTTP end
point.
I mean, there is no global "client side connection pool" (in contrast
to the usual pooling, like in thread pools or for "db connection
pools"), each HTTP end point is a "mini-pool" of its own.

Then an end point should expose an API to control how keep-alive is
handled and when an 1.1 connection will be closed (maybe with a "close
now" option, even without issuing a request).
The end point will of course create the socket on the fly for each
request that is issued when no socket is active.

Probably the "final" issue to fix would be how to handle multiple
requests: should the end point open more connections (sockets), or
should it serialize the requests?
Likely, also in this case it should expose an API to control its behavior.
Maybe the sensible default would be opening a new socket for 1.0
requests and queuing them on the existing socket for 1.1 requests,
unless the user asks otherwise.

Just my two cents "thinking aloud" around midnight :-)

Keep up the good work!
Massi

Paul Batum

unread,
Aug 11, 2011, 4:20:34 PM8/11/11
to kayak...@googlegroups.com
I like the idea of being able to do client<->server tests without any sockets or IPC, I'm just not sure if its worth it for you to spend effort on a proper HTTP client (one that does DNS resolution, connection pooling, etc). Have you looked at the new HttpClient that we (Microsoft) have on Codeplex and NuGet (http://nuget.org/List/Packages/HttpClient)?

Benjamin van der Veen

unread,
Aug 11, 2011, 5:13:40 PM8/11/11
to kayak...@googlegroups.com


On Thu, Aug 11, 2011 at 1:20 PM, Paul Batum <paul....@gmail.com> wrote:
I like the idea of being able to do client<->server tests without any sockets or IPC, I'm just not sure if its worth it for you to spend effort on a proper HTTP client (one that does DNS resolution, connection pooling, etc). Have you looked at the new HttpClient that we (Microsoft) have on Codeplex and NuGet (http://nuget.org/List/Packages/HttpClient)?

Hadn't heard of HttpClient. Looks like a nice implementation!

The idea here though is to have something that integrates well with Kayak's event loop and networking layer, which could eventually be implemented in terms of libuv. HttpClient seems to be implemented in terms of HttpWebRequest, which in turn is implemented in terms of System.Net.Sockets.Socket—ultimately the goal is to make that networking layer swappable/pluggable/mockable and keep the protocol implementation agnostic of it.

That said, a user could totally use HttpClient in the context of Kayak, they'd just have to properly marshal the callbacks back to Kayak's event loop.
Reply all
Reply to author
Forward
0 new messages