Jeremy Shaw <jer...@n-heptane.com> writes: > The biggest 'issue' with a port to the snap backend right now is that > the part happstack wants to use is all mixed up with the rest of the > code. For example, Snap.Types has the Request and Response types, > which happstack would use, but also the Snap monad. And snap-core > contains the types we want, but also things like file serving > utilities. We can, of course, ignore the things we don't want.. but > when those things also add even more cabal dependencies, that is > annoying. Happstack already has too many dependencies.
I was thinking instead of a package "snapstack", which exports a function:
>> The biggest 'issue' with a port to the snap backend right now is that >> the part happstack wants to use is all mixed up with the rest of the >> code. For example, Snap.Types has the Request and Response types, >> which happstack would use, but also the Snap monad. And snap-core >> contains the types we want, but also things like file serving >> utilities. We can, of course, ignore the things we don't want.. but >> when those things also add even more cabal dependencies, that is >> annoying. Happstack already has too many dependencies.
> I was thinking instead of a package "snapstack", which exports a > function:
> Basically: convert the request/response types and run the happstack > monad. The conversion would be a little bit of fixed overhead.
But, if we are converting the existing happstack Response type into a snap Response, then we wouldn't be able to use iteratees?
Also, it just complicates the code even more than it already is. Adds even more monads!
What is the advantage of doing things that way over just using the snap Request/Response types natively?
It would be useful if we simply want to use some existing happstack code in a snap application. But the goal is to 'permanently' improve the the happstack server backend and get rid of the existing lazy I/O code altogether..
>> HTTP 1.1, they *didn't* request a close, and the content we are >> sending includes a content-length.
> Why don't you do chunked transfer encoding on http/1.1?
I assume you are asking why we don't use chunked transfer encoding when we don't know the length so that we can support keep-alive even in that case?
I think it's because all the people interested in writing low-level http server code are working on their own servers instead of contributing to happstack :p
It's not too late though, you can still submit a patch ;)
Jeremy Shaw <jer...@n-heptane.com> writes: > But, if we are converting the existing happstack Response type into a > snap Response, then we wouldn't be able to use iteratees?
Yeah it'd strictly be a compatibility layer.
> Also, it just complicates the code even more than it already is. Adds > even more monads!
> What is the advantage of doing things that way over just using the > snap Request/Response types natively?
You get to run existing happstack code without modification, that's it.
> It would be useful if we simply want to use some existing happstack > code in a snap application. But the goal is to 'permanently' improve > the the happstack server backend and get rid of the existing lazy I/O > code altogether..
If you would want to move ServerPartT closer to Snap then you may as well just *use* Snap, the monad is almost identical after all.
Jeremy Shaw <jer...@n-heptane.com> writes: > I assume you are asking why we don't use chunked transfer encoding when we > don't know the length so that we can support keep-alive even in that case?
> I think it's because all the people interested in writing low-level http server > code are working on their own servers instead of contributing to happstack :p
> It's not too late though, you can still submit a patch ;)
On May 26, 2010, at 9:38 AM, Gregory Collins wrote:
>> It would be useful if we simply want to use some existing happstack >> code in a snap application. But the goal is to 'permanently' improve >> the the happstack server backend and get rid of the existing lazy I/O >> code altogether..
> If you would want to move ServerPartT closer to Snap then you may as > well just *use* Snap, the monad is almost identical after all.
I don't want to move ServerPartT closer to the Snap monad. The Snap monad is basically ServerPartT without it's monad transformer powers. We already have that, it's called, "ServerPart".
In fact, we can use a newtype wrapper so that type errors are friendlier, and make ServerPartT look every bit as simple as the Snap monad. (Note: this version doesn't store the Response in a State monad, but that could be done too).
Alas, when you look at the current version of simpleHTTP it is easy to get distracted by all those WebT and UnWebT and ununWebT things that really don't belong in that file. And the documentation does not really guide you in the right direction either.
The plan for 0.6 has been to refactor that stuff out, and then properly document happstack so that people are only exposed to 'ServerPart'. Later they can use the monad transformers, and various types classes, if they want. (I do use those features, and am glad to have them). Or they can just stick with ServerPart.
As for using the snap-backend: The aim is not to change ServerPartT monad, but to change the happstack Request type so that the rqBody is an iteratee instead of a lazy bytestring. (and some similar changes to the Request type). For that I think we need a library that exposes just four things:
1. Response type 2. Request type 3. Config type 4. run :: Config -> (Request -> IO Response) -> IO ()
I can get all those things out of the snap-core + snap-server libraries. But, I am concerned that as snap grows, I will start needing a bunch of extra dependencies that have nothing to do with those 4 features.. :-/
On Wed, May 26, 2010 at 12:13 PM, Jeremy Shaw <jer...@n-heptane.com> wrote: > As for using the snap-backend: The aim is not to change ServerPartT monad, > but to change the happstack Request type so that the rqBody is an iteratee > instead of a lazy bytestring. (and some similar changes to the Request > type). For that I think we need a library that exposes just four things: > 1. Response type > 2. Request type > 3. Config type > 4. run :: Config -> (Request -> IO Response) -> IO () > I can get all those things out of the snap-core + snap-server libraries. > But, I am concerned that as snap grows, I will start needing a bunch of > extra dependencies that have nothing to do with those 4 features.. :-/
The bulk of new development for Snap will happen outside of snap-core and snap-server. We're sensitive to dependency creep and intend to keep -core and -server is simple as possible.
On Wed, May 26, 2010 at 1:31 AM, Jeremy Shaw <jer...@n-heptane.com> wrote: > It is certainly tempting. I have been hoping to port to hyena for a long > time, but that project seems a bit stalled perhaps?
Unfortunately I haven't had any time to work on Hyena for a long time. I've been yak shaving, trying to improve the GHC I/O manager so that all Haskell web servers can benefit from better performance (if you want to help please, please try out the code on github [1] and report bugs). I still like to write a iteratee based HTTP server that does only HTTP (i.e. it's not a web framework) and does it well so that people can experiment with writing web frameworks in Haskell without going through the pain of writing an HTTP server (which you all know is tricky).
Jeremy Shaw <jer...@n-heptane.com> writes: > I can get all those things out of the snap-core + snap-server > libraries. But, I am concerned that as snap grows, I will start > needing a bunch of extra dependencies that have nothing to do with > those 4 features.. :-/
The snap-core library is going to stay roughly where it is going forward -- probably any additional utilities to go in there will be in the "doesn't introduce additional dependencies" category. We're planning on putting Snap's more "frameworky" stuff in other packages.
> > I can get all those things out of the snap-core + snap-server > > libraries. But, I am concerned that as snap grows, I will start > > needing a bunch of extra dependencies that have nothing to do with > > those 4 features.. :-/
> The snap-core library is going to stay roughly where it is going forward > -- probably any additional utilities to go in there will be in the > "doesn't introduce additional dependencies" category. We're planning on > putting Snap's more "frameworky" stuff in other packages.
This was a huge lesson from xmonad: keep the dependencies very light for basic functionality, or the distros -- and thus users and end-user facing apps -- can't keep up.
On 26 Maj, 21:54, Johan Tibell <johan.tib...@gmail.com> wrote:
> Unfortunately I haven't had any time to work on Hyena for a long time. I've > been yak shaving, trying to improve the GHC I/O manager so that all Haskell > web servers can benefit from better performance (if you want to help please, > please try out the code on github [1] and report bugs).
I find your work on GHC I/O really interesting and important. It seems to be a bit above my level of Haskell expertise, though. Anyway if you need assistance with for example Windows port or testing I'd glad to help.
> On 26 Maj, 21:54, Johan Tibell <johan.tib...@gmail.com> wrote: > > Unfortunately I haven't had any time to work on Hyena for a long time. > I've > > been yak shaving, trying to improve the GHC I/O manager so that all > Haskell > > web servers can benefit from better performance (if you want to help > please, > > please try out the code on github [1] and report bugs).
> I find your work on GHC I/O really interesting and important. It seems > to be a bit above my level of Haskell expertise, though. Anyway if you > need assistance with for example Windows port or testing I'd glad to > help.
The event lib offers a subset of the interface in GHC.Conc. GHC.Conc will be implemented in terms of this smaller interface once the new I/O manager has been merged into base. You can play with this interface today. It's defined in
Using the function in EventSocket it's possible to build a server in typical Haskell style, using forkIO to handle each request. Bryan wrote a small HTTP server using this interface
Running this server (it can be built using the Makefile) could help us find bugs on different platforms. For Windows support we still need to write a select() backend. This should be pretty easy as it can be modelled after the poll backend:
Johan Tibell <johan.tib...@gmail.com> writes: > Running this server (it can be built using the Makefile) could help us > find bugs on different platforms. For Windows support we still need to > write a select() backend. This should be pretty easy as it can be > modelled after the poll backend:
For Windows we really should be using I/O completion ports. Huge pain in the butt though!
> > Running this server (it can be built using the Makefile) could help us > > find bugs on different platforms. For Windows support we still need to > > write a select() backend. This should be pretty easy as it can be > > modelled after the poll backend:
> For Windows we really should be using I/O completion ports. Huge pain > in the butt though!
I agree. It would probably change the design though as they work differently, as far as I understand, than the *nix APIs. In the end we decided that select() will have to be good enough for Windows for now. Is anyone writing high-performance servers for Windows?
> Using the function in EventSocket it's possible to build a server in > typical Haskell style, using forkIO to handle each request. Bryan > wrote a small HTTP server using this interface
I tested this server with and without the USE_GHC_IO_MANAGER flag. It works fine both ways, though I saw nearly identical performance. Is that expected or am I doing something wrong? I am using GHC 6.13.
Jeremy Shaw <jer...@n-heptane.com> writes: > On May 27, 2010, at 5:39 AM, Johan Tibell wrote:
> Using the function in EventSocket it's possible to build a server in typical Haskell style, using forkIO to handle each request. Bryan wrote a small HTTP server using this > interface
> I tested this server with and without the USE_GHC_IO_MANAGER flag. It > works fine both ways, though I saw nearly identical performance. Is > that expected or am I doing something wrong? I am using GHC 6.13.
You won't see much a difference unless you push it past 1024 open connections -- event-based server will keep processing requests, GHC-IO-manager-based-server will start refusing requests.
>> On May 27, 2010, at 5:39 AM, Johan Tibell wrote:
>> Using the function in EventSocket it's possible to build a >> server in typical Haskell style, using forkIO to handle each >> request. Bryan wrote a small HTTP server using this >> interface
>> I tested this server with and without the USE_GHC_IO_MANAGER flag. It >> works fine both ways, though I saw nearly identical performance. Is >> that expected or am I doing something wrong? I am using GHC 6.13.
> You won't see much a difference unless you push it past 1024 open > connections -- event-based server will keep processing requests, > GHC-IO-manager-based-server will start refusing requests.
When I tried that I got,
~/n-heptane/projects/haskell/event/benchmarks $ ./static-http static-http: accept: resource exhausted (Too many open files)
>> On May 27, 2010, at 5:39 AM, Johan Tibell wrote:
>>> Using the function in EventSocket it's possible to build a server in >>> typical Haskell style, using forkIO to handle each request. Bryan wrote a >>> small HTTP server using this >>> interface
>>> I tested this server with and without the USE_GHC_IO_MANAGER flag. It >>> works fine both ways, though I saw nearly identical performance. Is >>> that expected or am I doing something wrong? I am using GHC 6.13.
>> You won't see much a difference unless you push it past 1024 open >> connections -- event-based server will keep processing requests, >> GHC-IO-manager-based-server will start refusing requests.
> When I tried that I got,
> ~/n-heptane/projects/haskell/event/benchmarks $ ./static-http > static-http: accept: resource exhausted (Too many open files)
> :-/
> - jeremy
> -- > You received this message because you are subscribed to the Google Groups > "HAppS" group. > To post to this group, send email to happs@googlegroups.com. > To unsubscribe from this group, send email to > happs+unsubscribe@googlegroups.com <happs%2Bunsubscribe@googlegroups.com>. > For more options, visit this group at > http://groups.google.com/group/happs?hl=en.
>>> On May 27, 2010, at 5:39 AM, Johan Tibell wrote:
>>>> Using the function in EventSocket it's possible to build a server in >>>> typical Haskell style, using forkIO to handle each request. Bryan wrote a >>>> small HTTP server using this >>>> interface
>>>> I tested this server with and without the USE_GHC_IO_MANAGER flag. It >>>> works fine both ways, though I saw nearly identical performance. Is >>>> that expected or am I doing something wrong? I am using GHC 6.13.
>>> You won't see much a difference unless you push it past 1024 open >>> connections -- event-based server will keep processing requests, >>> GHC-IO-manager-based-server will start refusing requests.
>> When I tried that I got,
>> ~/n-heptane/projects/haskell/event/benchmarks $ ./static-http >> static-http: accept: resource exhausted (Too many open files)
>> :-/
>> - jeremy
>> -- >> You received this message because you are subscribed to the Google Groups >> "HAppS" group. >> To post to this group, send email to happs@googlegroups.com. >> To unsubscribe from this group, send email to >> happs+unsubscribe@googlegroups.com <happs%2Bunsubscribe@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/happs?hl=en.
>>>> On May 27, 2010, at 5:39 AM, Johan Tibell wrote:
>>>>> Using the function in EventSocket it's possible to build a server in >>>>> typical Haskell style, using forkIO to handle each request. Bryan wrote a >>>>> small HTTP server using this >>>>> interface
>>>>> I tested this server with and without the USE_GHC_IO_MANAGER flag. It >>>>> works fine both ways, though I saw nearly identical performance. Is >>>>> that expected or am I doing something wrong? I am using GHC 6.13.
>>>> You won't see much a difference unless you push it past 1024 open >>>> connections -- event-based server will keep processing requests, >>>> GHC-IO-manager-based-server will start refusing requests.
>>> When I tried that I got,
>>> ~/n-heptane/projects/haskell/event/benchmarks $ ./static-http >>> static-http: accept: resource exhausted (Too many open files)
>>> :-/
>>> - jeremy
>>> -- >>> You received this message because you are subscribed to the Google Groups >>> "HAppS" group. >>> To post to this group, send email to happs@googlegroups.com. >>> To unsubscribe from this group, send email to >>> happs+unsubscribe@googlegroups.com<happs%2Bunsubscribe@googlegroups.com> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/happs?hl=en.
On Thu, May 27, 2010 at 7:17 PM, Jeremy Shaw <jer...@n-heptane.com> wrote: > On May 27, 2010, at 12:09 PM, Gregory Collins wrote: > You won't see much a difference unless you push it past 1024 open
>> connections -- event-based server will keep processing requests, >> GHC-IO-manager-based-server will start refusing requests.
> When I tried that I got,
> ~/n-heptane/projects/haskell/event/benchmarks $ ./static-http > static-http: accept: resource exhausted (Too many open files)
Setting up your kernel for high performance networking is a bit tricky. We created a helper program to make it easier:
