On Jun 21, 3:51 pm, Gregory Collins <g...@gregorycollins.net> wrote:
> This is exactly the kind of shim layer I was proposing! Are we on the
> same page yet? :)
I think so. I think part of my confusion is that I imagined that
happstack-fastcgi exercised most of the code in #3, when in probably
exercises none of it :)
In terms of the data types for Request/Response, I think the reason to
prefer hyena's version is that you can implement lazy I/O on top of
left-fold enumeration, but not the other way around. However, I am not
really sure how this would play out in practice.
And, if you are thinking of taking the stuff in SimpleHTTP
(ServerPartT, dir, path, etc), moving it out of happstack-server into
its own package, and generalizing it so that it can work with
happstack-server, hyena, etc, then I am totally in favor of that. I
think the tricky part of doing that right now is that ServerPartT,
dir, path, etc, are pretty tightly integrated with the specific
Request/Response types in happstack-server. So, you either need to add
a type class that abstracts out the type, or get happstack-server and
hyena to use the same type of the Response/Result ? Or, I guess, write
a glue layer than converts between hyena's types and the Response/
Request type (which is what the current happstack-fastcgi does?)
Anyway, it sounds like you have a better handle on this than me, so
I'm all in favor of whatever you propose ;)
stepcut <jer...@n-heptane.com> writes: > In terms of the data types for Request/Response, I think the reason to > prefer hyena's version is that you can implement lazy I/O on top of > left-fold enumeration, but not the other way around. However, I am not > really sure how this would play out in practice.
One of the things I wanted to do for Hac Phi was to spend some time reworking the ServerPartT stuff:
* it has an awkward name
* the towering ServerPartT/WebT/FilterT/MaybeT/ErrorT stack is unnecessarily complicated, which scares away beginners
* ServerPartT leaks its internals out all over the place
While we're looking at that, we could spend some time thinking about the response type also. We've already discussed extending it to support sendfile(), we could put in iteratee support there also.
The simplest way discussed so far is to add more constructors to the RqBody type:
data RqBody = Body ByteString | Stream (StreamG ByteString Word8) | SendFile FilePath
> And, if you are thinking of taking the stuff in SimpleHTTP > (ServerPartT, dir, path, etc), moving it out of happstack-server into > its own package, and generalizing it so that it can work with > happstack-server, hyena, etc, then I am totally in favor of that.
That would indeed be what I'd propose.
> I think the tricky part of doing that right now is that ServerPartT, > dir, path, etc, are pretty tightly integrated with the specific > Request/Response types in happstack-server. So, you either need to add > a type class that abstracts out the type, or get happstack-server and > hyena to use the same type of the Response/Result ? Or, I guess, write > a glue layer than converts between hyena's types and the Response/ > Request type (which is what the current happstack-fastcgi does?)
To me, ServerPart needs to depend on a particular set of Request/Response types, so I'd argue for going by the latter route.
Hey Gregory, first of all, I can't wait to see what you guys produce at Hac
Phi. I would love to abstract the "core" http server from the transformer
stack. This would lay the pavement for plugging in the various available
http backends currently available so we can "race them" (stealing anothers
words).
I want to add that I am currently working on the prototype of a small shim
library which exposes a sendfile interface portably. I have windows natively
supported already; I will release version 0.1 once I add Linux native
support. Another nice feature of this lib is that if no native
implementation is available, it seamlessly falls back to a portable haskell
implementation. This hopefully can be worked into http engines such as Hyena
and "happs classic".
If anyone wants to take a look at what I have so far and do some really
early/alpha testing & feedback, please take a look at:
> > In terms of the data types for Request/Response, I think the reason to
> > prefer hyena's version is that you can implement lazy I/O on top of
> > left-fold enumeration, but not the other way around. However, I am not
> > really sure how this would play out in practice.
> One of the things I wanted to do for Hac Phi was to spend some time
> reworking the ServerPartT stuff:
> * it has an awkward name
> * the towering ServerPartT/WebT/FilterT/MaybeT/ErrorT stack is
> unnecessarily complicated, which scares away beginners
> * ServerPartT leaks its internals out all over the place
> While we're looking at that, we could spend some time thinking about the
> response type also. We've already discussed extending it to support
> sendfile(), we could put in iteratee support there also.
> The simplest way discussed so far is to add more constructors to the
> RqBody type:
> data RqBody = Body ByteString
> | Stream (StreamG ByteString Word8)
> | SendFile FilePath
> > And, if you are thinking of taking the stuff in SimpleHTTP
> > (ServerPartT, dir, path, etc), moving it out of happstack-server into
> > its own package, and generalizing it so that it can work with
> > happstack-server, hyena, etc, then I am totally in favor of that.
> That would indeed be what I'd propose.
> > I think the tricky part of doing that right now is that ServerPartT,
> > dir, path, etc, are pretty tightly integrated with the specific
> > Request/Response types in happstack-server. So, you either need to add
> > a type class that abstracts out the type, or get happstack-server and
> > hyena to use the same type of the Response/Result ? Or, I guess, write
> > a glue layer than converts between hyena's types and the Response/
> > Request type (which is what the current happstack-fastcgi does?)
> To me, ServerPart needs to depend on a particular set of
> Request/Response types, so I'd argue for going by the latter route.
> G.
> --
> Gregory Collins <g...@gregorycollins.net>
On Mon, Jun 22, 2009 at 11:55 PM, Matthew Elder <m...@mattelder.org> wrote: > I want to add that I am currently working on the prototype of a small shim > library which exposes a sendfile interface portably. I have windows natively > supported already; I will release version 0.1 once I add Linux native > support. Another nice feature of this lib is that if no native > implementation is available, it seamlessly falls back to a portable haskell > implementation. This hopefully can be worked into http engines such as Hyena > and "happs classic".
Cool. Would you consider merging this into network-bytestring which already has sendfile support for unixes?
My only problem with this is that bytestring really has nothing to do
with sendfile, and more to do with network. If anything I would merge
it into network. For now I am releasing on hackage to stabilize it,
though.
On 6/22/09, Johan Tibell <johan.tib...@gmail.com> wrote:
> On Mon, Jun 22, 2009 at 11:55 PM, Matthew Elder <m...@mattelder.org> wrote:
>> I want to add that I am currently working on the prototype of a small shim
>> library which exposes a sendfile interface portably. I have windows
>> natively
>> supported already; I will release version 0.1 once I add Linux native
>> support. Another nice feature of this lib is that if no native
>> implementation is available, it seamlessly falls back to a portable
>> haskell
>> implementation. This hopefully can be worked into http engines such as
>> Hyena
>> and "happs classic".
> Cool. Would you consider merging this into network-bytestring which already
> has sendfile support for unixes?
On Wed, Jun 24, 2009 at 8:39 AM, Matthew Elder <m...@mattelder.org> wrote:
> My only problem with this is that bytestring really has nothing to do > with sendfile, and more to do with network. If anything I would merge > it into network. For now I am releasing on hackage to stabilize it, > though.
Yes, it probably fits better in network. There's an open ticket for merging network-bytestring into network:
Matthew Elder <m...@mattelder.org> writes: > The simplest way discussed so far is to add more constructors to the > RqBody type:
> data RqBody = Body ByteString > | Stream (StreamG ByteString Word8) > | SendFile FilePath
> Gregory, didn't you mean Response?
...yeah, I guess I did. Sorry, I mucked up Request (which has that RqBody type) and Response (which just uses a plain bytestring now.)
Note that between Response and the new SendFile constructor you're planning on introducing, the only difference is the body type; maybe we should think about moving SendFile / left-fold (if we do that) down into the body type?
Either way would be fine, of course, it's an aesthetic decision.
>> The simplest way discussed so far is to add more constructors to the
>> RqBody type:
>> data RqBody = Body ByteString
>> | Stream (StreamG ByteString Word8)
>> | SendFile FilePath
>> Gregory, didn't you mean Response?
> ...yeah, I guess I did. Sorry, I mucked up Request (which has that
> RqBody type) and Response (which just uses a plain bytestring now.)
> Note that between Response and the new SendFile constructor you're
> planning on introducing, the only difference is the body type; maybe we
> should think about moving SendFile / left-fold (if we do that) down into
> the body type?
> Either way would be fine, of course, it's an aesthetic decision.
> G.
> --
> Gregory Collins <g...@gregorycollins.net>
> > If your app uses 50 mb in fcgi it will also use 50 mb as a backend > > http server as they are both daemonlike persistent processes; why are > > we so worried about supporting fcgi?
> In principle I'm inclined to agree with you, on the other hand there's > little reason NOT to support it -- happstack-fastcgi is only ~220 > LOC. FastCGI is supposed to be marginally more efficient (~5%?), and if > you use it e.g. lighttpd will spawn/kill your server instances for you > (also not such a big deal...)
> Can we agree that (for example) something like happstack-hyena might get > implemented using the same basic strategy as happstack-fastcgi? We > should be able to knock three or four of those out during Hac Phi, no > sweat. Maybe the http server should be split out from the "rack"-like > middleware layer in happstack-server, as "just another backend"?
> > > If your app uses 50 mb in fcgi it will also use 50 mb as a backend
> > > http server as they are both daemonlike persistent processes; why are
> > > we so worried about supporting fcgi?
> > In principle I'm inclined to agree with you, on the other hand there's
> > little reason NOT to support it -- happstack-fastcgi is only ~220
> > LOC. FastCGI is supposed to be marginally more efficient (~5%?), and if
> > you use it e.g. lighttpd will spawn/kill your server instances for you
> > (also not such a big deal...)
> > Can we agree that (for example) something like happstack-hyena might get
> > implemented using the same basic strategy as happstack-fastcgi? We
> > should be able to knock three or four of those out during Hac Phi, no
> > sweat. Maybe the http server should be split out from the "rack"-like
> > middleware layer in happstack-server, as "just another backend"?
> Jinjing Wang has already had some success getting a happstack app
> (gitit) running through his 'hack' (a rack-like middleware layer):
On Wed, Jun 17, 2009 at 5:47 PM, stepcut<jer...@n-heptane.com> wrote:
> Oh, I also wanted to add that I think happstack will have achieved > ultimate success when it no longer exists. <snip> > Hence, at some future date, happstack could be merely a concept that > builds on top of a number of existing libraries. In the same way that > we don't think of mtl or stm as being an explicit part of happstack, > we might some day not think any of the current components as belonging > exclusively to happstack.
> At that point, happstack will be more of a philosophy of how to build > a highly scalable, stable website rather than any specific > implementation of code.
Getting in late on this discussion as I was busy packing/moving/unpacking, but I strongly second this sentiment. Also, I saw that there was a bit of discussion about future work on Happstack-State, multimaster, & sharding. To be honest, my main interest in the Happstack project is working on the persistence model & backend so I'm more than willing to take lead on any effort to rewrite the macid implementation if no one is chomping at the bit.
On Jun 25, 2:07 pm, Creighton Hogg <wch...@gmail.com> wrote:
> Also, I saw that there was a bit of discussion about future work on
> Happstack-State, multimaster, & sharding. To be honest, my main
> interest in the Happstack project is working on the persistence model
> & backend so I'm more than willing to take lead on any effort to
> rewrite the macid implementation if no one is chomping at the bit.
What do you mean by rewrite? The current code is largely undocumented,
the multimaster code is incomplete, and sharding has not yet been
started. On the other hand, for a single server system, it seems
pretty stable. I have looked at the code extensively and the code
looks pretty good. And it seems like it should be possible finish
multimaster and add sharding to the current code base.
But, rewrite makes it sound like you want to start over, perhaps with
a different approach? If so, why do you feel the current code is not
suitable, and what do you plan to do differently?
On Thu, Jun 25, 2009 at 1:08 PM, stepcut<jer...@n-heptane.com> wrote:
> On Jun 25, 2:07 pm, Creighton Hogg <wch...@gmail.com> wrote:
>> Also, I saw that there was a bit of discussion about future work on >> Happstack-State, multimaster, & sharding. To be honest, my main >> interest in the Happstack project is working on the persistence model >> & backend so I'm more than willing to take lead on any effort to >> rewrite the macid implementation if no one is chomping at the bit.
> What do you mean by rewrite? The current code is largely undocumented, > the multimaster code is incomplete, and sharding has not yet been > started. On the other hand, for a single server system, it seems > pretty stable. I have looked at the code extensively and the code > looks pretty good. And it seems like it should be possible finish > multimaster and add sharding to the current code base.
> But, rewrite makes it sound like you want to start over, perhaps with > a different approach? If so, why do you feel the current code is not > suitable, and what do you plan to do differently?
Sorry, I made an awful word choice. I meant 'revise', as in stream line, polish, & finish. Regardless of my linguistic handicap, I think we're on the same page.
On Jun 25, 3:13 pm, Creighton Hogg <wch...@gmail.com> wrote:
> Sorry, I made an awful word choice. I meant 'revise', as in stream
> line, polish, & finish. Regardless of my linguistic handicap, I think
> we're on the same page.
Sweet!
If you are looking for a good place to start, you might try debugging
the happstack-state test that fails randomly. Namely,
Mae and I looked at it briefly a while ago, and it looked like the
event right after the checkpoint was sometimes lost. If this is a real
bug, then it would be great to squash it. And, in the process, you
should get a clear understanding of how happstack-state works under
the hood (assuming you don't already).
