Frost (in case anyone is interested!)

[Andrew Cherry]

Hi all,

Been meaning to say something in here for a bit, just to share a little bit about a personal project i'm working on (and which i'm hoping to open source very shortly, when it's a little bit more complete, and slightly more stable - I'm a terrible compulsive refactorer). It's not very much like what others have been doing in here, which looks great, but I think there's plenty of different valid approaches and people seeing that F# has variety and live projects can never be a bad thing (I hope!)

Anyway, as a brief intro, I started as a rough port/re-implementation of Webmachine/Liberator (Erlang/Clojure respectively, Liberator is also inspired by Webmachine). It's evolved since then in to a little more of a set of components which work together, but are more of a stack.

Bottom level is the approach I took around OWIN. I'm not a wild fan of the OWIN approach of mutable state you "do stuff" to, but it is what it is, and after various experiments I decided to live with it and make the best of it. So I have a set of lenses which provide strongly typed access to aspects of the OWIN state, without you need to know quite what it's doing in the middle, plus a core monad (called frost) which is essentially asyncState, where state is an OWIN environment. There are then a few functions for using the lenses to manipulate that in a nice way, along with some convenience operators. A couple of examples, pulled from Frost internals at the moment...

let internal isMethod m =

    frost { return! (=) m <!> get Request.Method }

let internal defaultHandler code phrase =

    frost {

        do! Some code => Response.StatusCode

        do! Some phrase => Response.ReasonPhrase }

The first is a quick predicate function which checks if the current method is of type m (m here is of type Method, a Frost type) and the second sets the response code and phrase. These are all "strongly typed" to the end user, but underneath are simply mutating the OWIN dictionary (the lens implementation is FSharpx, with various combinations of lenses, particularly xmap ones!) It's quite nice to be able to do things like let! accept = get (Request.Header "Accept") and get back a strongly typed Header option (or update, set, etc. where relevant). It does make me pine for some kind of match! implementation though!

Next level up in frost is a simple router, which again has a monad interface to the end user, with a couple of custom operations (it's essentially designed to be a declarative approach). Another quick example...

let routes =

    frostRoutes {

        resource "/tuples" tuples

        resource "/tuples/:key" tuple }

Fairly obvious, but underneath it's a simple prefix-trie like router. They're composable, so you can include routes within routes, etc.

The current next level up from that (and the layer which adds the "resource" custom operation to the routing monad) is for resources, which essentially recreates Webmachine/Liberator in F#, with a monad allowing you to declaratively define which functions in the decision graph (RFC2616, essentially) you wish to override for this resource. (See the Clojure Liberator docs for a much better explanation if you're not already familiar!) 

Examples...

let tuple =

    frostResource {

        allowedMethods [ GET; DELETE; PUT ]

        availableMediaTypes [ "application/json" ]

        doDelete tupleRemove

        doPut tupleUpdate

        exists tupleExists

        handleOk tupleShow }

let tuples =

    frostResource {

        allowedMethods [ GET; POST ]

        availableMediaTypes [ "application/json" ]

        doPost tupleAdd

        handleOk tupleList }

Those are the functions referred to in the routing example snippet, and the functions provided in them are "frost" functions, of some correct return type. They probably look a bit of an odd way to do things if you're not familiar with Webmachine, etc.! (Those are not fully wired up, also, I'd need to implement a little bit more to make it work properly, but you get the idea, hopefully) Composition of these things applies as well, so you define something like "frostResource secure { ... }" with functions defined for authorization and permission, etc. and then include that in other resources, so it can be quite neat for composition and re-use.

Anyway, just thought I'd share in case anyone was interested. I've still got a fair amount of work to do on the internals, and I want to build in a great inspection/debugging system as well before it's really considered usable, but I'm hoping to have it out for people to use (with some docs, if I pull my finger out) in the not too distant future. It probably isn't to everyone's taste, but like most things it's a personal itch to scratch. I've also got a few things which I haven't seen elsewhere that I want to roll in to it, like supported for semantically versioned resources, that kind of thing.

Any comments are welcome, hope that people won't hate me too much for adding "yet another web *" to an ecosystem!

Andrew

[Ryan Riley]

On Tue, Jul 22, 2014 at 5:10 AM, Andrew Cherry <and...@xyncro.com> wrote:

Hi all,

Been meaning to say something in here for a bit, just to share a little bit about a personal project i'm working on (and which i'm hoping to open source very shortly, when it's a little bit more complete, and slightly more stable - I'm a terrible compulsive refactorer). It's not very much like what others have been doing in here, which looks great, but I think there's plenty of different valid approaches and people seeing that F# has variety and live projects can never be a bad thing (I hope!)

Absolutely! Welcome!

 

Anyway, as a brief intro, I started as a rough port/re-implementation of Webmachine/Liberator (Erlang/Clojure respectively, Liberator is also inspired by Webmachine). It's evolved since then in to a little more of a set of components which work together, but are more of a stack.

Bottom level is the approach I took around OWIN. I'm not a wild fan of the OWIN approach of mutable state you "do stuff" to, but it is what it is, and after various experiments I decided to live with it and make the best of it. So I have a set of lenses which provide strongly typed access to aspects of the OWIN state, without you need to know quite what it's doing in the middle, plus a core monad (called frost) which is essentially asyncState, where state is an OWIN environment.

1. Very glad you chose to build on OWIN. I agree about the mutable state. I think we can work around that, though. See the With member from Dyfrig.
2. I really like Webmachine from the little I know about it. I'm excited to see the full project!
3. AsyncState is an approach I've tried in several projects. If you go back to some of the earliest commits for Frank, you'll find I used this there before ripping it out in favor of System.Net.Http types. I think I also tried this as a stream combinator approach for Fracture but found some performance bottlenecks.

There are then a few functions for using the lenses to manipulate that in a nice way, along with some convenience operators. A couple of examples, pulled from Frost internals at the moment...

let internal isMethod m =

    frost { return! (=) m <!> get Request.Method }

let internal defaultHandler code phrase =

    frost {

        do! Some code => Response.StatusCode

        do! Some phrase => Response.ReasonPhrase }

The first is a quick predicate function which checks if the current method is of type m (m here is of type Method, a Frost type) and the second sets the response code and phrase. These are all "strongly typed" to the end user, but underneath are simply mutating the OWIN dictionary (the lens implementation is FSharpx, with various combinations of lenses, particularly xmap ones!) It's quite nice to be able to do things like let! accept = get (Request.Header "Accept") and get back a strongly typed Header option (or update, set, etc. where relevant). It does make me pine for some kind of match! implementation though!

I really like your approach. I had considered lenses but focused on a quick and dirty Environment implementation in Dyfrig. I'm not satisfied with that approach and would like to re-implement the parts of Dyfrig I care about on top of frost. Would you be open to that? I'll detail those parts below.

 

Next level up in frost is a simple router, which again has a monad interface to the end user, with a couple of custom operations (it's essentially designed to be a declarative approach). Another quick example...

let routes =

    frostRoutes {

        resource "/tuples" tuples

        resource "/tuples/:key" tuple }

Fairly obvious, but underneath it's a simple prefix-trie like router. They're composable, so you can include routes within routes, etc.

I'm amazed at how similar my idea of routes from Frank and Taliesin are to yours, specifically in that routes are composable and resource-oriented.

 

The current next level up from that (and the layer which adds the "resource" custom operation to the routing monad) is for resources, which essentially recreates Webmachine/Liberator in F#, with a monad allowing you to declaratively define which functions in the decision graph (RFC2616, essentially) you wish to override for this resource. (See the Clojure Liberator docs for a much better explanation if you're not already familiar!) 

Examples...

let tuple =

    frostResource {

        allowedMethods [ GET; DELETE; PUT ]

        availableMediaTypes [ "application/json" ]

        doDelete tupleRemove

        doPut tupleUpdate

        exists tupleExists

        handleOk tupleShow }

let tuples =

    frostResource {

        allowedMethods [ GET; POST ]

        availableMediaTypes [ "application/json" ]

        doPost tupleAdd

        handleOk tupleList }

Those are the functions referred to in the routing example snippet, and the functions provided in them are "frost" functions, of some correct return type. They probably look a bit of an odd way to do things if you're not familiar with Webmachine, etc.! (Those are not fully wired up, also, I'd need to implement a little bit more to make it work properly, but you get the idea, hopefully) Composition of these things applies as well, so you define something like "frostResource secure { ... }" with functions defined for authorization and permission, etc. and then include that in other resources, so it can be quite neat for composition and re-use.

I really like this approach. I had tinkered with computation expressions as a resource construction syntax in Frank many years ago, before the custom operations were available. I definitely like the direction you have taken.

 

Anyway, just thought I'd share in case anyone was interested. I've still got a fair amount of work to do on the internals, and I want to build in a great inspection/debugging system as well before it's really considered usable, but I'm hoping to have it out for people to use (with some docs, if I pull my finger out) in the not too distant future. It probably isn't to everyone's taste, but like most things it's a personal itch to scratch. I've also got a few things which I haven't seen elsewhere that I want to roll in to it, like supported for semantically versioned resources, that kind of thing.

Any comments are welcome, hope that people won't hate me too much for adding "yet another web *" to an ecosystem!

Not at all! I don't think there's a problem adding more to the mix. The primary goal of several of us banding together was that we found we were working on libraries with very similar ideas. In the case of Lev, Dmitry, and me, we were building web platform abstractions (HyperF and Frank) around System.Net.Http with HttpRequestMessage -> Async<HttpResponseMessage> as a primary handler model. James was looking to build a combinator library for Simple.Web and found the ones in Frank useful.

For what it's worth, I'll spell out the reasons I created Dyfrig and Taliesin, as well, for those who may not have spoken to me about them. I built Frank with its own HTTP abstraction originally, then adopted System.Net.Http as it seemed to address a number of issues while re-using a well-designed set of abstractions. However, I could never find a good way to model routes well on top of System.Web.Routing. The biggest hurdle was trying to model resources and not just a loose collection of route handlers. I also attempted to re-implement all of Web API in a more functional way that would still be able to take advantage of the add-on libraries like the documentation generation library available to Web API.

After completing the task and comparing the two implementations side-by-side, I decided to throw away the re-implementation and stick to the simple approach. I did so because I realized what I really wanted was a type provider to generate route boilerplate from an API document. We've discussed several on this list already, including Apiary, RAML, and Hydra. I started Taliesin to address that goal and began building it as a completely generic routing library on top of (I imagined) either OWIN or System.Net.Http with perhaps the ability to adapt to any other abstraction layer.

While that was happening, the qed project was starting to use F# on top of OWIN, and I had some code sitting around from Fracture to implement a slightly more F#-focused OWIN adapter. I pulled that into a new library called Dyfrig. I don't think the qed project pulled that in, and I was never really satisfied with what I had there.

I'm still very happy with System.Net.Http, so my most recent work has been focused around that. As you'll notice, Taliesin now uses those types exclusively. (I also have this idea that I want to build load balancing into Taliesin's routing mechanism, which is why it is built the way it is.) Dyfrig also supports a System.Net.Http-based programming model, both in the model mentioned above, HttpRequestMessage -> Async<HttpResponseMessage>, and the Railway-oriented approach described by Scott Wlaschin, which is called OwinRailway and follows something like HttpRequestMessage -> Async<Choice<HttpResponseMessage, exn>>. You can see an example of chaining building a railway handler in the tests. The railway approach is an attempt at flattening the middleware Russian doll model used in OWIN.

All that to say: I think frost fits in well with what we are trying to do. It certainly fits in with my goals, and I'd like to leverage it and replace or augment my Environment abstraction with what you are building. Any chance I could get a peek at your code, or would you like to discuss further?

Best regards,

Ryan

[Ryan Riley]

I almost to add that my other crazy goal with Taliesin is to allow hot-swapping of handler implementation. That could mean either or both:

• Replace a handler for a resource for a given resource state, i.e. on a 201 Accept replace with a handler that blocks subsequent requests until the process completes.
• Providing a mechanism by which developers could ship the handler definition to their running production web server. This would leverage the type provider concept to generate the boilerplate with "holes" for the handlers and allow for time-varying handlers to be applied.

[Andrew Cherry]

Hi Ryan,

Nice to have some feedback! I'm more than happy for anyone to build on any bits of Frost that they'd like to. I'd be delighted if people wanted to build on it (or grab any ideas and run with them).

I experimented with the Microsoft http types at once stage, but found that I thought they were likely to be restrictive (and difficult to use piecemeal - trying to just get a single header without a whole request seemed impossible). They also seemed rather restrictive in some senses (verbs for example being a locked down enum, with no way of extending it, to provide support for PATCH for example).

Anyway, it's still early days, but in case it's helpful, i've just done an early drop of the dramatically unfinished source as it stands of this afternoon. (https://github.com/xyncro/frost). There'll likely be big breaking changes all over the place, but if anyone's curious, there's some theoretical bits to look over.

As a quick run through of the most obvious things I'll be doing next...

- The lenses over the OWIN dictionary in Frost.Core are drastically incomplete/shonky/temporary. I've got nice (proper parser) based types coming for some of that in the near future, so those will be nicer I hope.

- The router is rather basic right now, and doesn't match on verb. It will do at some point! For now it's only being used for resources, which handle all of the verbs internally.

- The logic in Frost.Resource isn't complete. Although it can run through the decision tree now, a lot of the functions are just "return true/false" place holders to be implemented properly once nice lens based access to more of the Request/Response is ready.

- Frost.Resource doesn't currently return any representations (it just prints the object to stdout at the moment, for testing!). I'm still working out exactly how media type codecs should fit in to a really strongly typed world (I'm not a big fan of Web API and the like taking a very generic JSON serializer, for example, and it doesn't really fit with the explicit nature of the decision tree either).

Once those things are solved, I would say that it's pretty close to being a usable and useful set of libraries. I've got much bigger plans, but that would get it to a point of "worth existing" I think. I also am very keen to find a way of supporting the output of proper hypermedia resources (Hydra, etc.) as some kind of optional/pluggable/composable thing, so that discussion is one I'm going to be getting involved with perhaps!

For anyone curious, test/Frost.Example is the partial stage of what will likely form the Frost tutorial, building a resource based key/value server of some kind. It's probably a bit opaque with just the code, but it might be of interest...

Oh, as an aside, I like crazy goals! Hot swapping things, zero downtime changes, all of those kinds of things are great to talk about. I have a lot of fondness for Erlang and that approach, and would love to see F# go in that direction in some places. (I saw Vesa Karvonen posting in one of the other threads too, and in case he's reading, I've been blown away by the work on Hopac. I'd love to see what web stuff would look like and how pure we could get in that direction. I'd love to experiment with Hopac directly on sockets, but I don't yet understand enough about how that might work in terms of scheduling... From what he was saying though, could be interesting times ahead on that one! Must finish the Reppy book...)

Anyway, there's some code (of probably very dubious quality, sausages being made and all that), up on GitHub now. 

Cheers!

Andrew

--
You received this message because you are subscribed to the Google Groups "F# Web Stack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web-stack-fs...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ryan Riley]

We should have a video chat. I think I'm working on mostly parallel things with you. I'm happy to contribute some of my work into this project, or vice versa.

[Ryan Riley]

On Tue, Jul 22, 2014 at 12:22 PM, Andrew Cherry <and...@xyncro.com> wrote:

Hi Ryan,

Nice to have some feedback! I'm more than happy for anyone to build on any bits of Frost that they'd like to. I'd be delighted if people wanted to build on it (or grab any ideas and run with them).

Thanks!

 

I experimented with the Microsoft http types at once stage, but found that I thought they were likely to be restrictive (and difficult to use piecemeal - trying to just get a single header without a whole request seemed impossible). They also seemed rather restrictive in some senses (verbs for example being a locked down enum, with no way of extending it, to provide support for PATCH for example).

That is actually not an enumeration. It's a class with static members wrapping instances. You can just as easily create the GET method yourself using HttpMethod("GET"). It even equates to HttpMethod.Get. That said, there certainly are some wonky parts, but I've found that the attention to detail rather nice.

 

Anyway, it's still early days, but in case it's helpful, i've just done an early drop of the dramatically unfinished source as it stands of this afternoon. (https://github.com/xyncro/frost). There'll likely be big breaking changes all over the place, but if anyone's curious, there's some theoretical bits to look over.

As a quick run through of the most obvious things I'll be doing next...

- The lenses over the OWIN dictionary in Frost.Core are drastically incomplete/shonky/temporary. I've got nice (proper parser) based types coming for some of that in the near future, so those will be nicer I hope.

- The router is rather basic right now, and doesn't match on verb. It will do at some point! For now it's only being used for resources, which handle all of the verbs internally.

Take a look at Taliesin. I've got a working router in there, and I've contemplated transforming it to implement a trie. I would be interested in your thoughts. Also, contrary to my earlier remarks, it is built directly on top of OWIN (via Dyfrig).

 

- The logic in Frost.Resource isn't complete. Although it can run through the decision tree now, a lot of the functions are just "return true/false" place holders to be implemented properly once nice lens based access to more of the Request/Response is ready.

- Frost.Resource doesn't currently return any representations (it just prints the object to stdout at the moment, for testing!). I'm still working out exactly how media type codecs should fit in to a really strongly typed world (I'm not a big fan of Web API and the like taking a very generic JSON serializer, for example, and it doesn't really fit with the explicit nature of the decision tree either).

Mauricio Scheffer has some excellent tools here:

• https://github.com/fsprojects/fsharpx/blob/master/src/FSharpx.Http/Conneg.fs
  
• https://github.com/mausch/Fleece