net/http layering design and/or middleware

[Liam Staskawicz]

Hi!

I've been trying to consolidate my understanding of the options available for creating Go web apps that support a modular design for common layers of functionality.

My goals are to find a solution that:

• uses the standard net/http interface
• allows for a layered design, such that
• layers can pass information to one another if necessary - ie, an authentication layer can do its work and make that available to subsequent layers
• layers don't necessarily need to know anything about one another - ie, a logging layer doesn't need to know anything about a response compression layer

My basic assumption is that each layer should be implemented as an http.HandlerFunc, in order to adhere to the current standard interface. One difficulty here is that it's not clear how to pass arbitrary request-specific information between layers given that definition. This discussion suggests that it should be sufficient to wrap subsequent http.HandlerFuncs, which works well to provide the wrapped func with a specialized http.ResponseWriter, for instance, but not to pass any additional information.

One solution to this has been implemented in the Gorilla toolkit as gorilla/context, as discussed in http://groups.google.com/group/golang-nuts/msg/e2d679d303aa5d53, though this introduces an additional dependency such that we're no longer exclusively using the standard net/http interface. Additionally, it's a little bit of a bummer to access per-request data through a locked global map when access to the request itself is not threadsafe in the first place, but that's not necessarily a deal breaker.

Other packages, like mango, indigo and falcore, diverge from the standard interface, and provide a map[string]interface{} to share information between layers. The disadvantage here is, of course, not being able to share functionality between these projects due to the non-standard interfaces.

So my questions at this point are:

• is this summary correct?
• was the existing net/http package designed to support this kind of functionality?
• if so, are there examples that I've missed that demonstrate this?
• if not, does it make sense to design a new package (or adapt an existing one) to standardize these kinds of efforts around?

Thanks in advance for any insight,

Liam

[Kamil Kisiel]

Hi Liam,

I also really like the concept of layered handlers, and have used them to good effect in my applications. I actually started a package of such handlers at http://github.com/gorilla/handlers and have a few useful ones in there. I agree that there's not any one good way to pass things between layers, but that also seems to be at odds with the principle of layers not knowing about each other. Actually I've found that having one layer put things in to the request's Form attribute can be a good way to pass data down to lower layers.

I prefer to adhere to the net/http handler interface as much as a possible and haven't run in to too many limitations

[Liam Staskawicz]

On Sun, Mar 10, 2013 at 11:20 AM, Kamil Kisiel <kamil....@gmail.com> wrote:

Hi Liam,

Hi Kamil - thanks for your response.

 

I also really like the concept of layered handlers, and have used them to good effect in my applications. I actually started a package of such handlers at http://github.com/gorilla/handlers and have a few useful ones in there. I agree that there's not any one good way to pass things between layers, but that also seems to be at odds with the principle of layers not knowing about each other.

I'm not sure I'd totally agree with that. Just because they don't know about each other doesn't mean there can't be a well defined means of passing information between them.

And thanks for the pointer to the handlers package - I hadn't seen that. Looks nice! Though I'd still love to see some examples that get at this issue of passing data between layers at some point, to see how it plays out.

 

Actually I've found that having one layer put things in to the request's Form attribute can be a good way to pass data down to lower layers.

Hm - I suppose this could work in a pinch, but I wouldn't want to confuse any consumers of the request's Form attribute by artificially inserting values, if it could be avoided.

Liam

[James Pirruccello]

I'm a bit late to the game here, but I've been thinking about middleware recently. Internally I use the following conventions:

1) Each Handler middleware must accept an http.Handler as the first argument (and optionally may have additional arguments).

2) Each Handler middleware must return only an http.Handler.

This has worked pretty well for making my handlers pluggable on different projects. I'm curious to hear about how others have dealt with http middleware.

Cheers,

James

[Ukiah Danger Smith]

On Wednesday, December 25, 2013 7:16:11 PM UTC-5, James Pirruccello wrote:

I'm a bit late to the game here, but I've been thinking about middleware recently. Internally I use the following conventions:

1) Each Handler middleware must accept an http.Handler as the first argument (and optionally may have additional arguments).

2) Each Handler middleware must return only an http.Handler.

How are you passing data through the layers? Just with the extra arguments?

I'm just getting started with learning about ServeHTTP only as a middleware layer. I'm not sure how to pass information between the layers yet.

[James Pirruccello]

I think that, in general, http://justinas.org/writing-http-middleware-in-go/ gets it pretty much right. In general I try not to share information between layers of middleware (except by writing to the response), but in practice sometimes I have to do so for the middleware to function (e.g., his nosurf example).

This is slightly off topic, but if you're just getting started, a helpful realization for me (that might have been obvious to those who read the docs more carefully) is that you can "adapt" a function to an http.Handler without defining ServeHTTP on it. Starting with a function x with the following signature:

x := func Middleware(w http.ResponseWriter, r *http.Request)

... You can adapt it with http.HandlerFunc() to yield a fully-fledged http.Handler:

y := http.HandlerFunc(x) //y meets signature http.Handler

At that point, anything you can do with an http.Handler you can do with y, such as add it to your middleware chain.

- James

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/zu1Ouesgdeo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Ukiah Danger Smith]

On Wednesday, December 25, 2013 9:57:14 PM UTC-5, James Pirruccello wrote:

I think that, in general, http://justinas.org/writing-http-middleware-in-go/ gets it pretty much right. In general I try not to share information between layers of middleware (except by writing to the response), but in practice sometimes I have to do so for the middleware to function (e.g., his nosurf example).

I don't see how you can't share data between at least some layers. I'm thinking about Session handling. I want to create a User struct and pass it to all the middleware so it knows what the user can and can not do. I could recreate the struct from the Request at each layer, but I don't want to repeat myself. And what if one layer changes the User struct in a meaningful way and a latter layer needs that updated info.

[Quinlan Morake]

Hi,

Not sure of how close this is to the original question, but with regards to passing through authentication information and similar data, my strategy has been to load all such data in a wrapper, then call the page handlers from there, similar to what James has mentioned.

in my main function for example, my page handlers are wrapped by initWrapper.
 http.HandleFunc("/", initWrapper(pages.IndexPageHandler))

then in the initWrapper function, I load user information and system messages, and perform any other global type checks and functions; following which, I pass the user information to the page hanlder.

func initWrapper(pageHandleFunction SystemHandleFnc) HandleFnc {
  return func(writer http.ResponseWriter, request *http.Request) {
    // Handle any errors                                                                                                                                                                                                      
    defer func() {
      if x:= recover(); x != nil {
        log.Printf("[%v] caught pnic: %v", request.RemoteAddr, x)
      }
    }()

    // Authenticate user                                                                                                                                                                                                      
    var cookie *http.Cookie
    var err error
    // Does a user have a cookie
    if cookie, err = request.Cookie("www.site.com"); err != nil{
      // Send the user to the login page                                                                                                                                                                                      
      http.Redirect(writer, request, "/login.html", http.StatusFound)
      return
    }

    var loginInfo pages.LoginInfo
     // Is the cookie valid
    if loginInfo, err = pages.AuthenticateCookie(cookie.Value); err != nil {
      // Return to the login page                                                                                                                                                                                              
      http.Redirect(writer, request, "/login.html", http.StatusFound)
    } else {
      // Check for any system messages                                                                                                                                                                                        
      var message *pages.SysMessage
      if message, err = pages.CheckForMessages(loginInfo.UserDetail.UserID); err != nil {
        message.Class = "error"
        message.Code = 50
        message.Message = err.Error()
      }
      ...
      // Return page                                                                                                                                                                                                          
      pageHandleFunction(writer, request, &loginInfo, message)
}

To facilitate the updating of login information, as well as it being accessed and written to by different systems, the login information is kept in memcached, and can be accessed and updated through a global function too, in the event that a "lower" level needs to modify it. Thus far, that only occurs in the account editing page, where user would modifiy their username / password etc. But this is the mechanism I use so as to pass my user information to any lower levels.

Regarding layering functionality, as seems to be the second part of the first question, my thinking is not to use "layers" as such, but different (sub) systems. Right now, to achieve the interoperability, I'm using either memcached once again, where the subsystem would pull data items off of a queue; or web service calls, in which system A directly calls subsystem B. For example, with regards to logging I'm using a memcahced implementation, in which a logRequest{RequestID} is added to a "requests" item in memcache, and then the logging subsystem, a different execuable, pulls it off, pulls the logRequest specific info, and then logs it appropriately. My notifications subsystem works with soap calls, and when something takes place that would like someone to be notified, it calls a webservice with appropriate data to do so. My intention is to rewrite this so as to use ZeroMQ in the near future, but thus far, this has been my take on layered functionality.

Quinlan