RFC: Blog post: How to not use an HTTP router

[Axel Wagner]

Hey gophers,

in an attempt to rein in the HTTP router epidemic, I tried writing down a) why I think any router/muxer might not be a good thing to use (much less write) and b) what I consider good, practical advice on how to route requests instead. It's not rocket science or especially novel, but I wanted to provide more useful advice than just saying "just use net/http" and haven't seen that a lot previously.

Feedback is welcome :)

http://blog.merovius.de/2017/06/18/how-not-to-use-an-http-router.html

[Kevin Conway]

If I understand correctly, you're describing a simplified version of https://twistedmatrix.com/documents/current/web/howto/using-twistedweb.html which provides the concept of "path" by having a system that generates a graph of resource nodes than can be rendered. Routing to any specific endpoint is a matter of graph traversal until the system reaches a leaf node and calls a relevant HTTP method. It's a model I've used successfully in the past and found it to be enjoyable.

That being said, having access to easy-to-use, parameterized routes makes things much simpler IMO. It cleanly separates the logic required for resource graph traversal from the endpoint rendering. Having them combined was one of my major complaints about the "resource as a router" model.

A large concept that this article also ignores is middleware. The decorator pattern is quite a powerful one and is facilitated by nearly every 3rd party mux implementation. Top level support for middleware makes adding decorators to some, or all, endpoint rendering resources  an easy task regardless of the specific resource graph traversal required to activate them. The ability to take a single purpose, well tested endpoint and wrap it in other single purpose, well tested functionality (such as logging, stats, tracing, retries, backoffs, circuit breaking, authentication, etc.) without modifying the core logic of the endpoint is a large value add. It's unclear how the "resource as a router" model could easily provide such a feature. This is not to say it's impossible, I simply haven't seen it done well outside the mux model before.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Axel Wagner]

I like the notion/notation of a resource graph and will use it in the future :)

On Mon, Jun 19, 2017 at 12:31 AM, Kevin Conway <kevinjac...@gmail.com> wrote: 

That being said, having access to easy-to-use, parameterized routes makes things much simpler IMO. It cleanly separates the logic required for resource graph traversal from the endpoint rendering.

I don't think that's mutually exclusive with what I'm proposing. Indeed, ServeHTTP can contain all of the traversal logic, while the rendering logic is contained in separate methods of the Handler.

All I'm arguing is, that the graph traversal should be written as local as possible; best case, no node knows about anything but its immediate neighbors. But even that is not forced; a handler might contain more logic than just a single component, if necessary (an example of this in my post is the last UserHandler; it may shift more than one component, to consume the user-id and then continue routing). All I'm advocating for is that you somehow create a routing-graph in a logical manner and then make the decisions in the component that is responsible.

  

A large concept that this article also ignores is middleware. The decorator pattern is quite a powerful one and is facilitated by nearly every 3rd party mux implementation. Top level support for middleware makes adding decorators to some, or all, endpoint rendering resources  an easy task regardless of the specific resource graph traversal required to activate them. The ability to take a single purpose, well tested endpoint and wrap it in other single purpose, well tested functionality (such as logging, stats, tracing, retries, backoffs, circuit breaking, authentication, etc.) without modifying the core logic of the endpoint is a large value add. It's unclear how the "resource as a router" model could easily provide such a feature. This is not to say it's impossible, I simply haven't seen it done well outside the mux model before.

It is true that I largely ignore that. So far I've assumed that you likely want to have middleware near or at the top of the routing tree, so you'd end up wrapping your handler as a whole.

There are ways to integrate middleware for subtrees, though. If you forego the advantage of easy static analysis (which you'd inherently have to, with middleware) you could use http.Handlers in the individual routing notes, instead of concrete types. Alternatively, you could add a `Middleware func(http.Handler) http.Handler` to your nodes, defaulting to a nop.

These do require cooperation from the nodes (in that they need to provide the ability to hook things into subnodes), though. I find that realistic, given that it's one application, but if you heavily rely on third parties for individual nodes (think /debug/* handlers from the stdlib) this would be a problem.

Depending on the complexity of the graph and your needs an easy solution is probably to just combine the two approaches; create a resource graph with concrete nodes and easy, statically deducible traversal for routing the request and wrap it with a mux, to inject middle-ware. As the mux only needs to do matching, not routing, it can be a lot simpler.

I agree, that this needs some thinking. Personally, I don't rely heavily on middleware, much less deep in the routing graph, so this isn't a huge concern for me, personally. But it should get a better answer at some point :)

 

On Sun, Jun 18, 2017 at 5:02 PM 'Axel Wagner' via golang-nuts <golan...@googlegroups.com> wrote:

Hey gophers,

in an attempt to rein in the HTTP router epidemic, I tried writing down a) why I think any router/muxer might not be a good thing to use (much less write) and b) what I consider good, practical advice on how to route requests instead. It's not rocket science or especially novel, but I wanted to provide more useful advice than just saying "just use net/http" and haven't seen that a lot previously.

Feedback is welcome :)

http://blog.merovius.de/2017/06/18/how-not-to-use-an-http-router.html

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Egon]

I've also used this approach for very simple endpoints:

func ServeHTTP(w http.ResponseWriter, r *http.Request) {

type rr struct{ method, path string } // rr = "resource request"

switch (rr{r.Method, r.URL.Path}) {

default:

http.Error(w, "Invalid request.", http.StatusBadRequest)

retun

case rr{http.MethodGet, "/"}:

// handle index

case rr{http.MethodGet, "/favicon.ico"}:

// serve icon

case rr{http.MethodGet, "/list"}:

// serve list

case rr{http.MethodPost, "/save"}:

// ..

}

}

Of course this can be combined with getting the ShiftPath and other things, when necessary.

+ Egon

[fusi.enr...@gmail.com]

Hi Axel

I agree on the suggestions you give on the article, even I have another view of the "epidemic".

To be short: almost none of the "Basic/Simple/lightweight http rest/mux/routers/socallframeworks" is supposed to have a real usage.

Sometimes I have the task (I'm the lucky guy) to do technical interview with people applying for my company, and I see that almost EVERYBODY

has , in the resumee, a "personal project" to show. Seems good? Uhm....

In my experience, ~80% of that projects are "Basic something", "Simple whatever",  piece of code. Meaning they are very primitive, incomplete, not ready, but.... they  create a new entry in the Resumee or Linkedin. 

Nothing I would call "a product", neither close to it.

Lot of Resumee also mentions how many forks their projects has, so the best way to have forks is to write "libraries" and "API".  So that, some softwares in use by the HR are confirming "sure, this guy is very famous for his Basic HTTP proxy". (Until you don't try to use it, I mean. But no software in use by the HR is able to do it. Hope the AI will, someday.).

I am not a developer (I work as a system architect), nevertheless I find golang very good for prototyping.  Many times I've  hit my head on those "libraries", "so call frameworks" and "API". 

100% of times I implemented my POC following instructions and documentation (when given. ~90% has no decent documentation),  just to realize that the "framework" was barely able to fit 1/2 use-cases an inexperienced Junior was able to imagine. So I ended up to rewrite everything from scratch. Fortunately golang has this amazing feature of Interfaces, so I just needed to change "the meaning" of a specific call.

Anyhow, I think you are worrying for code which is not written to be useful, neither utilized, and not even taken in consideration  : is written for some Human Resource office here and there.

According with github, there is a plenty of good software written in golang. When you exclude the "Simple", "Easy", "Basic" stuffs, (=unfinished/incomplete) you find an amount of "ready+complete" software which is still impressive, but....  much smaller.  

The epidemic of "simple this/easy that/basic whatever" , regarding API / libraries /proxy /routers   (written in golang or others)  will not finish even if/when you convince everybody this is not useful:  none of that software is supposed to be useful. 

They are just another line on the Resumee/Linkedin. 

Just my two cents, of course.

Besides, nice article.

FEM

[mhh...@gmail.com]

hi,

I understand what / why you came to that given your explanation,
but I disagrees a lot to it.

I don t feel like expanding code helps to better explain
the how/what/why of a sudden 42 question ? (why this /whatever/ has failed, I believe it should not)

I m pretty sure its a good way to multiply keystrokes, and harden maintenance and changes.

In the example you taken, because its not representative of all 42 questions** (only few ones),
the solution is to provide a more speaky app.

In your post you take that route example,
"/articles/{category}/{id:[0-9]+}"

And later expand and rewrite it to return a proper error when an id is not an int,
Behavior that is totally excluded at the moment you decided
to apply a pattern on your route.

You are comparing things that are not comparable,

the route can be
"/articles/{category}/{id}"

The gorilla handler can handle the type checking and return a proper error code.

So, you can resolve the 42 question,
not by diving in the code,
but by reading the error messages,
or reading at the documentation.

(note: reading the error messages=> totally fail when you develop a website, not an api, as the error page is often generic and do not provide a deep explanation of the error, as the end user does not care about it, sometimes you can trick about it (put that in some comments/whatever/) but any serious security test team will warn you about it)

Or that /foo/123 is, technically, an illegal argument, not a missing page. I couldn't really find a good answer to any of these questions in the documentation of gorilla/mux for what it's worth. Which meant that when my web app suddenly didn't route requests correctly, I was stumped and needed to dive into code.

None of the solution compared here, ie gorilla vs plain code, manages correctly this feature.
In both case its an additional job to be done, in one way or another.

Somewhere you also talk about the idea that using a router would tightly link
components by their types, adn that is bad in your opinion.

In my opinion, I m very happy about tight types coupling.
Like very very very very very very very happy about that.

Its the only thing i can rely on
to apply an understanding and validates that things gonna work
without being able to provide a formal proof (if i could, tbh).

Type here can be an interface or a value, that does not matter, their guarantees matter.


Something i truly agree with you,

They only need to know about the immediate handlers they delegate to and they only need to know about the sub-path they are rooted at

Local problem solving is always way easier to handle
than large scaled problems across files/packages/concepts.

In that regard i do agree with you about the last proposal you give in its intentions,
I clearly disagree in its implementation,
it just not gonna work for me to write that much of code to solve those problems.

** the other kind of 42 questions is about functions that behaves differently from what the user expects.
Which happens in few cases, wrong documentation from author, misreading from consumer, and at least that third one, a name which is both too vague and too precise to give sufficient meaning about its purposes by itself,
example

/whatever/.Each() /whatever/ {...}

You can t tell what s going to happen in Each (will that call allocate?) without knowing what is /whatever/,
if whatever is an array, you can deduce there will be an allocation,
if its something else like a stream, you can deduce it won t.
Because the name is not sufficient to describe its meaning on a so vague / general context,
the problems takes scale, thus have an air of mystery.

That happens only with some kind of names,
as soon as you enter a specific context (what mean to Put to a dht table ?),
you gonna need to learn the specifics of the problem before,
by definition the local question is scaled to the context of your general situation (something like this),
naturally its has an air of mystery,
naturally you need to learn and assert about the details (via the code or the doc).

_________________

about the question, why there s so much alternative routers.

My personal opinion, gorilla mux is great, but its an enter step for many people,
so they need to learn both the language + the framework + quickly they need to assimilate
some pretty fine tune trick of the language to keep going,
and so it become difficult, and so alternatives has a reason to happen.

Also some constructs, while being very verbose, are not obvious...

        http.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.Dir("./public"))))

Super verbose.

So clear that we need stackoverflow to make it happen the first time.

Another possible reason, as all in things in golang,
it has to be verbose,
we ain t all like that,
easy path for a developer to think that i can fix that.

And no, i m not saying verbosity is bad,
i m saying that verbosity is not affordable,
it is super expensive when most of the code produced out there is trash code,
or consumable code if you prefer to be more politically correct.

When a project succeed
the code is created, consumed/maintained for a while,
then replaced.

Put simply we don t all work on a programming language
that s going to live for decades (its the best i hope for golang).

If i d try to resume that in one (or two) statement(s),
verbosity is not simplicity ... simplicity is complex.

So,  how to solve the epidemic ?
IF you could, that is by providing a package that answers all POV with 100% matching,
but you can t really,
at best you can only provide a choice that is favorable to the re use Vs re create most of the time,
at the end, its always the user that balances the benefits and the costs from his POV,
assuming he passed the technical barrier.

Last dump:
https://fr.wiktionary.org/wiki/l%E2%80%99enfer_est_pav%C3%A9_de_bonnes_intentions

[Axel Wagner]

Hey,

On Mon, Jun 19, 2017 at 12:43 PM, <mhh...@gmail.com> wrote:

I don t feel like expanding code helps to better explain
the how/what/why of a sudden 42 question ? (why this /whatever/ has failed, I believe it should not)

I may be biased in this regard by personal experience. I am running (as an ops-person) several large web service written in Java and C++ (and some written in go) and I am pretty consistently frustrated by the lack of good comprehension tools for routing. I very often run into the situation of, for example, a black-box monitoring system telling me that a request failed and needing to understand how it traces from the main entry point of the server to the failure site and how all the parameters used are chosen. Or developing against a service and needing to understand what I am doing wrong when one of my requests is failing. Or just getting paged because we are serving errors and needing to grep through logs for patterns in the failing requests and then figuring out why these specific requests are the ones that are failing.

I talking about the concern of tracing routing through code not because it's fun to me to make up problems, but because it's a frequent source of frustration for me.

 

I m pretty sure its a good way to multiply keystrokes, and harden maintenance and changes.

I don't think it is either more typing or making maintenance harder, obviously :) For the latter, I already described my reasoning in the post itself, so I'm not going into detail. For the latter, I gave an example here, of how you can reduce the number of keystrokes necessary, if that's your primary concern while maintaining the same pattern and the advantages I'm talking about.

I don't think you should do that, though; just as you shouldn't use test frameworks with assert-functions. If what you are doing is comparing values, you should write them as an if-clause, that's readable and what they are designed for. But if you really want to, you can do it. It's an orthogonal concern to what I'm talking about.

 

The gorilla handler can handle the type checking and return a proper error code.

But that ignores the issue. The issue isn't, that the muxer (let's not talk too specifically about gorilla, I only use it as an example) can't figure out that there is a non-int given as a parameter. The issue is, that by design (that is pretty much the point of a muxer), you can have intersecting sets of routes that might or might not match; the muxer will pick a strategy and try each route out in series, until it finds a match.

A request like "/foo/12a" would fail a pattern like "/foo/{id:[0-9]+}". But should the muxer, when it tries this pattern for this path return a 400, because the pattern is invalid, a 404, because there is no matching pattern defined, a 405, because you used GET instead of POST or should it continue trying? There might be another route to try, which defines a pattern for /foo/12a. Or a route that defines "/foo/{id:0-9a-z]+". Or any other set of routes.

Should it, when it exhausted all the configured routes, return the 400 of the pattern "/foo/{id:[0-9]+}" it tried, the 405 of the route Methods("GET").Path("/foo/12a") it tried or a 404?

I would boldy claim, that to solve this problem in general, a muxer API would need to basically be Turing-complete (which is exactly why this exists, by the way). In my view, this negates the supposed advantages of a muxer. Instead of calling the checks as Methods on the muxer, I can also just write them in my handler, in the component, they belong to anyway.

It's not actually more verbose or complicated than calling methods on a muxer-value, but it enables you to make an authoritative decision purely based on local knowlege, so it becomes a *much* simpler problem to solve. Plus, you actually *have* a Turing-complete language at your disposal.

The fact that it also makes the handler self-contained, so it can be maintained by a separate team than the rest of the application or reused in completely different applications is a then bonus.

 

So, you can resolve the 42 question,
not by diving in the code,
but by reading the error messages,
or reading at the documentation.

(note: reading the error messages=> totally fail when you develop a website, not an api, as the error page is often generic and do not provide a deep explanation of the error, as the end user does not care about it

Totally disagree here. HTTP is primarily built for web pages and provides well-defined status codes. A user might not need to care about the detailed reasons for a 500, but they definitely want to know, that this is an unforseen error condition in the server, as opposed to something they did wrong. They want to know, if they entered an invalid phone number, that the problem is the phone number they entered and not the database server not responding, the form using the wrong method or that they need to log in first. It's endlessly frustrating, as a user, to just get an "There was an error" message. It's not actionable; should I call a support line? Change the inputs I gave? Switch browsers?

It's just normal error-handling stuff; just like you want the message of your error to be sufficiently descriptive to a user wanting to know what changes they need to do to their usage to make your thing run, you want the details delivered to a visitor of your website to be sufficiently descriptive that they know what they could change to make it go (Do they need to login? Do they need to change what they entered in a form? Is a website not available in their language or does their browser not understand the format the server is sending?).

 

In both case its an additional job to be done, in one way or another.

Somewhere you also talk about the idea that using a router would tightly link
components by their types, adn that is bad in your opinion.

No, not by their types, just in general. The router needs to get a handle to all transitive dependencies. The issue even is the opposite: The router is generic, so it doesn't know anything about actual, concrete types, making static analysis hard or impossible.

[Marwan abdel moneim]

I am not sure, but i think you may find this interesting https://github.com/mrwnmonm/handler 

On Monday, June 19, 2017 at 12:02:37 AM UTC+2, Axel Wagner wrote:

[Steve Roth]

Hello, Axel,

I like your concept and I am applying it.  But I believe that the code in your blog post is broken.  In various places you call

head, r.URL.String = SplitPath(r.URL.String)

But r.URL.String is a function, not a property.  This code doesn't compile.  I think perhaps maybe you meant r.URL.Path?

Regards,

Steve

[Axel Wagner]

Yeah, I do. And that's what I get for not testing my code, this is the second mistake someone found :) Thanks for noticing it, I'll push a fix in a minute.

--

[prade...@gmail.com]

The goal of a (proper) router is to decouple routes from handlers, thus making refactoring easier by adopting a declarative form rather than an imperative one. It's the good old builder pattern. By deeming it unnecessary you did nothing but couple your handlers with your routes. That's the only thing you did. You didn't make your code easier to read, you cluttered it with unnecessary imperative logic that could be simply abstracted while remaining no less readable if not more. 

Your solution certainly doesn't get more readable as the amount of routes in an app gets larger. 

[Axel Wagner]

On Sat, Jun 24, 2017 at 11:43 AM, <prade...@gmail.com> wrote:

The goal of a (proper) router is to decouple routes from handlers, thus making refactoring easier by adopting a declarative form rather than an imperative one.

Yes. And it is my opinion that this is a bad thing.

 

It's the good old builder pattern. By deeming it unnecessary you did nothing but couple your handlers with your routes. That's the only thing you did. You didn't make your code easier to read, you cluttered it with unnecessary imperative logic that could be simply abstracted while remaining no less readable if not more. 

I think I also showed pretty clearly, that there is no actual difference in the level of abstraction. You can, if you want, create a 1:1 correspondence of LOC; the difference just being, where they are.

TBH, I rather think that the abstraction is cleaner if the handler does its own routing. That way I can simply take it and plug it into a different web-app at any path I like.

Say my RPC library provides an HTTP interface for debugging. It has a couple of pages with html-forms, that you can input your requests and see the resulting responses, and the like. What I'm proposing is, that this handler makes the promise to handle `/` and do the routing for all those forms and subpages (including static assets) relative to that. That way I can then, in my app, put it behind an authenticated `/debug` sub-path, just by adding an if-statement.

Think about how the muxer-pattern would do that. Would I need to manually set up a bunch of routes? And if they changed, would everyone who imports that handler need to change their routes? Would the constructor take a muxer to set up it's own routes? If so, which muxer and would libraries now have a dependency for your app to use a specific muxer?

What you'd probably end up with as the best solution is for the constructor of that handler to set up its own muxer, wire up the routes and then call into that in ServeHTTP. And because it can't know where the user wants the http-endpoint end up in the final app, do it relative to / (or, alternatively, take a prefix. The difference doesn't really matter for my point). But that's pretty much exactly, what I suggest already.

So yes, the difference comes exactly down to what you say; whether you use an imperative, Turing complete language to make the routing decisions. Or whether you take a "declarative" DSL, embed it into go and hope it's powerful enough to express all the routes you want to take. And I think, there is a strong argument to be made that the abundance of existing routers shows that there *isn't* any one implementation of such a DSL powerful enough to fulfill everyone's needs. That this nice abstraction that you are talking about just doesn't exist and will always be leaky.

It is also my opinion (but this comes down to taste), that imperative code is easier to read and always easier to debug and reason about (that is pretty much the reason I love go; source code maps very closely to computation). But even if you disagree about that, you can still write exactly the same kind of declarative code that you are talking about; just doing it in your Handler, instead of as methods on a muxer. Just add a couple of helpers like

AcceptMethods(res http.ResponseWriter, req *http.Request, methods ...string)

AcceptPathPrefix(res http.ResponseWriter, req *http.Request, prefix string)

AcceptContentTypes(res http.ResponseWriter, req *http.Request, contentTypes ...string)

 

Your solution certainly doesn't get more readable as the amount of routes in an app gets larger. 

We just have to agree to disagree on this one :)

 

Le lundi 19 juin 2017 00:02:37 UTC+2, Axel Wagner a écrit :

Hey gophers,

in an attempt to rein in the HTTP router epidemic, I tried writing down a) why I think any router/muxer might not be a good thing to use (much less write) and b) what I consider good, practical advice on how to route requests instead. It's not rocket science or especially novel, but I wanted to provide more useful advice than just saying "just use net/http" and haven't seen that a lot previously.

Feedback is welcome :)

http://blog.merovius.de/2017/06/18/how-not-to-use-an-http-router.html

--

[Kean Ho Chew]

IMO, this article itself is indeed indicating there is an important feature not fulfilled by the standard package. It is asking people to write their own routers indirectly, thus, leading to the current situation (e.g: more routers). There is a much better guide here: https://stackoverflow.com/questions/6564558/wildcards-in-the-pattern-for-http-handlefunc. Judging the fact that I need to roll out my own router (which is also a pain), it's much better for me to spend the resources on Gorilla/mux that has the collective contribution effect from its contributors pool. Hopefully the developer in the standard package can see this and look into it (We had enough with Python2 vs Python3 incident already).

p/s: I'm a new comer (transition from C and Python) to Go but due to the fact that I need just a simple dynamic parameter url feature that the standard package couldn't offer, it led me into researching about this "router" markets which is kind of disappointing from both personal and commercial projects perspectives. (So much time wasted studying these router studies.)

Kean

[Axel Wagner]

On Mon, Dec 18, 2017 at 5:50 AM, 'Kean Ho Chew' via golang-nuts <golan...@googlegroups.com> wrote:

IMO, this article itself is indeed indicating there is an important feature not fulfilled by the standard package. It is asking people to write their own routers indirectly

I disagree. I am making the argument that the concept of a router is inherently broken. Writing your own router doesn't improve on that. Don't write a router, write routing logic.

Or, if you prefer: Instead of writing a domain specific language (in the form of a single router type) for routing which is general enough to cover any and all usecases, just use the existing language to express the logic. As the graph of routes is the same and needs to be written down either way, you end up with effectively the same amount of code, but it is more readable, more understandable, more localized and overall clearer. IMHO.

p/s: I'm a new comer (transition from C and Python) to Go but due to the fact that I need just a simple dynamic parameter url feature

The post specifically mentions how to do this.

You don't have to heed my advice, of course. Feel free to use gorilla/mux or any other router, if you prefer. I just wanted to give the people who want to know how to get by with the stdlib the tools necessary to do so :)

that the standard package couldn't offer, it led me into researching about this "router" markets which is kind of disappointing from both personal and commercial projects perspectives. (So much time wasted studying these router studies.)

Kean

On Monday, June 19, 2017 at 6:02:37 AM UTC+8, Axel Wagner wrote:

Hey gophers,

in an attempt to rein in the HTTP router epidemic, I tried writing down a) why I think any router/muxer might not be a good thing to use (much less write) and b) what I consider good, practical advice on how to route requests instead. It's not rocket science or especially novel, but I wanted to provide more useful advice than just saying "just use net/http" and haven't seen that a lot previously.

Feedback is welcome :)

http://blog.merovius.de/2017/06/18/how-not-to-use-an-http-router.html

[Kean Ho Chew]

I disagree. I am making the argument that the concept of a router is inherently broken. Writing your own router doesn't improve on that. Don't write a router, write routing logic.

At some point you might have to agree with me. By abstracting the routing logic into a library to 'go get' in the future, it will become a new one like Gorilla/mux. Example, it took me a while to write the dynamic parameter that I'm looking for (https://gitlab.com/holloway/golang-web-tutorial/blob/master/sockets/rest/standard/standard.go#L27). This is rather a simple feature. If the request is being overlook since the beginning, I don't think the router epidemic will come out at the first place. (Unless, I'm not aware that Go is on a mission to simplify the web url, like taking the dynamic url out the game.)

Due to my personal learning requirement sticking to the standard library, I ended up exploiting the handleFunc "/" instead, which yielded a different code pattern for server side; far from any guides that I learnt, neither the boilerplate in the post.

Or, if you prefer: Instead of writing a domain specific language (in the form of a single router type) for routing which is general enough to cover any and all usecases, just use the existing language to express the logic. As the graph of routes is the same and needs to be written down either way, you end up with effectively the same amount of code, but it is more readable, more understandable, more localized and overall clearer. IMHO.

It's really depends on the objectives. Yes, I agree with you that writing codes that fosters linear reading instead of hopping around libraries greatly helps in debugging but it also contributes a whole new level of scaling problem (reusability). Keep in mind that the goal of creating library is to ensure re-usability that leads to battle-tested codes. It's a matter of striking the balance between those 2 constraints.

p/s: I'm a new comer (transition from C and Python) to Go but due to the fact that I need just a simple dynamic parameter url feature

The post specifically mentions how to do this.

You don't have to heed my advice, of course. Feel free to use gorilla/mux or any other router, if you prefer. I just wanted to give the people who want to know how to get by with the stdlib the tools necessary to do so :)

Well, I took both paths but I prefer my own parser ;-). There are still some works to do with the new pattern but that comes after when I learn how to create a library and start splitting the 1 page source codes into a manageable format. Keep up the education though. You did a good job.

p/s: The sole reason I voiced out, mainly because I don't want such a wonderful, artistic and masterpiece language heading towards the Python2 vs. Python3 war direction. This language is majestic and beautifully crafted for modern computing that one who been through the pain from Java, both Pythons, C, Embedded C and Ruby on Rails can really appreciate its beauty.

Peace,

Kean

 

 

[Axel Wagner]

On Tue, Dec 19, 2017 at 4:52 AM, 'Kean Ho Chew' via golang-nuts <golan...@googlegroups.com> wrote:

I disagree. I am making the argument that the concept of a router is inherently broken. Writing your own router doesn't improve on that. Don't write a router, write routing logic.

At some point you might have to agree with me.

"I am right, so you are just going to have to agree with me at some point" is not a very healthy base for discourse.

 

By abstracting the routing logic into a library to 'go get' in the future, it will become a new one like Gorilla/mux. Example, it took me a while to write the dynamic parameter that I'm looking for (https://gitlab.com/holloway/golang-web-tutorial/blob/master/sockets/rest/standard/standard.go#L27). This is rather a simple feature. If the request is being overlook since the beginning, I don't think the router epidemic will come out at the first place. (Unless, I'm not aware that Go is on a mission to simplify the web url, like taking the dynamic url out the game.)

This whole section is ultimately circular: "Routers are good, because Routers are good".

You are arguing a) if you write your own Router, it will grow in features until it ultimately competes with other Routers. Yes, literally the point of the article. b) Routers implement a whole bunch of complicated code, to provide a DSL for routing (that "dynamic parameter thing" you are talking about) - literally the point of the article. c) It's a simple feature - but it's *not*. It's a super-duper-complicated feature (as evidenced by how much trouble it was to implement it just for your own, application-specific usecase) to replace something completely simple, which is something to the effect of userid := ShiftPath(req) (or whatever you want to use that for) at the correct spot of your code.

The whole premise of this section is that you'd need to have a central, programmable component that takes as input a URL and as output some sort of representation of all its routing decisions (in the form of parameters and the like) and as output a specific location to jump through - which will then have to introspect this representation of routing decisions and reverse-engineer them, to figure out the pieces it needs.

I'm arguing, that you should instead just get rid of that component, do the routing piece-by-piece, with golden variety control flow structures. You use the pieces of the URL you are making your decision on (the "dynamic parameters" you are talking about) at the place where you are extracting them instead of separating them out and trying to find a generic representation. And you end up with less total code, a comparable amount of self-written code and, most importantly, completely easy to read code. Because it's just regular control flow, like in any other Go program too.

Try to, at least for the sake of argument, get rid of the premise that there needs to be a central, programmable component - a router.

It's really depends on the objectives. Yes, I agree with you that writing codes that fosters linear reading instead of hopping around libraries greatly helps in debugging but it also contributes a whole new level of scaling problem (reusability). Keep in mind that the goal of creating library is to ensure re-usability that leads to battle-tested codes.

I posit, that my code is far more reusable than any code relying on any muxer out there. Having Handlers do their own routing is a prerequisite of re-using them elsewhere. By coupling them to a muxer, you are walling yourself off from people who don't use one or use a different one.

For example, a common complaint from people is, that a bunch of stdlib-packages will register themselves on http.DefaultServeMux under /debug. This is an incarnation of exactly that problem: These packages where written under the assumption that there has to be a central routing component that dispatches all URLs, so they decide, for themselves, that they want to be under, e.g. <domain>/debug/pprof/heap. Which is not scalable (as it creates the risk of collisions), not reusable, not well-isolated.

Contrast that with a world where they just provide an http.Handler that just assumes it's handling / and then does the routing below that itself. As a user of that library, you decide whether you want to handle it on /debug/pprof/heap, on /heapz or on /_/admin/heapdump. You strip whatever prefix you chose before dispatching to pprof.Handler and it does all the routing it needs for itself. Much more scalable (no collisions anymore, as you can just change the URL in case of a collision), much more reusable, much better isolated (suddenly, the user actually knows what URLs go where, instead of magically getting URLs registered).

You should at least try to consider that having your routing logic in a single value might not be a good idea. Yes, there certainly could be things we can do to make it easier to do the routing in-Handler (for example, we could probably use an API to determine absolute URLs to link to, from a handler), but writing fifteen hundred DSLs to express something Go already can express just as well is not the solution.

The basic arguments, again, are a) Having a Router doesn't actually save you code in a significant way, because you are replacing a conditional with a function call and b) instead you are pulling a whole lot of unnecessary code into your program, that implements a DSL to express the routing control-flow; but which is less powerful than the Go code it replaces. The argument isn't "write your own Router vs. use an existing one", it's "not have a Router vs. write your own", because it seems the latter is, where the idea of Routers has lead us.

 

It's a matter of striking the balance between those 2 constraints.

p/s: I'm a new comer (transition from C and Python) to Go but due to the fact that I need just a simple dynamic parameter url feature

The post specifically mentions how to do this.

You don't have to heed my advice, of course. Feel free to use gorilla/mux or any other router, if you prefer. I just wanted to give the people who want to know how to get by with the stdlib the tools necessary to do so :)

Well, I took both paths but I prefer my own parser ;-). There are still some works to do with the new pattern but that comes after when I learn how to create a library and start splitting the 1 page source codes into a manageable format. Keep up the education though. You did a good job.

p/s: The sole reason I voiced out, mainly because I don't want such a wonderful, artistic and masterpiece language heading towards the Python2 vs. Python3 war direction. This language is majestic and beautifully crafted for modern computing that one who been through the pain from Java, both Pythons, C, Embedded C and Ruby on Rails can really appreciate its beauty.

Peace,

Kean

 

 

--

[roger peppe]

> You strip whatever prefix you chose before dispatching to pprof.Handler and it does all the routing it needs for itself.

It would be great if that was actually a viable approach in general,
but unfortunately it's not, because it's not uncommon to need to know
the absolute path that's being served, which is lost when using this
technique. One example is when you need to form a relative URL to
another part of the name space (you know absolute path of the
destination, but you can only create a relative URL path if you know
the absolute path being served too).

> a) Having a Router doesn't actually save you code in a significant way, because you are replacing a conditional with a function call

It might not save *much* code, but it's generally just boilerplate,
and every conditional is another condition that can be wrong.

> b) instead you are pulling a whole lot of unnecessary code into your program, that implements a DSL to express the routing control-flow

Sure, there's an argument to remove *any* external dependency - there
are costs and benefits here, and they need to be evaluated for every
project.

Personally, I think there's room for a hybrid approach - routers,
particularly structured routers such as
github.com/julienschmidt/httprouter, can make the code more obvious,
maintainable (and probably faster too), but there's nothing stopping
you from combining that kind of routing with custom routing code for
some sub-paths.

Using a router DSL can have other advantages. If you express things
that way, the routes become amenable to programmatic analysis. We're
using that to automatically generate API client code, for example.

[Sanjay]

I also think its a bit of a false dichotomy; I use a hybrid approach, where I have a server's public interface be a http.Handler, but its internal implementation of ServeHTTP uses a router to dispatch to several methods on the server.

https://play.golang.org/p/-W4tHmiUve is a stripped down example from a real server I wrote at some point.

I think this is nicely layered; you can have different components of your server use completely different routing logic if you want (say for portions of the server contributed by different people/teams; the pprof endpoints would be an example of this), and then its all just registered in main() generally using good old http.ServeMux for hostname-routing (perhaps getting the hostname itself from a flag).

Sanjay

[Kean Ho Chew]

The basic arguments, again, are a) Having a Router doesn't actually save you code in a significant way, because you are replacing a conditional with a function call and b) instead you are pulling a whole lot of unnecessary code into your program, that implements a DSL to express the routing control-flow; but which is less powerful than the Go code it replaces. The argument isn't "write your own Router vs. use an existing one", it's "not have a Router vs. write your own", because it seems the latter is, where the idea of Routers has lead us.

My bad, we should focus on "not having a router vs write your own". Keep in mind that I'm only pointing out the resilience of the standard library team developing your "ShiftPath" or my "parse_url" is the primary cause of why people charging towards writing a new routers. In my next refactoring, it is either I roll out the a new parser library or a new router. The latter makes more sense since it simplifies the code.

Just look at the httprouter source code example (https://astaxie.gitbooks.io/build-web-application-with-golang/de/08.3.html) vs my creation (https://gitlab.com/holloway/golang-web-tutorial/blob/master/sockets/rest/standard/standard.go). Personally, in large scale project, I prefer the former due to cleaner code. So that 2 months later when I returned from another large scale project, I can easily relearn what I did. The latter requires new considerations:

1. I need educate every single developer in the project about the structure I'm using (MVC? no. Looks like MVVM? maybe, let's call it MVVMVC?.).

2. It's so easy to have inexperienced player to screw things up because it's something new.

3. Pray that I'm not encouraging some hot-headed developers rolling out their own in the company.

It boils down to:

1. Is it worth the cost for sticking to standard libraries by creating an entire architecture and roll out a new set of educations?

Learning:

As I said earlier: striking the balance - just as others said: hybrid. Not all self-written artwork is good, neither abandoning the collaborative efforts help either. Objective of the development should be given as priority one. If I'm given a 1 day deadline, I'll still use Gorilla/mux router (skip the reeducation). If I don't face any time constraint, I will use my refactored parse_url / your shiftPath method as I clearly can see the performance by just following the flows (unlike the mux).

If we're going to be religious about "doing one thing right", which is using standard library, as least, don't leave your tail for others to pull. This whole standard library is clearly missing something for the web development crew, something crucial yet simple: such as the parser feature both of us are looking for. You do not need to use regex for the DSL parameters like Gorilla/mux did because that introduce double validations instead. Yet, you still need to work with pattern such as: /Articles/{aid}/Comments/{cid}/likes, which AFAIK, httprouter can't handle (correct me if I'm wrong; one of my research point leads to here: https://husobee.github.io/golang/url-router/2015/06/15/why-do-all-golang-url-routers-suck.html).

p/s: I'm amazed by people bringing {id [0-9]+} regex validation into DSL parameter to complicate simple things. Eisenstein was right: simple is good, not simpler. 

Regards,

Kean

[Kean Ho Chew]

 Yet, you still need to work with pattern such as: /Articles/{aid}/Comments/{cid}/likes, which AFAIK, httprouter can't handle (correct me if I'm wrong; one of my research point leads to here: https://husobee.github.io/golang/url-router/2015/06/15/why-do-all-golang-url-routers-suck.html).

I've tried and tested out httprouter as guided (https://gitlab.com/holloway/golang-web-tutorial/blob/master/sockets/rest/httprouter/httprouter.go). Correction to my statement above, httprouter can work with the mentioned pattern. Using httprouter produces:

1. A lot more cleaner code

2. Its the library is straight and easy to read as well (It took me 15 mins to understand the library compared to 6 hours of research on self-invention as guided in the blog)

3. It demonstrates a clearer MVVM pattern (not framework) so at least it helps in communications and GoLang new comers.

Hence, I'm now officially throwing away my tested but not benchmarked parser_url and aligned with the httprouter team, mainly due to its benchmark and already tested codes. This exploration was an interesting experience and thanks all.

Signing off,

Kean