Routing performance

[Christopher Lord]

Hello,

I'm interested in the routing latency of various web frameworks. Given a complicated web app with a very deep and wide tree of possible paths, which web framework responds quickest to a given request?

For simple apps with few paths, latency will be low, but for quite complicated applications a framework that does a simple linear scan through all of the paths will have a larger variance than one that uses some sort of tree mechanism. 

Is this already being tested? if so, how can I access this information? If this is not part of the test, perhaps it's a good test to add to the suite? 

The test could be something along the lines of "given N unique pages organized into a set of nesting paths (/a/b/c, /x/1, etc), how fast (and with what variance) can you give me a response to all of those paths?" A uniform distribution of queries is interesting, but so is one that favors a subset of the paths (perhaps the framework can optimize routes after observing user behaviour?) 

[Brian Hauer]

Hi Christopher,

Thanks for the message!  It's an interesting topic you raise.  To date, our tests do not exercise the incremental impact of additional routing complexity except incidentally and in very small numbers as we add test types to the project.

I've added a link to your idea to the relevant issue at GitHub:

https://github.com/TechEmpower/FrameworkBenchmarks/issues/133

Implementing this test would require the use of an alternate benchmark tool or continued enhancement to Wrk by its author.  Presently, Wrk exercises a single URL.  It won't be trivial, but I assume we can figure out some approach for exercising a set of URLs via the benchmark tool.

It's premature without having that worked out, but as an academic matter, what size N do you think would be interesting?  In my own experience, I've not built web applications with more than a few hundred routing endpoints.  But perhaps N=1,000 would be a decent test?

[Christopher Lord]

Thanks for the link to the issue, and for the details on what would be required. 

I was originally thinking N=5000, which should show the difference between O(N) and O(lg(N)) routing. but N=1000 should work too (same order). At first I can't imagine an app with such complicated pathing, but remembering some of our internal Katamari-style apps, that doubt goes away.