Partial HTTP requests

[Jonathan Azoff]

Just a total brain bomb here, but I thought I'd kick it around to see
if I can get some feedback -

Basically, I have a proposition to extend the HTTP spec that could
make partial page updates work better with URL modification. However,
before I talk about my proposition, let me first try to explain the
problem. Currently, if we want to do partial page updates, it seems
easy enough to do: just open up an AJAX request and swap in the
partial content. However, if we want to change the URL cleanly, it
seems like our options are limited to using the new HTML5 pushState
functionality or one of the many "hash bang" hacks in the URL. For all
intensive purposes, libraries and polyfills have abstracted this
problem to the point where its not so much of a problem but more of an
annoyance (extra stuff to be downloaded to ensure graceful
degradation). In the end, the solution for a clean partial update in
the URL and the DOM seems to live in the browser.

My suggestion is to move this solution to the server, where adoption
isn't as hard to force. Essentially, extending the HTTP language to
include a new verb (say PARTIAL) would be pretty sweet. In this
hypothetical scenario, the same HTTP requests to the server can result
in full or partial content being returned. Now I know what you are
thinking, "I can totally do this without messing around with HTTP",
but now let's consider the scenario where the browser adopts this.

In the scenario where browsers start to adopt "partial" requests, you
can imagine a handshake similar to how CORS works. For instance:

- Browser sends HTTP GET / to example.com
- After loading the markup for example.com, the user clicks on an
anchor tag formatted like such
- <a href="/subpage.html' target="#content">Sub Page</a>
- The browser looks for a frame with that ID and cannot find it
- Typical browsers will just open up a new tab at this point, but
let's assume that this browser implements partial HTTP and now it goes
down a path similar to CORS
- The browser sends an HTTP OPTIONS request to /subpage.html asking if
the server supports partial HTTP requests for #content. for instance,
it could send headers:
- Partial-Content-Selector: #content
- If supported the server could respond with affirmative header
validation for that element, or all elements. For example:
- Partial-Content-Set: #content, .footer, .navigation
- Now the browser can make an HTTP PARTIAL request to /subpage.html
and insert it's response into the DOM under the correct selector.
- For completeness, I suppose the browser could even fire off an event
indicating that the DOM has changed, so that behavioral code may be
applied to the new content.
- Finally the browser changes the URL to /subpage.html, all without
any JavaScript magic to coerce the URL!

This seems obvious to me, and I wouldn't be surprised if there is
something similar in the works that I may have glossed over. I
especially like it because it makes modern web apps a bit cleaner, and
gracefully degrades to regular navigation for older browsers. It also
aligns developers with developing sites in an accessible way, lending
their sites to better indexing and so forth. Anyway, I'm interested to
see if this makes sense to anyone else, and if there is anything else
I can do to help.

- Jon

[Paul Irish]

Just to add prior art, pjax is a way of doing something similar.

https://github.com/defunkt/jquery-pjax  see the "pjax on the server side" section.

As to the solution proposed here, this basically means all web servers supporting PARTIAL would need to implement html5lib. That alone I think will make this a pretty tough sell.

I do like your fallback scenario, though :)

[Jonathan Azoff]

yup, just took a look at pjax and it is totally what I was thinking. I imagine pjax would be the polyfill for HTTP PARTIAL.

[Scott González]

I love this type of behavior, and have written code to handle this plenty of times, but I really don't think this is a good idea to add to HTTP. Even with 100% support in browsers, this will not remove the need for nearly identical code in JavaScript.

- Sometimes you don't want to update the URL (the changes aren't significant enough to warrant a history entry).
- Sometimes you want to perform animations or provide other visual feedback during loading.
- You still need to handle the full array of HTTP response codes, often with custom JavaScript anyway.

I'm sure all of the common use cases can be handled by a spec, but the size of the API will be much larger than what you're proposing. Even common scenarios can get really hairy like a session timeout prior to a partial page load. I also don't believe there's anything that a new API would be able to handle that isn't currently able to be done with JavaScript and XHR.

[Jonathan Azoff]

Thanks for the feedback scott! I don't think that this would really be
worth anyone's time unless someone started the debate :)

I think you have very solid points - namely, the point about the
expanding complexity of such a problem. I agree that if you try to
cover every use case with a spec, not only will the spec be complex,
but so will the implementation. My justification has more to do with
the first sentence of your response more than anything else:

"I love this type of behavior, and have written code to handle this
plenty of times"

*That* is exactly my point: *everyone* who has had to build a realtime
app once in their career has had to do this *at least* once. Now,
while I can't expect everyone to have the same opinion as me, my
personal opinion of specifications is that the best ones are the ones
that are based on adoption. When a browser implements an off-spec
feature that gets popular adoption, and then is retroactively added to
the spec - I think it's a win for everyone involved. Granted, there
are times where this failed (ahem, microsoft...), but when the
rationale is based on adoption - instead of agenda - I think
retroactive adoption is warranted and beneficial.

Hence, even if HTTP PARTIAL doesn't solve all the problems involved in
modern web apps, just being able to write less code because a common
problem solved at the browser/server level is a *big* win. Complexity
will always creep in, and specs will have to evolve.

Degenerate gamblers have a common saying, "A win is a win is a win" -
I'll take my profit, even if it is small.

[Scott González]

On Fri, Jan 20, 2012 at 2:24 AM, Jonathan Azoff <jaz...@gmail.com> wrote:

*That* is exactly my point: *everyone* who has had to build a realtime
app once in their career has had to do this *at least* once.

I bet the percentage is much lower than you think. Many developers that require this logic will either use a framework or library to help them out, solving many problems for them, not just one. Some will go with a solution specific to this problem, such as pjax. Some will never have this problem because they only work with JSON APIs and they never have partial page loads that are based on HTML anyway. I think the win is extremely small.