media_type condition

Ted Milker

unread,

Aug 13, 2013, 4:10:11 PM8/13/13

to scor...@googlegroups.com

What's the point of this condition? How does it work? I've tried everything I can think of(including things I came up with after digging into rack-accept's source and reading the section of the RFC) and I can't make it not match every request. As far as I can tell, because browsers include */* as an acceptable media type for requests, every media_type condition will always match any request. Even when I throw junk in for the media_type to match, it still comes back true.

From the 'First Impressions' example, I thought the point of it was to separate routes for browser page requests from something like a jQuery $.getJSON() but that doesn't seem to be the case.

Tom Wardrop

unread,

Aug 13, 2013, 6:51:55 PM8/13/13

to scor...@googlegroups.com

You're right that browsers will (and should) match any media type, because they accept every media type. But that's not a problem, and here's why.

In the case of jQuery and every other JavaScript library, a call to retrieve JSON or XML from a web server will always include the corresponding accepted media type in the request headers, and usually nothing else. This is exactly the type of scenario the `media_type` condition is intended for. The difficulty you're experiencing is due to the ordering of your routes.

If you have a resource that is available in both HTML (for the browser) and JSON (for REST) media types, here's how you'd serve that:

get '/data', media_type: 'text/html' do 
  <<-HTML
  <!DOCTYPE html>
  <html>
  ...
  </html>
  HTML
end

get '/data', media_type: 'application/json' do 
  {name: 'blah', occupation: 'bleh'}.to_json
end

HTML has to come first, otherwise browsers will always match the JSON route. If a request comes through that matches neither, they'll get an empty response with a 406 status code, unless you've got an after filter to handle that scenario differently, such as Scorched does when in a development environment where it will include a bit of friendly HTML in the response.

Don't worry though, everyone falls into this trap, even I still have my moments.

Cheers,

Tom

Ted Milker

unread,

Aug 13, 2013, 8:38:31 PM8/13/13

to scor...@googlegroups.com

Did you test the code you posted? It doesn't work for me with jQuery's $.getJSON. The request still gets the 'text/html' route output, not 'application/json'.

test.ru:

require 'scorched'

require 'json'

class App < Scorched::Controller

get '/test', media_type: 'text/html' do

puts "text/html"

<<-HTML

<!DOCTYPE html>

<html>

...

</html>

HTML

end

get '/test', media_type: 'application/json' do

puts "application/json"

{name: 'blah', title: 'title', occupation: 'bleh'}.to_json

end

run App

test.html:

<!DOCTYPE html>
<html>
<head>
  <title>Media type test</title>
  <script src="http://code.jquery.com/jquery-1.10.2.min.js"></script>
  <script type="text/javascript">
    $(document).ready(function() {
      $("#clickem").click(function(event) {
        $.getJSON('http://gamma:8000/test', function(data) {
          alert(data.title);
        });
      });
    });
  </script>
</head>
<body>
  <button id="clickem">Click Me</button>
</body>
</html>

console log:

[2013-08-13 19:37:08] INFO  WEBrick 1.3.1
[2013-08-13 19:37:08] INFO  ruby 2.0.0 (2013-06-27) [x86_64-freebsd8.1]
[2013-08-13 19:37:08] INFO  WEBrick::HTTPServer#start: pid=92813 port=8000
text/html
198.183.6.154 - - [13/Aug/2013 19:37:13] "GET /data HTTP/1.1" 200 - 0.0198
198.183.6.154 - - [13/Aug/2013 19:37:13] "GET /favicon.ico HTTP/1.1" 404 - 0.0032
text/html
198.183.6.154 - - [13/Aug/2013 19:37:15] "GET /data HTTP/1.1" 200 - 0.0044

First request is the browser page load, second is the $.getJSON() call.

I was also messing with this originally with controller conditions, like the 'First Impressions' example and that didn't work either when I reversed the controller definitions. The only way I got this to work was to make a new condition that checks request.xhr? I must be missing something here but I just can't figure it out.

Ted Milker

unread,

Aug 13, 2013, 8:40:53 PM8/13/13

to scor...@googlegroups.com

There's a typo in the test.html I posted for the $.getJSON call that should be /data not /test, this was an old version of the file I pasted on accident.

Here's the right one that I was using:

<!DOCTYPE html>
<html>
<head>
  <title>Media type test</title>
  <script src="http://code.jquery.com/jquery-1.10.2.min.js"></script>
  <script type="text/javascript">
    $(document).ready(function() {
      $("#clickem").click(function(event) {

$.getJSON('http://gamma:8000/data', function(data) {

Tom Wardrop

unread,

Aug 13, 2013, 9:00:44 PM8/13/13

to scor...@googlegroups.com

I tested it using "Advanced REST Client" from chrome. I just generated a `jQuery.getJSON()` request to check the headers, and I can see that it includes "*/*", though it does list "application/json" and "text/javascript" before it. In an ideal world, it wouldn't include the "*/*".

Unfortunately at the moment, Scorched doesn't include a mechanism for serving the most "appropriate" content type. It merely traverses through the routes from top to bottom, checking conditions as it goes. As soon as a route and it's conditions match, it gets the request, which for any request that includes "*/*" as an accepted media type, means it'll always match the first route, at least in our example.

Now's probably a good time for me to begin looking into making Scorched's media type condition to behave more inline with the intentions of the HTTP spec. Leave it with me. I'll update this thread as I make progress.

Tom

On Wednesday, 14 August 2013 06:10:11 UTC+10, Ted Milker wrote:

Ted Milker

unread,

Aug 14, 2013, 7:43:05 AM8/14/13

to scor...@googlegroups.com

It seemed to me like something was wrong with the way rack-accept was comparing media types. I would think something like:

env['rack-accept.request'].media_type? '*/*;q=0, application/json'

would eliminate the */* from matching(take note of the q values for each of the following) for either the normal page request("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") or the json request("application/json, text/javascript, */*; q=0.01") but it is still true. However, the RFC seems to define it from the client's perspective, not the server, so assigning q values on the comparison may not be valid(but maybe it should?).

In any case, until you have a fix, anyone who sees this can use:

conditions << { xhr: proc { |bool| request.xhr? == bool } }

to distinguish json requests from normal page requests for now.

Ted Milker

unread,

Aug 14, 2013, 7:59:51 AM8/14/13

to scor...@googlegroups.com

I guess, thinking about it some more, it wouldn't matter if it did. That 'application/json' part has an assumed q=1 so it's always going to match any */* with any value of q(other than I assume 0). What a bind.

Tom Wardrop

unread,

Aug 15, 2013, 11:48:30 PM8/15/13

to scor...@googlegroups.com

(didn't realise a "reply to author" doesn't reply to the mailing list, that sucks)

Hmm, I thought I added another reply to this. Must have closed the tab without saving? Anyway, by leveraging rack-accept a little further I'm now re-ordering matched mappings based on their `media_type` condition, in addition to their priority and definition order which I was already doing. For mappings of the same priority, the better matching mapping as far as media type is concerned, will be invoked first.

`rack-accept` doesn't provide the best facilities for doing this properly though. For example, the HTTP spec states that where q-value's are the same, the more specific media type should win (i.e a match against `application/json` should win over a match against `application/*`), or where specificity is the same, the first media type listed should win. `rack-accept` doesn't help us in this respect, so it really only get's us half of the way there.

I've spent some time looking at patching/extending `rack-accept`, but it's in dire need of a refactor in my opinion, so it's a matter of either now worrying about it, or re-writing/refactoring `rack-accept`.

Tom

On Wednesday, 14 August 2013 06:10:11 UTC+10, Ted Milker wrote:

Tom Wardrop

unread,

Aug 27, 2013, 6:59:55 AM8/27/13

to scor...@googlegroups.com

Just as an update. I've been casually working on writing my own version of `rack-accept`, doing it the right way, using Treetop to parse the accept headers as per the grammar defined in the RFC. I've never used Treetop or any kind of PEG before, so there's been a nice little learning curve. Once I've got the headers parsed and into a convenient data structure, I'll then start building out a nice little API, and turn it into optional rack middleware. I make it sound so quick and easy.

Tom

Jack Chu

unread,

Aug 27, 2013, 6:00:16 PM8/27/13

to scor...@googlegroups.com

That's awesome news Tom. rack-accept seems like it's abandoned at this point. I submitted a PR to add accept-extension support but it's been ignored and it wasn't pretty to work with.

Tom Wardrop

unread,

Sep 2, 2013, 2:58:36 AM9/2/13

to scor...@googlegroups.com

The biggest challenge at this point is naming things. Naming the gem, naming the modules and classes, etc. As Phil Karlton is often quoted, "There are only two hard things in Computer Science: cache invalidation and naming things".

I'm going to add a third point to that, which is dealing in domains you're not an expert on, which is why I'm only going to add (at least initially) support for the "Accept" header, and drop "Accept-Language", "Accept-Encoding" and "Accept-Charset". "Accept" is a way more commonly used header with many real-world use cases, especially with today's web services. Encoding and charset stuff is typically more relevant to the web server than the web application, and language is probably too niche to be worried about at this point.

So I'll focus on nailing media types for now, and maybe I'll give the other accept headers some attention when someone comes along with some real-world scenario's, with which I can use to build up an appropriate solution. In the mean time, anyone will be able to require `rack-accept` and add their own charset, encoding or language conditions.

Tom

Reply all

Reply to author

Forward