Pyramid content negotiation with accept parameters in @view_config

[Graham Klyne]

I'm new to Pyramid, Pylons, etc., but have previously used other
Python web frameworks.

I'm trying to use view_config accept parameters for HTTP content-type
negotiation, e.g.

@view_config(route_name='service', request_method='GET',
accept='application/rdf+xml')
def service_rdf_xml(request):
:
@view_config(route_name='service', request_method='GET', accept='text/
turtle')
def service_turtle(request):
:
@view_config(route_name='service', request_method='GET', accept="text/
html")
def service_html(request):
:
if __name__ == '__main__':
# configuration settings
settings = {}
settings['reload_all'] = True
settings['debug_all'] = True
# configuration setup
config = Configurator(settings=settings)
config.add_route(name='service', pattern='/')
config.scan()
# serve app
app = config.make_wsgi_app()
server = make_server('0.0.0.0', 8080, app)
server.serve_forever()

...

I am assuming that this is what the accept parameter is provided for,
as I can't see any other useful purpose, but maybe that's a
misunderstanding on my part.

...

Using the above structure, I am running into a couple of issues:

(a) content-type matching is case sensitive. This if I send a request
with, say:

Accept: application/RDF+XML

it is not recognized by the URI dispatcher.

(b) I can't see any way to create a catch-all match, to be used when
none of the view_config options *with* accept parameters are matched.
I'm using "accept="text/html" for now, but that's not very
satisfactory for the reason above, and also...

(c) The matching doesn't honour the q= values generated by browsers;
e.g. I have Firefox generating:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8,application/rdf+xml;q=0.93,text/rdf+n3;q=0.5

but I'm actually seeing the HTML version returned.

Is there something important I'm overlooking here?

Thanks.

#g

[Graham Higgins]

On Fri, 2012-05-11 at 02:18 -0700, Graham Klyne wrote:

> I'm trying to use view_config accept parameters for HTTP content-type
> negotiation, e.g.

I can provide a partial answer ...

> I am assuming that this is what the accept parameter is provided for,
> as I can't see any other useful purpose, but maybe that's a
> misunderstanding on my part.

Your understanding is correct.

> Using the above structure, I am running into a couple of issues:

Yes, I can confirm that, the match is case-sensitive.

> (b) I can't see any way to create a catch-all match, to be used when
> none of the view_config options *with* accept parameters are matched.

The Pyramid URL dispatch mechanism is basically "An ordered set of
patterns is checked one-by-one" so, if you simply remove the
specialising match parameter, you're left with a catch-all which, to be
effective, needs to appear last in the batting order:

@view_config(route_name='service', request_method='GET')
def service_default(request):
return Response("Default\n")

> (c) The matching doesn't honour the q= values generated by browsers;
> e.g. I have Firefox generating:
>
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
> *;q=0.8,application/rdf+xml;q=0.93,text/rdf+n3;q=0.5
>
> but I'm actually seeing the HTML version returned.
>
> Is there something important I'm overlooking here?

You just need to set the content-type of the response:

@view_config(route_name='service', request_method='GET',

accept='text/turtle')
def service_turtle(request):
return Response("TTL\n", content_type="text/turtle")


--
Graham Higgins

http://bel-epa.com/gjh/

[Graham Klyne]

On May 11, 5:37 pm, Graham Higgins <gjhigg...@gmail.com> wrote:
> On Fri, 2012-05-11 at 02:18 -0700, Graham Klyne wrote:
> > I'm trying to use view_config accept parameters for HTTP content-type
> > negotiation, e.g.
>
> I can provide a partial answer ...

Graham,

Thanks for your clarifications.

They raise a couple of further questions...

> > (b) I can't see any way to create a catch-all match, to be used when
> > none of the view_config options *with* accept parameters are matched.
>
> The Pyramid URL dispatch mechanism is basically "An ordered set of
> patterns is checked one-by-one" so, if you simply remove the
> specialising match parameter, you're left with a catch-all which, to be
> effective, needs to appear last in the batting order:
>
> @view_config(route_name='service', request_method='GET')
> def service_default(request):
>     return Response("Default\n")

If using decorators to configure the views, how is "batting order"
defined. It seems to me that it;'s somewhat at the whim of the Python
compiler?

> > (c) The matching doesn't honour the q= values generated by browsers;
> > e.g. I have Firefox generating:
>
> >   Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
> > *;q=0.8,application/rdf+xml;q=0.93,text/rdf+n3;q=0.5
>
> > but I'm actually seeing the HTML version returned.
>
> > Is there something important I'm overlooking here?
>
> You just need to set the content-type of the response:
>
> @view_config(route_name='service', request_method='GET',
>              accept='text/turtle')
> def service_turtle(request):
>     return Response("TTL\n", content_type="text/turtle")

I'm doing that, but I don't see how it affects the selection of which
view to invoke.

(Maybe I wasn't clear: when the different views are invoked, they
each return the appropriate content-type. It's just that mis-spelled
or complex accept headers in the incoming request don't trigger the
desired view. I'm wondering if this should be raised as a bug against
Pyramid, because as it stands it seems one can't use this mechanism to
achieve effective content negotiation.)

Again, thanks for clarifying - I was thinking that maybe I was trying
to approach this goal from a completely wrong direction.

#g

[Graham Higgins]

On Fri, 2012-05-11 at 14:03 -0700, Graham Klyne wrote:

> If using decorators to configure the views, how is "batting order"
> defined. It seems to me that it;'s somewhat at the whim of the Python
> compiler?

Sorry for the nonsense - I should have spent more time investigating.

> > > (c) The matching doesn't honour the q= values generated by browsers;

It transpired that WebOb was improperly rounding float q values to a
single decimal via a "%0.1f". That is now fixed in the latest WebOb
repos:

https://github.com/Pylons/webob

[Graham Klyne]

Sorry I went quiet on this - I got a bit tied up with other stuff.
I've installed the latest version of webob, from github, and it
doesn't fix either problem. I've also been playing with a local copy.

I've worked out a fix for the case sensitive matching of MIME types -
simply adding .lower() at the appropriate places in MIMEAccept._match
(similar to what Accept._match does)

The whole selection of the best match by q-value seems to be subverted
by the calling environment. As far as I can tell the logic within
Accept.best_match is working just fine, but for a single HTTP request,
I'm seeing the Accept object constructed 4 times, and best_match
called three times, with different "offer" lists. As far as I can
tell, the final instantiation of Accept doesn't call best_match at
all, and that's what's returned to the caller:

**** Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8,application/rdf+xml;q=0.93,text/rdf+n3;q=0.5
**** best_match ['application/rdf+xml', 'text/turtle', 'text/html']
:
**** Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8,application/rdf+xml;q=0.93,text/rdf+n3;q=0.5
**** best_match ['application/rdf+xml', 'text/turtle']
:
**** Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8,application/rdf+xml;q=0.93,text/rdf+n3;q=0.5
**** best_match ['text/turtle']

**** Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8,application/rdf+xml;q=0.93,text/rdf+n3;q=0.5

So it seems that the Pyramid request routing logic is making multiple
calls with a reducing list of possible server offers for handling the
MIME type.

#g

>  signature.asc
> < 1KViewDownload

[Graham Higgins]

On Fri, 2012-05-18 at 09:34 -0700, Graham Klyne wrote:

> I've worked out a fix for the case sensitive matching of MIME types -
> simply adding .lower() at the appropriate places in MIMEAccept._match
> (similar to what Accept._match does)

Noted, I'll pass that on.

> So it seems that the Pyramid request routing logic is making multiple
> calls with a reducing list of possible server offers for handling the
> MIME type.

That's rather puzzling and I'm at a loss to explain it.

I transcribed Gunnar's Flask-based LODserver / SPARQLendpoint [1] to
Pyramid [2] (for use with Python 3) and it is passing all but one of the
conneg tests posed by Virtuoso's "Vapour" LOD validator facility [3]
and, as far as I can tell from debug statements, I'm only seeing one
call on best_match() for each step in the conneg sequence.

When I get a moment, I will check my Pyramid transcription under Python
2, to see if anything shows up there.

[1] https://github.com/RDFLib/rdfextras-web
[2] http://delta.bel-epa.com/
[3] http://validator.linkeddata.org/vapour?uri=http%3A%2F%
2Fdelta.bel-epa.com%2Fresource%2FBook%
2Fbook1&mixedAccept=1&defaultResponse=dontmind&userAgent=vapour.sourceforge.net

Cheers,

[Graham Klyne]

Hmm... maybe it's my code... I'll try and isolate the part of my
server code that causes this and post it here.

#g

>  signature.asc
> < 1KViewDownload

[Graham Klyne]

Hmmm.... turns out it works. Nothing like isolating some code to
figure the problem :)

I had missed that Firefox was slipping in a teeny little "text/html"
in the Accept line without a q-value, so of course the "application/rdf
+xml;q=0.93" was being ignored. Ho hum.

I'm still seeing best_match called repeatedly, but at least it's doing
the right thing.

So it's just the case sensitivity of content-types, which is easily
fixed.

Thanks for your patience and help.

#g


On May 19, 4:56 am, Graham Higgins <gjhigg...@gmail.com> wrote:

>  signature.asc
> < 1KViewDownload