making URL resolving/reversing gradually more flexible
Sam Lai
I have missed some protocol etc.]
First up, this is not about adding an alternate URL
resolution/reversal method to the core; I've noticed a fair bit of
resistance to that.
PROBLEM:
I want to make my website available via an API with URLs like this -
http://example.com/newitems/ => returns HTML
http://example.com/newitems.xml => returns XML
http://example.com/newitems.json => returns JSON.
To represent this in urls.py, I have to either:
a) write a separate url entry for each one
b) write 2 url entries, one for newitems/ and another for
newitems\.(?P<format>\w+). This is the better option, but still
annoying. Plus it forces my view to validate whether or not the format
value is acceptable (e.g. is either xml or json).
I have to do this for every URL I wish to make available via an API,
bloating out my urls.py file. (I'm aware I can use the HTTP-ACCEPT
header instead, but there is a need to be able to force the response
format in the URL for certain uses.)
MY DESIRED SOLUTION:
Subclass RegexURLPattern, and override the resolve method so that it
will replace a placeholder, e.g. (?#format), with the appropriate
regexps (/|\.xml|\.json). Effectively it will perform something
similar to this regexp instead -
^newitems(/|(?P<format>(\.xml|\.json))) where the list of accepted
formats will be defined in settings. This subclass will be returned
using an alternate url(...) call, e.g. murl(...).
So my urls.py will look like -
urlpatterns = patterns('project.app.views',
murl(r'^/(?P<model_id>\d+)(?#format)$', AppView.as_view(),
name='app_view1'),
)
(For completeness, the view is a CBV, and uses the format arg to
determine which template to render, and using what MIME type to
respond.)
This is a proven way of extending the URL system, as demonstrated by
the various projects out there for alternative URL specification
syntaxes, e.g. django-easyurls.
ROADBLOCK:
The issue with this solution is that while resolving will work fine,
reversing will not. The list of possible URLs for a particular view is
determined by _populate in RegexURLResolver, and is based on the
regexp pattern alone. Django doesn't support | in regexps
(understandably), and there is no way to supplant this with additional
regexps or possibilities at the moment, even though the infrastructure
is there during reversal.
RESOLUTION - PHASE 1:
Because of the friction and work required to fully revamp the URL
resolution system, this phase is a short, simple fix that will suffice
for most cases where people want to extend the URL resolution system.
Refactor out line 218 (in trunk) in django/core/urlresolvers.py:
bits = normalize(p_pattern)
... into a separate method in RegexURLPattern:
def get_possibilities(self):
return normalize(self.regex.pattern)
... and replace line 218 in django/core/urlresolvers.py with:
bits = pattern.get_possibilities()
That's it. I'll create a patch for this later if the consensus is
positive. The above change allows subclassed RegexURLPattern classes
to alter what is returned as possible URLs from that pattern. I'm
hoping this simple change can be made in Django 1.3.
Of course, the possibilities returned still have to be regexps, which
leads to phase 2...
RESOLUTION - PHASE 2:
The ultimate goal should be a URL resolution system that allows
alternate URL spec syntaxes to be first-class citizens, allowing
regexp based URL specs and say, URI Template specs to exist
side-by-side.
My plan would be to create an abstract base class for all URLPatterns,
which RegexURLPattern will extend. The existing behaviour will mostly
stay, except the get_possibilities from phase 1 will be deprecated in
favour of a new reverse method. The reverse method will be called by
the new universal URLResolver class for matches when reversing URLs,
and if a match exists, that will be returned. During _populate(), the
new universal URLResolver class will group URLPattern objects by view
callback and name, instead of the output of
get_possibilities/normalize.
This approach requires no changes to existing urls.py; in fact, from a
dev perspective, they would only notice the difference if they choose
to use alternate URL spec syntaxes. The existing regexp system will
work as it has always worked. And it makes sense that the URLPattern
object is responsible for resolving and reversing itself, and not the
resolver.
Until this phase is reached, the API should be considered private so
devs are on notice that things will change and may break existing
custom URL resolution code.
Again, I'm happy to have a crack at making this work if the consensus
is positive.
bur...@gmail.com
what's the problem with regexp = '^newitems'+SUFFIX+'$' where
SUFFIX='(/|\.xml|\.json)' ?
And if you need more shortcuts, there are surlex (
http://codysoyland.com/2009/sep/6/introduction-surlex/ ) and
alternatives.
> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
>
>
--
Best regards, Yuri V. Baburov, ICQ# 99934676, Skype: yuri.baburov,
MSN: bu...@live.com
bur...@gmail.com
I used to do the following:
alternatives = {'html': '/', 'xml': '.xml', 'json': '.json'}
for name, alt in alternatives.iteritems():
urlpatterns += ('^newitems'+alt+'$', 'views.newitems', {'format':
name}, 'newitems-'+name)
You can make your own method that will do this.
Anyway, only your code can know how to select one of the suggested
alternatives for reverse, so your suggested approach has no
advantages.
>This approach requires no changes to existing urls.py
But you have to update all django 3rd party libraries to realize every
url pattern they use can have get_possibilities !
And you now can't pass secondary pattern into django libs that are not
aware of your feature.
So, many-to-many relation between urlpatterns entry and view name only
complicates things.
Sam Lai
> Ah, sorry, tl;dr happened to me in previous message.
>
> I used to do the following:
>
> alternatives = {'html': '/', 'xml': '.xml', 'json': '.json'}
> for name, alt in alternatives.iteritems():
> urlpatterns += ('^newitems'+alt+'$', 'views.newitems', {'format':
> name}, 'newitems-'+name)
Then you're cluttering urls.py through code. You need to repeat that
for every API-enabled URL.
> You can make your own method that will do this.
> Anyway, only your code can know how to select one of the suggested
> alternatives for reverse, so your suggested approach has no
> advantages.
... and that's the point. It is the pattern's job to suggest
alternatives, not the resolver's.
>>This approach requires no changes to existing urls.py
> But you have to update all django 3rd party libraries to realize every
> url pattern they use can have get_possibilities !
How many third-party libraries out there use custom URL pattern
classes, ones that don't subclass from RegexURLPattern?
I doubt there are many. But to cater for this fact, the resolver can
simply check for the existence of get_possibilities, and if it isn't
there, revert to the existing behaviour. For example,
if hasattr(pattern, 'get_possibilities'):
bits = pattern.get_possibilities()
else:
bits = normalize(p_pattern)
> And you now can't pass secondary pattern into django libs that are not
> aware of your feature.
Not sure what you're talking about here. I can't see how my changes
will break anything, unless a library is using a URLPattern that
doesn't extend RegexURLPattern. And again, as above, a small fix can
solve that.
Just so I'm clear, this is a *backwards-compatible* change. Nothing
should break if the above change is incorporated into the original
proposal.
> So, many-to-many relation between urlpatterns entry and view name only
> complicates things.
I am not suggesting a many-to-many relationship between urlpatterns
and view names. Each urlpattern can still only match 1 view name. It
is a many-to-one relationship. Not that this case already exists if
you use ? in a regexp in a URL pattern.
bur...@gmail.com
On Fri, Nov 12, 2010 at 2:45 PM, Sam Lai <samue...@gmail.com> wrote:
> On 12 November 2010 19:14, bur...@gmail.com <bur...@gmail.com> wrote:
>> Ah, sorry, tl;dr happened to me in previous message.
>>
>> I used to do the following:
>>
>> alternatives = {'html': '/', 'xml': '.xml', 'json': '.json'}
>> for name, alt in alternatives.iteritems():
>> urlpatterns += ('^newitems'+alt+'$', 'views.newitems', {'format':
>> name}, 'newitems-'+name)
>
> Then you're cluttering urls.py through code. You need to repeat that
> for every API-enabled URL.
You can hide this with your_patterns() method, or with your_reverse.
It won't be more complex than your URL fix.
>> You can make your own method that will do this.
>> Anyway, only your code can know how to select one of the suggested
>> alternatives for reverse, so your suggested approach has no
>> advantages.
>
> ... and that's the point. It is the pattern's job to suggest
> alternatives, not the resolver's.
Well, such change is meaningful, because of
bits = normalize(p_pattern)
lookups.appendlist(pattern.callback, (bits, p_pattern))
lookups.appendlist(pattern.name, (bits, p_pattern))
appendlist method call convinced me immediately.
>>>This approach requires no changes to existing urls.py
>> But you have to update all django 3rd party libraries to realize every
>> url pattern they use can have get_possibilities !
>
> How many third-party libraries out there use custom URL pattern
> classes, ones that don't subclass from RegexURLPattern?
>
> I doubt there are many. But to cater for this fact, the resolver can
> simply check for the existence of get_possibilities, and if it isn't
> there, revert to the existing behaviour. For example,
>
> if hasattr(pattern, 'get_possibilities'):
> bits = pattern.get_possibilities()
> else:
> bits = normalize(p_pattern)
>
>> And you now can't pass secondary pattern into django libs that are not
>> aware of your feature.
>
> Not sure what you're talking about here. I can't see how my changes
> will break anything, unless a library is using a URLPattern that
> doesn't extend RegexURLPattern. And again, as above, a small fix can
> solve that.
I was talking about reverse with more than one result to the same name.
But they do exist already!
> Just so I'm clear, this is a *backwards-compatible* change. Nothing
> should break if the above change is incorporated into the original
> proposal.
>
>> So, many-to-many relation between urlpatterns entry and view name only
>> complicates things.
>
> I am not suggesting a many-to-many relationship between urlpatterns
> and view names. Each urlpattern can still only match 1 view name. It
> is a many-to-one relationship. Not that this case already exists if
> you use ? in a regexp in a URL pattern.
First I thought you're going to return some instance with reverse
which has get_possibilities.
And you just want RegexURLPattern to register 3 usual regexes for you
instead of one.
Could you please create a patch now -- at least someone will use it.
Maybe you also thought of a patch for this specific case of simple
reversible ORs in the urls?
Examples: (a|b|c) and (?P<a>a|b|c)
Sam Lai
> First I thought you're going to return some instance with reverse
> which has get_possibilities.
> And you just want RegexURLPattern to register 3 usual regexes for you
> instead of one.
Effectively. I should've been more concise when writing the initial email :)
> Could you please create a patch now -- at least someone will use it.
Will do. It'll help iron out any possible quirks in my proposal too.
> Maybe you also thought of a patch for this specific case of simple
> reversible ORs in the urls?
> Examples: (a|b|c) and (?P<a>a|b|c)
I have, and I'm happy to have a look at fixing that, but it'll
definitely not make release 1.3. I'm not sure how open the core devs
are to non-core devs poking around in core parts of the framework
though.
Gabriel Hurley
> are to non-core devs poking around in core parts of the framework
> though.
framework! There's no sacred portion that only a core dev can touch.
It sounds like you've given thought to the impacts on backwards
compatibility, which is always the biggest concern when dealing with
something as central as URL resolution.
You've done the right thing by opening the thread here, and I'm sure
some of the core folks who are more familiar with this area of the
codebase will weigh in. Having a patch to look at is even better.
Thanks for the contribution and for following through on the right
process!
- Gabriel