Django urls in JavaScript

[Marco Louro]

I don't really know how to approach this issue the best way, so I'm
just going to be direct, I'd love to see Django provide support for
django.core.urlresolvers.reverse (at least) on the client (in
JavaScript).


I'm pretty sure all who have needed to make AJAX queries have dealt
with the issue of "where do we store our urls". The urls are, I'd
guess, 99% of the time, defined in Django, and I think that it usually
comes down to two approaches:
1) hard-code them: $.post("/hello/" + what + "/")
2) Store them in a variable or object: var hello_url = "{% url
hello_url what.id %}" - But if you need to "generate" the url based on
an id, this doesn't work.

There are more solutions of course, but these are the quick and easy
and I assume it's what most people use, and yes, it's just not elegant
and DRY.


What I'd like to propose to fix it:

1) Create a django namespace in JavaScript
2) Create a django.urls module, and implement reverse (and maybe
resolve)
3) Create a way to load urls from Django into JavaScript's django.urls
module (as django.urls.url_list or whatever) - of course, we don't
want all urls to be listed.


So my question is, is there any interest in getting something like
this into Django itself?

I think that with staticfiles bundling JavaScript for use outside of
the admin is possible and easy, but I also understand it's a sensible
matter even thou this wouldn't require any external libraries. I do
have an implementation of a urls module (https://github.com/mlouro/
olivejs/blob/master/olive/urls/urls.js) and tests (https://github.com/
mlouro/olivejs/blob/master/tests/urls.js), but it has a lot more stuff
attached that would be out of scope, as I think the urls themeselves
are a really important bit that Django should make available on the
client-side.


Thanks,
Marco

[Carl Meyer]

Hi Marco,

On 03/17/2011 10:44 AM, Marco Louro wrote:
> I don't really know how to approach this issue the best way, so I'm
> just going to be direct, I'd love to see Django provide support for
> django.core.urlresolvers.reverse (at least) on the client (in
> JavaScript).

I think this is really interesting, and I'd like to play with your
approach. It does seem to me like something that, while quite useful for
some projects, not all projects will need, and that it ought to stand on
its own as a reusable app for quite a while to prove its usefulness and
shake out any issues (and probably just stay that way).

Just to give you a point of comparison, database migrations are a
feature that is much more core to Django than this, and even those were
left out of core for years to allow several different approaches to sort
themselves out. Even now that South is the clear winner in that space,
only a portion of South is currently being considered for core inclusion.

I don't immediately see anything that would prevent implementing your
proposal as a reusable app outside core. If you do run into problems
where a small change in core could make your life a lot easier in
implementing this as a reusable app, please do bring it up here or file
a bug.

Carl

[Łukasz Rekucki]

On 17 March 2011 16:58, Carl Meyer <ca...@oddbird.net> wrote:
> Hi Marco,
>
> On 03/17/2011 10:44 AM, Marco Louro wrote:
>> I don't really know how to approach this issue the best way, so I'm
>> just going to be direct, I'd love to see Django provide support for
>> django.core.urlresolvers.reverse (at least) on the client (in
>> JavaScript).
>
> I think this is really interesting, and I'd like to play with your
> approach. It does seem to me like something that, while quite useful for
> some projects, not all projects will need, and that it ought to stand on
> its own as a reusable app for quite a while to prove its usefulness and
> shake out any issues (and probably just stay that way).

I would be more then interested in contributing to such app. I think
it would be a good addition to i18n JS library that Django provides.

Right now, I'm using my own view that generates a dictionary from all
*named* views and provides a simple reverse() with syntax similar to
yours. My biggest issues right now are a) efficient caching b) making
it work well with asset managers. Maybe I'm missing something, but
last time I checked staticfiles didn't had an option to include
dynamically generated content.

--
Łukasz Rekucki

[Marco Louro]

@Carl

Thanks, I'll probably start with that. Won't use the django namespace
in JS of course.



@Łukasz
don't think staticfiles supports that either. I imagine something like
this in a template:

<script type="text/javascript">
django.urls.load({% get_urls with format='json' group='all' %});
</script>

so you can optionally cache with django_compressor.





On Mar 17, 4:24 pm, Łukasz Rekucki <lreku...@gmail.com> wrote:

[sdcooke]

I realise this doesn't apply to everyone but we've been coming up
against
this recently and every time I've looked at creating a JavaScript
version of
the URLs functionality I felt like it was overkill for our needs. 90%
of our
situations (again, might not apply to all!) could be solved by
attaching a
"data-" attribute to an HTML element. For example, if you're popping
up an
AJAX dialog box you could include the url on a button like this:

<button data-url="{% url ... %}"></button>

That way, you don't need to do any resolving in JS. Just thought I'd
mention
it here in case people come across this thread and unnecessarily start
adding resolver apps that they don't need.

On Mar 17, 4:40 pm, Marco Louro <mlo...@gmail.com> wrote:
> @Carl
>
> Thanks, I'll probably start with that. Won't use the django namespace
> in JS of course.
>

> @£ukasz

> don't think staticfiles supports that either. I imagine something like
> this in a template:
>
> <script type="text/javascript">
> django.urls.load({% get_urls with format='json' group='all' %});
> </script>
>
> so you can optionally cache with django_compressor.
>

> On Mar 17, 4:24 pm, £ukasz Rekucki <lreku...@gmail.com> wrote:
>
>
>
>
>
>
>
> > On 17 March 2011 16:58, Carl Meyer <c...@oddbird.net> wrote:
>
> > > Hi Marco,
>
> > > On 03/17/2011 10:44 AM, Marco Louro wrote:
> > >> I don't really know how to approach this issue the best way, so I'm
> > >> just going to be direct, I'd love to see Django provide support for
> > >> django.core.urlresolvers.reverse (at least) on the client (in
> > >> JavaScript).
>
> > > I think this is really interesting, and I'd like to play with your
> > > approach. It does seem to me like something that, while quite useful for
> > > some projects, not all projects will need, and that it ought to stand on
> > > its own as a reusable app for quite a while to prove its usefulness and
> > > shake out any issues (and probably just stay that way).
>
> > I would be more then interested in contributing to such app. I think
> > it would be a good addition to i18n JS library that Django provides.
>
> > Right now, I'm using my own view that generates a dictionary from all
> > *named* views and provides a simple reverse() with syntax similar to
> > yours. My biggest issues right now are a) efficient caching b) making
> > it work well with asset managers. Maybe I'm missing something, but
> > last time I checked staticfiles didn't had an option to include
> > dynamically generated content.
>
> > --

> > £ukasz Rekucki

[Ayaz Ahmed Khan]

On Thu, Mar 17, 2011 at 7:44 PM, Marco Louro <mlo...@gmail.com> wrote:
> I don't really know how to approach this issue the best way, so I'm
> just going to be direct, I'd love to see Django provide support for
> django.core.urlresolvers.reverse (at least) on the client (in
> JavaScript).
>

> [...]

>
> So my question is, is there any interest in getting something like
> this into Django itself?

Having run into this issue one too many times, I would like to raise
my voice to say that I am *very much* interested in having this
included in Django.

Ayaz

[Matt Robenolt]

How could this even begin to be solved without incurring another http request to resolve the url pattern?

The only way I can imagine it is if we had a generic /resolve/ path that took some get parameters to return a full URL, or even it translate to a 301 redirect, but that'll get messy with POSTs.

[onelson]

Url patterns don't tend to change at runtime, right?

I don't think resolving urls one by one via ajax calls is going to work. Reimplementing the resolve/reverse mechanisms in javascript - or better yet, using the "native" resolve/reverse code to generate javascript functions that take args based on the url patterns.

some_ns.urls.my_view( 123, {slug:'foo'} )

You'd just have to analyze the url patterns once. Only think that would get tricky would be named views, eg having more than one pattern matching a name. I don't know if that would be considered bad practice generally, but it works on the python side -  kind of like overloading your urlpatterns.

[onelson]

Somewhat incomplete thought there -- once the urlpatterns have been analyzed, the resulting chunk of generated js can be cached (where ever) and served up statically, or inline in your templates.

[Matt Robenolt]

So you're basically proposing to write a Javascript library that is a translation of URLResolver, and essentially have a dynamic "Javascript" file that could be included that would contain your URL patterns? Just trying to make sure we're on the same page.

[onelson]

"dynamic" is a bit strong, but yes. A generated js script that, based on the regex in your urlpatterns, provides functions that plug vars into url templates... so yeah! A lot of the resolving logic wouldn't (read "shouldn't") be needed so long as unique names are used for the views.

[onelson]

I might also add the script could literally be something that is created via a management command and served via staticfiles.

[Marco Louro]

There would be no additional http request to get the urls, you just
dump the url list in JSON format in the template, with something like:

{% get_urls with format='json' group='all' %}

Then you could have a django.urls module in JavaScript to load the
JSON object with the list of urls:

<script type="text/javascript">
django.urls.load({% get_urls with format='json' group='all' %});
</script>

One approach I took in the past was have a management command that
generates that url list in JSON format (based on show_urls from
django_extensions), and it's probably not a bad approach since it
avoids running the template_tag on every request.

[Matt Robenolt]

It could just be a combo of both. There'd be one file for the urlresolver.js, and a patterns.js. Interesting.

Now, could the patterns in Python be translated to Javascript properly? I'll do some playing around today and see if I can come up with some basics. Even if this doesn't land in Django core, it could be a nice library.

[Matt Robenolt]

There also proposes the problem of selecting which urls are "published" in this file and which aren't. Any ideas for that? I'm sure lots of people wouldn't want their entire sitemap exposed to the public in one large js file.

[David Danier]

I currently handle this issue in a much easier way:
I've created an app which allows creating a context for Javascript (this
basically works like the template context processors do, calling defined
functions and combining its results into a dict). The resulting context
will be available in the templates as a normal variable (a context
processor puts it there). Inside my base template I define something
like: "var config = {% json js_context %};". This could be moved into
its own template tags, but I kind of like it this way.

This will store my own, extendable configuration into an project
specific variable (javascript object), which I can easily access
(config.foo.bar). It can be used to store URLs, but may also contain
generic things like MEDIA_URL.

So far it only solves one problem, getting variables into JS in an sane,
structured way...it doesn't solve handling URLs which accept parameters.
URL that don't need parameters are easy to, resolve() does all the work.
Parameters get tricky, but I have come up with an easy and efficient
solution, although it does look kind of strange. I use resolve() to get
a valid URL, but put some placeholder into it, which means:
resolve('my_named_view', kwargs={'slug': '__slug__'})

On the JS-side creating the final url just needs a simple string
replacement. Of course the URL needs to be valid, which means it may not
fit all situations (numeric ids: '(?P<pk>[0-9]+|__pk__)'?). Anyways this
works fine for me, while still being extenable enough. In addition it
solves the more generic problem of putting variables into JS, which is
kind of nice I think.

The resulting JS could be cached using normal template caching tags, or
may be moved into its own HTTP request if wanted. The idea of using some
kind of context processors for JS-context is what this email is
about. ;-)

David

[Matt Robenolt]

I think the biggest problem with translating the reverse() lookup is the lack of kwargs and named capture groups in Javascript regex. So a pattern such as: /page/(?P<page_id>\d+)/ would not translate whatsoever. Then on the Javascript side, we wouldn't be able to use:  reverse('goto_page', [], {page_id: 5});  It would have nowhere to map up the page_id variable to. We could probably get away with some sort of pseudo regex rules in Javascript.

[Marco Louro]

That's also already done, check
https://github.com/django-extensions/django-extensions/blob/master/django_extensions/management/commands/show_urls.py,
it can be easily converted to JSON (I have a branch that does it, but
it's not up-to-date).

I also have the urls module in JavaScript already, but as part of a
larger library (https://github.com/mlouro/olivejs/blob/master/olive/
urls/urls.js), both have been used in projects.

An example:

/* below is the output from "manage.py show_urls" in JSON format */
olive.urls.load({
'dashboard': '/dashboard/',
'time_edit': '/projects/<project_id>/time/save/<time_id>/',
'task_edit': '/projects/<project_id>/task/save/<task_id>/'
});

olive.urls.get('task_edit', {
'project_id': 2,
'task_id': 1
})

[Matt Robenolt]

Ahh, I missed that from your original post.

I like that. :)

[Dan Fairs]

> That's also already done, check
> https://github.com/django-extensions/django-extensions/blob/master/django_extensions/management/commands/show_urls.py,
> it can be easily converted to JSON (I have a branch that does it, but
> it's not up-to-date).
>
> I also have the urls module in JavaScript already, but as part of a
> larger library (https://github.com/mlouro/olivejs/blob/master/olive/
> urls/urls.js), both have been used in projects.
>
> An example:
>
> /* below is the output from "manage.py show_urls" in JSON format */
> olive.urls.load({
> 'dashboard': '/dashboard/',
> 'time_edit': '/projects/<project_id>/time/save/<time_id>/',
> 'task_edit': '/projects/<project_id>/task/save/<task_id>/'
> });
>
> olive.urls.get('task_edit', {
> 'project_id': 2,
> 'task_id': 1
> })
>

This all sounds pretty cool - I'd love to be able to reverse URLs in
JavaScript. Keep in mind, though, that URLConfs can change on a
per-request basis, by setting request.urlconf. Any JSON data structure
representing the URLConf would need to be generated after this has been
set (ie. some time after request middleware has been processed, look at
django.core.handlers.base:94-98). Having a template tag would probably
be fine, as the URLConf is set by render time, but don't expect to be
able to do a one-off analysis of the URLConf at application startup.

Cheers,
Dan

--
Dan Fairs | dan....@gmail.com | www.fezconsulting.com

[Dimitri Gnidash]

Hey Marco,

I have forked your library on GitHub and added the management command
to build a list of urls and output the file in the format that is
specified by django-js-utils.

https://github.com/Dimitri-Gnidash/django-js-utils

Cheers

On Mar 24, 11:37 am, Marco Louro <mlo...@gmail.com> wrote:
> That's also already done, checkhttps://github.com/django-extensions/django-extensions/blob/master/dj...,

[Jonathan Slenders]

On 18 mar, 13:43, sdcooke <sdco...@gmail.com> wrote:
> I realise this doesn't apply to everyone but we've been coming up
> against
> this recently and every time I've looked at creating a JavaScript
> version of
> the URLs functionality I felt like it was overkill for our needs. 90%
> of our
> situations (again, might not apply to all!) could be solved by
> attaching a
> "data-" attribute to an HTML element. For example, if you're popping
> up an
> AJAX dialog box you could include the url on a button like this:
>
> <button data-url="{% url ... %}"></button>
>
> That way, you don't need to do any resolving in JS. Just thought I'd
> mention
> it here in case people come across this thread and unnecessarily start
> adding resolver apps that they don't need.

Weird, I have missed this thread. But anyway, like sdcooke, that's
also the way that we handle URLs in javascript.
Attach a data attribute to the HTML node to which it applies, and read
it from inside the javascript. It's clean.

<td><a href="{% url accounts_edit_name %}"
x:ajax-url="{% url accounts_ajax_edit_name %}">{% trans "Edit" %}</
a></td>

Like gettext, a seperate, dynamically generated javascript file for
URL resolving is not scalable to lange web applications.
And further, I think that the urls and names of views are not meant to
be exposed to the client. I don't want a visitor to be able to reverse
engineer my website, and read all the possible URL patterns.

Jonathan

[Phui-Hock]

> Weird, I have missed this thread. But anyway, like sdcooke, that's
> also the way that we handle URLs in javascript.
> Attach a data attribute to the HTML node to which it applies, and read
> it from inside the javascript. It's clean.
>
>  <td><a href="{% url accounts_edit_name %}"
>     x:ajax-url="{% url accounts_ajax_edit_name %}">{% trans "Edit" %}</
> a></td>
>
> Like gettext, a seperate, dynamically generated javascript file for
> URL resolving is not scalable to lange web applications.
> And further, I think that the urls and names of views are not meant to
> be exposed to the client. I don't want a visitor to be able to reverse
> engineer my website, and read all the possible URL patterns.
>
> Jonathan

Just sharing thought here. Another approach that I use is creating an
"indirect" url and view that reads the url_name and args from
request.REQUEST, reverse the path, resolve the view and call it on
your behalf.

## urls.py
#
from django.conf.urls.defaults import patterns, url

urlpatterns = patterns('js.views',
url(r'^url_reverse/$', 'url_reverse', name='js_url_reverse'),
)


## views.py
#
from django import http
from django.core import urlresolvers

def url_reverse(request):
if request.method in ('GET', 'POST'):
data = getattr(request, request.method)
url_name = data.get('url_name')
try:
path = urlresolvers.reverse(url_name,
args=data.getlist('args'))
(view_func, args, kwargs) = urlresolvers.resolve(path)
return view_func(request, *args, **kwargs)
except urlresolvers.NoReverseMatch:
raise http.Http404
return http.HttpResponseNotAllowed(('GET', 'POST'))


Somewhere in javascript:
$.get('/js/url_reverse/', { url_name: "account_profile", args:
[username]});

Of cause, this approach exposes your url names to javascript. Some
people might not like it.

[Tom Evans]

On Wed, May 4, 2011 at 12:38 PM, Jonathan Slenders
<jonathan...@gmail.com> wrote:
>
> Like gettext, a seperate, dynamically generated javascript file for
> URL resolving is not scalable to lange web applications.
> And further, I think that the urls and names of views are not meant to
> be exposed to the client. I don't want a visitor to be able to reverse
> engineer my website, and read all the possible URL patterns.
>
> Jonathan
>

+1