Challenge teaching Django to beginners: urls.py

650 views
Skip to first unread message

Emil Stenström

unread,
Sep 12, 2016, 4:32:45 PM9/12/16
to Django developers (Contributions to Django itself)
Hi Djangonauts,

I'm just back from my second year of teaching Django to absolute beginners. The course is a combination of HTML, CSS, Python, and Django, and after five days of coding they have a real live website that they can show to friends. It's always such a great experience to see the look in their eyes when they finally understand how they can tame Django to do what they want.

There's one big thing that keeps tripping them up is urls.py. When specifying URL:s I get lots of questions about the regexes that they have to specify. First: there's a strange "r" in front of each line: r"regex". That means I will have to explain string escaping to them. Then there's the "^" and "$" signs, both which requires explaining regular expressions at length. Then there's [0-9]+ and finally there's the parenthesis around the regex. All in all, looking at URLs from a beginners perspective, they are a bunch of hieroglyphs, and very hard for beginners to grasp right away.

I'm not suggesting that urls.py are changed for most users, I'm suggesting that *simple_url* method (name inspired by simple_tag) is added to django.conf.urls that new users can use to get started quickly. This means that most beginners can postpone learning regexes a couple of months. The exact syntax that simple_url takes isn't important to me, as long it's a lot more beginner friendly than what we have today:

https://docs.djangoproject.com/en/1.10/topics/http/urls/#example

Just to get the ideas flowing, here's a suggestion, inspired by rails (again, exact syntax isn't important to me, simplicity to beginners is, so feel free to suggest something else if you agree that this is an important issue):

from django.conf.urls import simple_url
from . import views
urlpatterns
= [
    simple_url
('articles/2003/', views.special_case_2003),
    simple_url
('articles/:year)/', views.year_archive),
    simple_url
('articles/:year/:month/', views.month_archive),
    simple_url
('articles/:year/:month/:day/', views.article_detail),
]

All parameters would be passed to the view as keyword parameters with the name given and as a string, and validation would happen there instead.

I'm thinking there should be no settings
with simple_url, and that any more advanced use-case should switch to using url instead.

Two questions:

A) What do you think about the prospect of simplifying urls.py for beginners?
B) What do you think about the specific suggestion to mimic Rails urls with a simple_url tag?

Thanks for reading!

Constantine Covtushenko

unread,
Sep 13, 2016, 1:20:55 AM9/13/16
to django-d...@googlegroups.com
Hi Emil,

It is a very interesting idea.

+1 from me

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3d002c25-9d98-49b1-b84c-55bc39c6a0f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ares Ou

unread,
Sep 13, 2016, 2:26:44 AM9/13/16
to django-d...@googlegroups.com
Hi,

Actually flask uses a style very similar to what you want. 

To my knowing, you must use this pattern for Django because it has the concept of including 
URLs.

Routing in flask:

@app.route('post/<integer:post_id>', methods=['GET'])
def post_view(post_id=None):
    # do something to render the post
    return render_template('post.html', post=post)

But the problem is, you have to specify the whole URL for every view you define.

Django avoids this by separating URL patterns into different levels, that's why it uses regex to 
identify the exact path. I guess it is hard for Django to organize URLs in different apps by using
such simple method.

Looking forward to more ideas!

Best regards,
Ares Ou

Software Engineer / Full-Stack Python Developer  |  Phone: (510) 328 - 5968


Ares Ou

Message has been deleted

Florian Apolloner

unread,
Sep 13, 2016, 3:18:51 AM9/13/16
to Django developers (Contributions to Django itself)
Hi Emil,

There are projects like https://github.com/codysoyland/surlex which I like very much. In the end, whatever you can come up which translates to regex in the background would work just fine for django. I personally like what werkzeug does -- "stealing" that sounds like an idea too :D I do not think any core dev would be opposed to get rid of regexes by default (as long as you can still fall back to them). I personally would really like a system that works without regex on the "outside", and I'd be happy about a DEP after some further discussions here.

Cheers,
Florian

Michal Petrucha

unread,
Sep 13, 2016, 3:33:14 AM9/13/16
to django-d...@googlegroups.com
Tjena Emil,
I can certainly see where this proposal is coming from. I understand
that diving into regular expressions can derail the learning process,
and regular expressions are quite a heavy topic, especially when you
just want to move forward through the URL routing part.

On the other hand, regular expressions are a bit more flexible than
what for example Flask does with is own syntax, and I'm not a big fan
of having two ways of doing things. Then again, one very rarely needs
the full power of regular expressions, so having something simpler by
default wouldn't hurt the readability of most URL patterns.

I guess I'm about +0 on this question...

> B) What do you think about the specific suggestion to mimic Rails urls with a simple_url tag?

FWIW, if we choose to implement your proposal, I'm not very convinced
by the Rails syntax (just based on your example). It feels like it
hides too much – how does it decide on which character it will end the
":year" argument? There's no information that would make it apparent
what characters will land inside the argument, and which will not.

I'd be more in favor of the syntax used by Flask (or is it
Werkzeug...?), where you add type information as well: ``<int:year>``.
This, in my opinion, gives a better idea of what kind of characters is
expected there. On the other hand, it could also lead to
misunderstandings if we just turn those patterns into regular
expressions, and otherwise use the same resolver machinery that we
already have – the view would then still get the string extracted from
the URL, not an integer as it says in the pattern. Not sure if that
can be a problem or not.

Cheers,

Michal
signature.asc

Sjoerd Job Postmus

unread,
Sep 13, 2016, 7:13:11 AM9/13/16
to django-d...@googlegroups.com
Hi,

I don't think that the 'including' URLs part forms a problem here.

For the given example, it should be easily doable

> from django.conf.urls import simple_url
> from . import views
> urlpatterns = [
> simple_url('articles/2003/', views.special_case_2003),
> simple_url('articles/:year)/', views.year_archive),
> simple_url('articles/:year/:month/', views.month_archive),
> simple_url('articles/:year/:month/:day/', views.article_detail),
> ]

The following should make it possible:

import re

def simple_url(route, view, *args, **kwargs):
regex_route = re.escape(route)
regex_route = re.sub(r':([A-Za-z0-9_-]+)', r'(?P<\1>[^/]+)', regex_route)
# Anchor it to the beginning.
regex_route = '^' + regex_route

if not isinstance(view, (list, tuple)):
# It's not an include...
regex_route += '$'

return url(regex_route, view, *args, **kwargs)

(Not 100% tested, but I tested the re.sub ;) ).

So I think it is at least possible. Whether or not it's desirable though
is another question which I think I'm not suited to answer.

Kind regards,
Sjoerd Job

On Mon, Sep 12, 2016 at 11:26:36PM -0700, Ares Ou wrote:
> Hi,
>
> Actually flask uses a style very similar to what you want.
>
> To my knowing, you must use this pattern for Django because it has the
> concept of *including *
> URLs.
>
> Routing in flask:
>
> @app.route('post/<integer:post_id>', methods=['GET'])
> def post_view(post_id=None):
> # do something to render the post
> return render_template('post.html', post=post)
>
> But the problem is, you have to specify the whole URL for every view you
> define.
>
> Django avoids this by separating URL patterns into different levels, that's
> why it uses regex to
> identify the exact path. I guess it is hard for Django to organize URLs in
> different apps by using
> such simple method.
>
> Looking forward to more ideas!
>
> Best regards,
> Ares Ou
>
> *Software Engineer / Full-Stack Python Developer | **Phone:* (510) 328 -
> 5968
>
> *Blog:* http://aresou.net | *Github:* https://github.com/aresowj | *Stack
> Overflow:* http://stackoverflow.com/users/5183727/ares-ou
> >> email to django-develop...@googlegroups.com.
> >> To post to this group, send email to django-d...@googlegroups.com.
> >> Visit this group at https://groups.google.com/group/django-developers.
> >> To view this discussion on the web visit https://groups.google.com/d/ms
> >> gid/django-developers/3d002c25-9d98-49b1-b84c-55bc39c6a0f9%
> >> 40googlegroups.com
> >> <https://groups.google.com/d/msgid/django-developers/3d002c25-9d98-49b1-b84c-55bc39c6a0f9%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> .
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-develop...@googlegroups.com.
> > To post to this group, send email to django-d...@googlegroups.com.
> > Visit this group at https://groups.google.com/group/django-developers.
> > To view this discussion on the web visit https://groups.google.com/d/
> > msgid/django-developers/CAK52boUa_D-%2BVaf6VPgrA0YDAK4CWFbjFSWxV3Ri
> > Vf-JEDXm1Q%40mail.gmail.com
> > <https://groups.google.com/d/msgid/django-developers/CAK52boUa_D-%2BVaf6VPgrA0YDAK4CWFbjFSWxV3RiVf-JEDXm1Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
> > .
> >
> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --
> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFCGC%3DaX-0fvW9K%3D6dZa_wMAQVDLUAKc6mKoSzaOfCObhbmViw%40mail.gmail.com.
Message has been deleted

Anthony King

unread,
Sep 13, 2016, 7:13:11 AM9/13/16
to Django developers (Contributions to Django itself)

This is actually an interesting concept, and wouldn't incur an overheard at runtime if simple_url translated in to full regex format at launch time (or on first request, which is when the urls get loaded if I recall correctly).

I don't think this would get in the way of includes, and if it's a translator to full regex format, it can be done in a separate format. 


The example from flask is interesting. What if we could define regex for our urls to have it pluggable.


simple_url.register('integer', r'[0-9]+')

Ares Ou

unread,
Sep 13, 2016, 1:29:52 PM9/13/16
to django-d...@googlegroups.com
Hi,

Good to hear so much from you.

Just want know more, could anyone also explain why at first Django 
chose to use regex instead of a simple URL routing, except for the flexibility?

By the way, like you said, routing in flask (or should say Werkzeug?) is also 
converting the simple URLs into Regex and store it into the map.

Cheers,

Best regards,
Ares Ou

Software Engineer / Full-Stack Python Developer


On Mon, Sep 12, 2016 at 11:37 PM, Anthony King <anthon...@gmail.com> wrote:

This is actually an interesting concept, and wouldn't incur an, overheard at runtime if simple_url translated in to full regex format at launch time (or on first request, which is when the urls get loaded if I recall correctly).

I don't think this would get in the way of includes, and if it's a translator to full regex format, it can be done in a separate format.

Your example of flask's is interesting. What if we could define regex for our urls to have it pluggable.

simple_url.register('integer', r'[0-9]+')

On 13 Sep 2016 07:26, "Ares Ou" <are...@gmail.com> wrote:
Hi,

Actually flask uses a style very similar to what you want. 

To my knowing, you must use this pattern for Django because it has the concept of including 
URLs.

Routing in flask:

@app.route('post/<integer:post_id>', methods=['GET'])
def post_view(post_id=None):
    # do something to render the post
    return render_template('post.html', post=post)

But the problem is, you have to specify the whole URL for every view you define.

Django avoids this by separating URL patterns into different levels, that's why it uses regex to 
identify the exact path. I guess it is hard for Django to organize URLs in different apps by using
such simple method.

Looking forward to more ideas!

Best regards,
Ares Ou

Software Engineer / Full-Stack Python Developer  |  Phone: (510) 328 - 5968


Ares Ou


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

Emil Stenström

unread,
Sep 13, 2016, 3:33:25 PM9/13/16
to Django developers (Contributions to Django itself)
So it looks to me that the consensus is that this IS in fact a good idea, to supply a simpler, regex-free method to define URL:s.

It also seems that the best liked version is something that's similar to what flask uses: /articles/<int:year>/<int:month>/.

I've never written a DEP before, but it sounds like a fun challenge. I'll try to look at existing DEPs for a pattern and then apply that.

Does anyone have something in particular that they would like to add to the DEP? I figure I'll try to keep this first version as simple as possible, while maintaining extension points for features that can be added later on.

Tim Graham

unread,
Sep 13, 2016, 3:40:47 PM9/13/16
to Django developers (Contributions to Django itself)
I would like to see if this could be done as a third-party project (allow "pluggable URLs" which could use any syntax). If not, then let's accept a patch to Django to support it. Over time, if there's some strong consensus about a particular third-party package, then we could bring it in to core. I think this approach is less controversial then Django adopting some new, untested syntax right now.

Emil Stenström

unread,
Sep 13, 2016, 3:50:22 PM9/13/16
to Django developers (Contributions to Django itself)
One problem with this approach, to let the community decide on this type of feature, is the target audience of it. This is mainly a feature for beginners, and more experienced users are expected to switch over to using regexes directly to get the exact behavior they want. Beginners likely won't look at all the different options and choose one based on it's merits, they'll pick whatever their teacher suggests they use. Also installing an extra package when setting up django feels a bit strange.

So in summary, I think this COULD be done outside of Django, but that most of the benefit of this feature will come from it being part of Django. All because of the target audience.

/Emil

ludovic coues

unread,
Sep 13, 2016, 3:54:49 PM9/13/16
to django-d...@googlegroups.com
There is third party module providing third party url function. Surlex
[1] have been mentionned. But any third party solution will need to
provide function compatible with django.conf.urls.url.
Line 64 of django/urls/revolvers.py is get_resolver. This function
return a RegexURLResolver, using is argument or the setting
ROOT_URLCONF as argument.

This make impossible, for exemple, to have resolver giving to the view
an int argument.

[1] http://codysoyland.com/2009/sep/6/introduction-surlex/
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/37e44d86-696d-4b36-803a-0089232eedf9%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--

Cordialement, Coues Ludovic
+336 148 743 42

Ares Ou

unread,
Sep 13, 2016, 4:17:34 PM9/13/16
to django-d...@googlegroups.com
Emil,

Please count me in if you'd like to start a new DEP. I'm also 
very interested to take this new challenge. And maybe we 
should dive into the code of URL resolver part of Django before 
we actually kick off?

Anyway, I myself as a experienced user also like this idea. But as 
Tim and Ludovic mentioned, we might better research a bit first. 
Understanding how third-party libraries implemented this kind of 
simple URL could be very helpful.

Thanks.

Best regards,
Ares Ou

Software Engineer / Full-Stack Python Developer


Ares Ou

> email to django-developers+unsubscribe@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
--

Cordialement, Coues Ludovic
+336 148 743 42
--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Florian Apolloner

unread,
Sep 14, 2016, 4:38:20 AM9/14/16
to Django developers (Contributions to Django itself)
Hi Emil,


On Tuesday, September 13, 2016 at 9:50:22 PM UTC+2, Emil Stenström wrote:
and more experienced users are expected to switch over to using regexes directly to get the exact behavior they want.

How so? Personally I would use this quite quickly since a few builtin types would cover 99%. While I can write regex in sleep nowadays, I still find it kinda tedious to redefine what "slug" means in every URL I wanna match something… I am sure others think the same.
 
Beginners likely won't look at all the different options and choose one based on it's merits, they'll pick whatever their teacher suggests they use. Also installing an extra package when setting up django feels a bit strange.

I think the eco system is far enough to support that, after all south lived long and well as external package. Either way, DEP or not, having an implementation out there would definitely help.

Cheers,
Florian

sjoerdjob

unread,
Sep 14, 2016, 5:02:23 AM9/14/16
to Django developers (Contributions to Django itself)
Hi all,

Since it seemed like an interesting idea to me, I started development of a third-party plugin.

It's currently at:
    https://github.com/sjoerdjob/django-simple-url

Since I only started today, I have no readme/setup.py yet. Will come later this week I hope.

Current usage is

    from django_simple_url import simple_url

    urlpatterns = [
        simple_url('hello/world/', hello_world_view),
        simple_url(':year/:month/', posts_for_month_view),
    ]

It works proper with includes (not adding a $ to the URL), and leaf views (adding a $ to the URL).

Maybe this week, or early next week I will also add support for the '<int:year>' syntax.

Kind regards,
Sjoerd Job

Emil Stenström

unread,
Sep 15, 2016, 2:20:03 AM9/15/16
to Django developers (Contributions to Django itself)
Great initiative!

I really think you should use the flask syntax instead of the rails one that I first suggested. Seems this is the consensus from this thread, and that makes it more likely to get it to core one day.

/Emil

Emil Stenström

unread,
Sep 15, 2016, 2:24:17 AM9/15/16
to Django developers (Contributions to Django itself)
See Sjoerd's code in a later reply to this thread, let's start experimenting there. /E
> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
--

Cordialement, Coues Ludovic
+336 148 743 42

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Emil Stenström

unread,
Sep 15, 2016, 2:28:07 AM9/15/16
to Django developers (Contributions to Django itself)


On Tuesday, 13 September 2016 21:54:49 UTC+2, ludovic coues wrote:
There is third party module providing third party url function. Surlex
[1] have been mentionned. But any third party solution will need to
provide function compatible with django.conf.urls.url.
Line 64 of django/urls/revolvers.py is get_resolver. This function
return a RegexURLResolver, using is argument or the setting
ROOT_URLCONF as argument.

This make impossible, for exemple, to have resolver giving to the view
an int argument.

[1] http://codysoyland.com/2009/sep/6/introduction-surlex/

I fully agree that this is a problem. If you write "/articles/<int:year>/" I think the expectation is that year will be sent in as an int, not a string. So the int serves as two different things: a validator and a typecast.

Tim Graham: Does this change your view that this should be done outside of core? Do you buy the argument that beginners are unlikely to install third party packages when learning django?

Curtis Maloney

unread,
Sep 15, 2016, 2:38:21 AM9/15/16
to django-d...@googlegroups.com, Florian Apolloner
Somewhere I have code to provide a "parse" based URL router.

Will dig it up now 1.10 has has shipped
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Sjoerd Job Postmus

unread,
Sep 15, 2016, 3:03:21 AM9/15/16
to Django developers (Contributions to Django itself)
Hi :).

Yes, I also added the other syntax yesterday evening, so the <int:year> syntax is now fully supported. (But it does not yield an int!!).

Currently only `'int'` is registered as a valid type, with the regex r'[0-9]+'. More can be registered using `django_simple_url.register('hex', '[0-9a-fA-F]+')`.

One downside (still) is that it does not get cast to an int. Although I'm not really sure if I find it logical that it gets cast.

I don't really have that much time to work on it, but I'm hoping to add the `setup.py` either later today or shortly after the weekend.

Kind regards,
Sjoerd Job

Anthony King

unread,
Sep 15, 2016, 3:27:11 AM9/15/16
to django-d...@googlegroups.com

In my opinion, it should remain a string. That's the behaviour it is now, and it'll mean it can remain as a 3rd party package.

Perhaps to show it isn't being cast, it could be renamed to "integer", which would avoid confusion


--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Sjoerd Job Postmus

unread,
Sep 15, 2016, 3:56:16 AM9/15/16
to Django developers (Contributions to Django itself)
I'm not sure if I agree.

On the one hand I would like to say:

"I agree. For instance, if the type is `hex`, it would be really weird if it were to be cast to an int. For `uuid`, would you expect a `UUID` instance, or just a string?"

but alternatively, ...

"Wouldn't it be really cool if you could specify like `<model(myapp.Blog):post>`, and it does a `get_object_or_404(myapp.Blog, post)`, and passing the actual **instance** to the view?"

Django is really great in that it has a lot of "magic" behind the scenes which allow you to write simple code. To me a good library is simple, but not too simple.

Some libraries are made in order to make complicated tasks easy.
Some libraries are made in order to make repetitive tasks less repetitive.
Some libraries do both.

For instance, the Django ORM allows you to replace complicated SQL with simple Python (although for me I still sometimes prefer the 'simplicity' of SQL ;) ).
And Django generic views allow you to have less repetitive code for writing forms and listings.

In this case, we have a potential to get rid of the

    def my_sum_view(request, left_term, right_term):
        left_term = int(left_term)
        right_term = int(right_term)
        extra_logic_here

and move all the casting to some "magic" place in a library, where it is written only once instead of many times over.

As far as naming 'int' or 'integer'... Either the library does casting or does not. I think 'int' is more obvious **once** you know Python. But 'integer' might be an *alias*, though...

Let me re-iterate: I'm not sure what will be the best approach. It really is the trade off between a simple library having a singe responsibility, or a more complex library allowing simpler code. In this case I'm somewhat considering having a bit more complex library to allow a lot more simple code.

Kind regards,
Sjoerd Job
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Florian Apolloner

unread,
Sep 15, 2016, 4:20:29 AM9/15/16
to Django developers (Contributions to Django itself)


On Thursday, September 15, 2016 at 8:28:07 AM UTC+2, Emil Stenström wrote:
Tim Graham: Does this change your view that this should be done outside of core? Do you buy the argument that beginners are unlikely to install third party packages when learning django?

Imo it should still stay outside of core and we should adjust the current urlresolvers to allow for such typecasts somehow. This would have the benefit of allowing other url implementations to do the same and wouldn't limit us to a single option. 

Michal Petrucha

unread,
Sep 15, 2016, 4:38:09 AM9/15/16
to django-d...@googlegroups.com
On Thu, Sep 15, 2016 at 12:56:16AM -0700, Sjoerd Job Postmus wrote:
> I'm not sure if I agree.
>
> On the one hand I would like to say:
>
> "I agree. For instance, if the type is `hex`, it would be really weird if
> it were to be cast to an int. For `uuid`, would you expect a `UUID`
> instance, or just a string?"
>
> but alternatively, ...
>
> "Wouldn't it be really cool if you could specify like
> `<model(myapp.Blog):post>`, and it does a `get_object_or_404(myapp.Blog,
> post)`, and passing the actual **instance** to the view?"

As cool as this idea sounds, I really don't think the URL dispatcher
is a correct component to make database queries. FWIW, I've seen
similar magic implemented in view decorators, and the thing I remember
the most from this experience was that it made it a lot harder to
follow what was happening where.

Moreover, I can imagine this turning into a complicated mess of a
syntax to allow query customizations using a weird DSL really quickly.
After all, if we allow PK lookups, it's not that unreasonable to also
want to be able to lookup by other keys, and it all goes downhill from
here.

Cheers,

Michal
signature.asc

Sjoerd Job Postmus

unread,
Sep 15, 2016, 4:52:46 AM9/15/16
to Django developers (Contributions to Django itself)


On Thursday, September 15, 2016 at 10:38:09 AM UTC+2, Michal Petrucha wrote:

As cool as this idea sounds, I really don't think the URL dispatcher
is a correct component to make database queries. FWIW, I've seen
similar magic implemented in view decorators, and the thing I remember
the most from this experience was that it made it a lot harder to
follow what was happening where.

Moreover, I can imagine this turning into a complicated mess of a
syntax to allow query customizations using a weird DSL really quickly.
After all, if we allow PK lookups, it's not that unreasonable to also
want to be able to lookup by other keys, and it all goes downhill from
here.

Cheers,

Michal


Agreed. It all goes downhill from there, so let's at least not do database queries.

To me, that settles it: no typecasting.

Marc Tamlyn

unread,
Sep 16, 2016, 1:49:45 AM9/16/16
to django-d...@googlegroups.com

Fwiw, I spent a little time investigating this a couple of years ago. I would like to see Django officially bless the idea of alternative URL systems, and provide the "full" regex system and preferably at least one "simple" system - even if all that system supports is integers and slugs. This would allow third party authors to focus on designing their "to regex" translation, rather than the details of Django's resolving.

I did some investigation into swapping at a "deeper" level, including allowing mixed includes from one resolver type to another. This is made somewhat harder by the removal of "patterns", and was much more complex. However it did give much more flexibility in allowing URL patterns which route based on other attributes than the path. I dont have any working code, it was very conceptual. I think we should at least consider a more dramatic approach in a DEP, even if it is not the intended course.

Marc


--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Curtis Maloney

unread,
Sep 16, 2016, 3:31:12 AM9/16/16
to django-d...@googlegroups.com, Florian Apolloner
On 15/09/16 16:37, Curtis Maloney wrote:
> Somewhere I have code to provide a "parse" based URL router.
>
> Will dig it up now 1.10 has has shipped

Ah, found it...

So, here is a gist of a sample of using parse
(https://pypi.org/project/parse/) instead of regex.

https://gist.github.com/funkybob/3d90c57a837bc164d8b402a1c5b95a8b

Since you can register extra type names, and those types can cast also,
it covers a lot of things some people expect [but don't get] from regex.


--
C
> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com
> <mailto:django-develop...@googlegroups.com>.
> To post to this group, send email to django-d...@googlegroups.com
> <mailto:django-d...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/B35D1655-D658-41FC-9EB5-83311B6C892C%40tinbrain.net
> <https://groups.google.com/d/msgid/django-developers/B35D1655-D658-41FC-9EB5-83311B6C892C%40tinbrain.net?utm_medium=email&utm_source=footer>.

ludovic coues

unread,
Sep 16, 2016, 8:57:24 AM9/16/16
to django-d...@googlegroups.com
In my opinion, there is two point. First, core django should allow
different url resolver. Second, these different url resolver should
start as third party package.

Without first point, people need to hack django if they want to
experiment new kind of resolver. Like one providing typecasting or a
faster one or a localized one. And the resolver isn't "loosely
coupled" unlike most of the other part of django.

For the second point, I have a simple reason. Choosing between the
rail or the werkzeug syntax is bike shedding. It is not a technically
choice. It's an aesthetic one. There is no right answer, only lost
time.
> email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/9ab6342b-f969-4a3b-1263-aabd27cc0eb9%40tinbrain.net.
>
> For more options, visit https://groups.google.com/d/optout.



Chris Foresman

unread,
Sep 16, 2016, 9:34:41 AM9/16/16
to Django developers (Contributions to Django itself)
I'd really, really like an alternate URL resolver which does typecasting. I mean, if I'm specifying the type right there, why not expect the resolved to be the type I just specified? In 995 of URLs, you're talking about three basic types anyway: strings, integers, and (increasingly) UUIDs. After 5 years it still trips me up when I do a comparison with a number grabbed from a URL and, say, Object.otherobject_id.

Emil Stenström

unread,
Sep 16, 2016, 9:51:12 AM9/16/16
to django-d...@googlegroups.com
On 2016-09-16 09:30, Curtis Maloney wrote:
> On 15/09/16 16:37, Curtis Maloney wrote:
>> Somewhere I have code to provide a "parse" based URL router.
>>
>> Will dig it up now 1.10 has has shipped
>
> Ah, found it...
>
> So, here is a gist of a sample of using parse
> (https://pypi.org/project/parse/) instead of regex.
>
> https://gist.github.com/funkybob/3d90c57a837bc164d8b402a1c5b95a8b
>
> Since you can register extra type names, and those types can cast also,
> it covers a lot of things some people expect [but don't get] from regex.

This look great, thanks for sharing!

Would love to see casting like this included in Sjoerds library.

/Emil

Marten Kenbeek

unread,
Sep 16, 2016, 11:01:39 AM9/16/16
to Django developers (Contributions to Django itself)
I actually do have working code[1] that among other things abstracts the regex to a Constraint, which allows you to easily inject your own custom logic for resolving and reversing url fragments. This allows for a "simple" system at a more fundamental level, rather than just translating the "simple" system to a regex-based system. It has full support for routing on any property of the request. The code is mostly in a finished state, and the public API is fully backwards compatible, though there are a few small changes in Django's current master that'll have to be merged manually or rewritten. The biggest hurdle is that it still doesn't have any documentation.

I'm afraid I don't have the time to finish it by myself at the moment, as I'm juggling work, a new bachelor (computer science!) and other activities. I'd like to continue and finish my work when I have the time (and motivation), but if someone were to step in where I left off, I'd be happy to provide any assistance required. 


On Friday, September 16, 2016 at 7:49:45 AM UTC+2, Marc Tamlyn wrote:

Fwiw, I spent a little time investigating this a couple of years ago. I would like to see Django officially bless the idea of alternative URL systems, and provide the "full" regex system and preferably at least one "simple" system - even if all that system supports is integers and slugs. This would allow third party authors to focus on designing their "to regex" translation, rather than the details of Django's resolving.

I did some investigation into swapping at a "deeper" level, including allowing mixed includes from one resolver type to another. This is made somewhat harder by the removal of "patterns", and was much more complex. However it did give much more flexibility in allowing URL patterns which route based on other attributes than the path. I dont have any working code, it was very conceptual. I think we should at least consider a more dramatic approach in a DEP, even if it is not the intended course.

Marc

On 15 Sep 2016 9:52 a.m., "Sjoerd Job Postmus" <sjoe...@sjec.nl> wrote:


On Thursday, September 15, 2016 at 10:38:09 AM UTC+2, Michal Petrucha wrote:

As cool as this idea sounds, I really don't think the URL dispatcher
is a correct component to make database queries. FWIW, I've seen
similar magic implemented in view decorators, and the thing I remember
the most from this experience was that it made it a lot harder to
follow what was happening where.

Moreover, I can imagine this turning into a complicated mess of a
syntax to allow query customizations using a weird DSL really quickly.
After all, if we allow PK lookups, it's not that unreasonable to also
want to be able to lookup by other keys, and it all goes downhill from
here.

Cheers,

Michal


Agreed. It all goes downhill from there, so let's at least not do database queries.

To me, that settles it: no typecasting.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Sjoerd Job Postmus

unread,
Sep 19, 2016, 2:57:12 AM9/19/16
to Django developers (Contributions to Django itself)
Just wanted to announce the following: it's available on pypi:

https://pypi.python.org/pypi/django-simple-url/0.0.2

Kind regards,
Sjoerd Job

Sjoerd Job Postmus

unread,
Sep 20, 2016, 6:30:56 PM9/20/16
to Django developers (Contributions to Django itself)
Hi,

Before I looked into the code, I found this really hard to believe. In my mind, the whole resolving framework would be built upon classes providing `resolve` and `reverse` methods. I was only partially right. I looked into it a bit more and wrote up my conclusions here: http://sjoerdjob.com/post/is-djangos-url-routing-tightly-coupled/ .

TL;DR:

- The URL routing is mostly loosely coupled, with the exception of the `checks` and the `reverse` method. Furthermore, the `RegexURLResolver` is coupled to the idea that all sub-resolvers/patterns are also regex-based.
- By moving the `check` responsibility to the resolvers (same as with model-checks) and `reverse` responsibility to the resolvers as well, the coupling is lowered, and we would actually be able to switch in a completely different set of resolvers.
- It might be a good idea to make `RegexURLResolver.reverse` agnostic of the sub-resolvers.

Is there any form of coupling I missed?

Kind regards,
Sjoerd Job

Emil Stenström

unread,
Sep 20, 2016, 8:16:27 PM9/20/16
to django-d...@googlegroups.com
Impressive! Using it in a project right now, so much nicer for a
non-beginner like me too. No more regexs!

Looking forward to a Django patched that allows casting of the variables
too.

/Emil
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/u6sQax3sjO4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> django-develop...@googlegroups.com
> <mailto:django-develop...@googlegroups.com>.
> To post to this group, send email to django-d...@googlegroups.com
> <mailto:django-d...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/54cdc465-ff11-47ab-905b-0ae6e19c4e83%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/54cdc465-ff11-47ab-905b-0ae6e19c4e83%40googlegroups.com?utm_medium=email&utm_source=footer>.

Tom Christie

unread,
Sep 22, 2016, 8:09:53 AM9/22/16
to Django developers (Contributions to Django itself)
Wanted to add my support for this.
Personally I'd be totally in favour of considering something along these lines for core, once there's a fully proven design.

The capture syntax is:

* Far simpler.
* Meets pretty much every single real-world case I ever see.
* Gives us type coercion.

Given that we could continue to keep the existing style around as well, this is an area where I'd be happy to see Django take a step forward.

Sticking with the path->existing regex mapping implementation sounds like the best tack, at least initially, rather than getting bogged down in API discussions around how we'd tackle a routing API if we wanted to support non-path routing.

Cheers,

  Tom

Sjoerd Job Postmus

unread,
Sep 22, 2016, 10:59:12 AM9/22/16
to Django developers (Contributions to Django itself)
I'll try and update my library over the next couple of days to also support regex fragments and converters supporting parameters.

Another part I see is that the high coupling between Django and the URL resolvers (as mentioned in http://sjoerdjob.com/post/is-djangos-url-routing-tightly-coupled/ ) should probably be cleaned up, if it is desired to someday support alternative URL resolvers. I'm willing to provide a patch for the checking part, but I'm not sure if it would be accepted. Do I need to open a ticket for that?

Kind regards,
Sjoerd Job

Alasdair Nicol

unread,
Sep 22, 2016, 11:39:31 AM9/22/16
to Django developers (Contributions to Django itself)
On Thursday, 22 September 2016 15:59:12 UTC+1, Sjoerd Job Postmus wrote:
Another part I see is that the high coupling between Django and the URL resolvers (as mentioned in http://sjoerdjob.com/post/is-djangos-url-routing-tightly-coupled/ ) should probably be cleaned up, if it is desired to someday support alternative URL resolvers. I'm willing to provide a patch for the checking part, but I'm not sure if it would be accepted. Do I need to open a ticket for that?

I wrote the initial url checks code. The current tight coupling is the way the code turned out, it wasn't a deliberate design choice on my part. I read your blog post, and changing the code as you suggest so that the resolver can check itself sounds good to me. Yes, please open up a ticket to go with your patch.

Cheers,
Alasdair

Sjoerd Job Postmus

unread,
Sep 23, 2016, 2:53:49 AM9/23/16
to Django developers (Contributions to Django itself)

Hi,

Thank you for your feedback. I created the ticket: https://code.djangoproject.com/ticket/27262. I'm not sure on what timescale I can supply a patch though. Should it be marked as easy picking in the hope somebody else can take it up?

I hope I did not offend you with my post on how the code turned out. That was not my intention, but if it did offend you: my apologies.

Kind regards,
Sjoerd Job

Alasdair Nicol

unread,
Sep 23, 2016, 5:09:18 AM9/23/16
to Django developers (Contributions to Django itself)
On Friday, 23 September 2016 07:53:49 UTC+1, Sjoerd Job Postmus wrote:

I hope I did not offend you with my post on how the code turned out. That was not my intention, but if it did offend you: my apologies.


Don't worry, no offence taken :)

Cheers,
Alasdair 

Alexandr Shurigin

unread,
Sep 29, 2016, 1:29:46 PM9/29/16
to Django developers (Contributions to Django itself)
Hi all,

I already solved the problem for hard urls writing/reading. Please take a look, maybe you are interested in it :)

https://github.com/phpdude/django-macros-url

It provides simple & clear way to build your strong urls patterns.

Example from project page is attached

urlpatterns = patterns(
    'yourapp.views',
    url('^:category_slug/$', 'category'),
    url(':category_slug/:product_slug/', 'category_product'),
    url(':category_slug/:product_slug/:variant_id', 'category_product_variant'),
    url('news/', 'news'),
    url('news/:year/:month/:day', 'news_date'),
    url('news/:slug', 'news_entry'),
    url('^order/:id$', 'order'),
    url('^$', IndexView),
)


On Monday, September 12, 2016 at 10:32:45 PM UTC+2, Emil Stenström wrote:
Hi Djangonauts,

I'm just back from my second year of teaching Django to absolute beginners. The course is a combination of HTML, CSS, Python, and Django, and after five days of coding they have a real live website that they can show to friends. It's always such a great experience to see the look in their eyes when they finally understand how they can tame Django to do what they want.

There's one big thing that keeps tripping them up is urls.py. When specifying URL:s I get lots of questions about the regexes that they have to specify. First: there's a strange "r" in front of each line: r"regex". That means I will have to explain string escaping to them. Then there's the "^" and "$" signs, both which requires explaining regular expressions at length. Then there's [0-9]+ and finally there's the parenthesis around the regex. All in all, looking at URLs from a beginners perspective, they are a bunch of hieroglyphs, and very hard for beginners to grasp right away.

I'm not suggesting that urls.py are changed for most users, I'm suggesting that *simple_url* method (name inspired by simple_tag) is added to django.conf.urls that new users can use to get started quickly. This means that most beginners can postpone learning regexes a couple of months. The exact syntax that simple_url takes isn't important to me, as long it's a lot more beginner friendly than what we have today:

https://docs.djangoproject.com/en/1.10/topics/http/urls/#example

Just to get the ideas flowing, here's a suggestion, inspired by rails (again, exact syntax isn't important to me, simplicity to beginners is, so feel free to suggest something else if you agree that this is an important issue):

from django.conf.urls import simple_url
from . import views
urlpatterns
= [
    simple_url
('articles/2003/', views.special_case_2003),
    simple_url
('articles/:year)/', views.year_archive),
    simple_url
('articles/:year/:month/', views.month_archive),
    simple_url
('articles/:year/:month/:day/', views.article_detail),
]

All parameters would be passed to the view as keyword parameters with the name given and as a string, and validation would happen there instead.

I'm thinking there should be no settings
with simple_url, and that any more advanced use-case should switch to using url instead.

Two questions:

A) What do you think about the prospect of simplifying urls.py for beginners?
B) What do you think about the specific suggestion to mimic Rails urls with a simple_url tag?

Thanks for reading!

Message has been deleted

Sven R. Kunze

unread,
Sep 29, 2016, 2:11:31 PM9/29/16
to Django developers (Contributions to Django itself)
Hi Emil,

that's not only a teaching issue. Also experienced devs struggle with this way to declaring URLs.


Internally, we developed a UrlMixin for our view classes such as:


class MyView(UrlMixin, View):
    url_pattern = r'^bar/foo/$'
    view_name = 'my-index'
    # rest of view code


That's all we need to do in order to define a view with a working url pattern.


Reverse works with the optional view_name or by reversing the class itself.

Maybe, the simplification of your proposal could be explored by providing such a UrlMixin first.


Best,
Sven

Sjoerd Job Postmus

unread,
Sep 29, 2016, 2:42:47 PM9/29/16
to django-d...@googlegroups.com

I have a feeling this is orthogonal to the original request.

The original request/problem was "regex is hard".
Your response answers/solves "the URL definition is somewhere different from the view definition".

Both issues are realistic [1], and orthogonal.

[1]: I myself have great aversion to the approach where URL patterns are spread out over the codebase, but I know others are greatly prefer the approach that Flask takes.


--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Sven R. Kunze

unread,
Sep 29, 2016, 3:21:36 PM9/29/16
to Django developers (Contributions to Django itself)


Am Donnerstag, 29. September 2016 20:42:47 UTC+2 schrieb Sjoerd Job Postmus:

I have a feeling this is orthogonal to the original request.

The original request/problem was "regex is hard".
Your response answers/solves "the URL definition is somewhere different from the view definition".

Both issues are realistic [1], and orthogonal.


Oh maybe, I didn't fleshed that response out appropriately. What I wanted to say, is that his regex-light approach might be worth-while implemented in a third-party package such as described by others and me before inclusion in Django core (which also prefer sooner rather than later).
 

[1]: I myself have great aversion to the approach where URL patterns are spread out over the codebase, but I know others are greatly prefer the approach that Flask takes.


I tended to have this kind of aversion as well but after a while of using the UrlMixin, I am in love with it.

I think kind of this adaptation stems from the fact that in 99%, 1 URL corresponds to 1 View and vice-versa. Remove the view, you remove the URL as well and vice versa. Put it differently, it makes no sense to separate URl and view in different locations. That's at least our experience while doing Web-based applications and APIs for years.

Best,
Sven

João Sampaio

unread,
Sep 30, 2016, 8:59:29 AM9/30/16
to django-d...@googlegroups.com
I also prefer URLs all concentrated in a single place, instead of spread out all over the codebase. The reason is that it is dead simple to track which view is associated with a URL, just look at the ROOT_URLCONF file and track the view following the patterns. If you have them spread out, you'll need to use grep or something to find the URL pattern. But that's not always simple, because the URL could be a complex regex with custom parameters and grep is not suitable anymore.

URLs in view code works great for small projects. Now try that with a project with many apps, many urls.py files and many URL endpoints to track.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages