[Django] #30728: Django urls patterns resolver shadowing exceptions thrown everywhere else
Django
---------------------------------------+------------------------
Reporter: maks500 | Owner: nobody
Type: Bug | Status: new
Component: Core (URLs) | Version: 2.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------+------------------------
The new version of django somehow captures all the exceptions ocurred
while parsing the urlpatterns.
How to reproduce:
* Create a django project
* Create a an app
app/views.py
{{{
from django.shortcuts import render
class MyCustomExceptionShadowed(Exception):
pass
raise MyCustomExceptionShadowed('I have a description of your error')
# Create your views here.
def myview(request):
pass
}}}
urls.py
{{{
from django.contrib import admin
from django.urls import path
from bugtest import views
urlpatterns = [
path('', views.myview),
path('admin/', admin.site.urls),
]
}}}
When running the server I get
{{{
django.core.exceptions.ImproperlyConfigured: The included URLconf
'bug.urls' does not appear to have any patterns in it. If you see valid
patterns in the file then the issue is probably caused by a circular
import.
}}}
When in fact I expect it to say
{{{
File "/home/ariel-nt/Desktop/test/bug/bugtest/views.py", line 6, in
<module>
raise MyCustomExceptionShadowed('I have a description of your error')
bugtest.views.MyCustomExceptionShadowed: I have a description of your
error
}}}
I tested this with django 2.1 and it works as expected, so I'll now try to
find the root cause of the error and give an update here
--
Ticket URL: <https://code.djangoproject.com/ticket/30728>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: maks500 | Owner: nobody
Type: Bug | Status: new
Component: Core (URLs) | Version: 2.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by maks500:
Old description:
New description:
class MyCustomExceptionShadowed(Exception):
pass
from bugtest import views
}}}
The issue in introduced in this commit
[https://github.com/django/django/commit/097457afe47e50e76d53b1cd3312ba8364f866cb
#diff-46e69f287173eef41fcbfeba05501954R274]
This line
[https://github.com/django/django/blob/097457afe47e50e76d53b1cd3312ba8364f866cb/django/utils/autoreload.py#L274]
absorbs all the exceptions not showing them to the developer.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:1>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ariel Torti):
* component: Core (URLs) => Utilities
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:2>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Claude Paroz):
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:3>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* cc: Tom Forbes (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:4>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tom Forbes):
I cannot seem to reproduce this with the latest master
(bb9e82f2748ace292a584841ab9af8696df27f53), even with the same code as the
description:
{{{
Exception in thread django-main-thread:
Traceback (most recent call last):
...
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed
File
"/Users/tom/PycharmProjects/test_django_app/test_django_app/urls.py", line
6, in <module>
from test_app import views
File "/Users/tom/PycharmProjects/test_django_app/test_app/views.py",
line 6, in <module>
raise MyCustomExceptionShadowed('I have a description of your error')
test_app.views.MyCustomExceptionShadowed: I have a description of your
error
}}}
There is this snippet of code in the autoreloader:
{{{
try:
get_resolver().urlconf_module
except Exception:
# Loading the urlconf can result in errors during development.
# If this occurs then swallow the error and continue.
pass
}}}
which catches the exception (added to fix #30323), but the exception
should always be re-thrown by the system checks code in `runserver()`, or
if that's somehow disabled/silenced then when the view is requested.
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:5>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tom Forbes):
Also I cannot see how the given exception ("The included URLconf ...")
could be caused by the autoreloader. It's triggered from this code:
{{{
patterns = getattr(self.urlconf_module, "urlpatterns",
self.urlconf_module)
try:
iter(patterns)
except TypeError:
msg = (
"The included URLconf '{name}' does not appear to have any
"
"patterns in it. If you see valid patterns in the file
then "
"the issue is probably caused by a circular import."
)
raise ImproperlyConfigured(msg.format(name=self.urlconf_name))
}}}
If the urls.py code was evaluated at all then this code wouldn't be
reached and the exception not thrown. So, if this is not something
specific to the users environment then something very fishy is going on.
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:6>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Keryn Knight):
This looks like it could ''possibly'' be the same problem described in
#30500 (and #30674). As with those, knowing the version of Python being
used would be useful.
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:7>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Ariel Torti):
Replying to [comment:6 Tom Forbes]:
> Also I cannot see how the given exception ("The included URLconf ...")
could be caused by the autoreloader. It's triggered from this code:
>
> {{{
> patterns = getattr(self.urlconf_module, "urlpatterns",
self.urlconf_module)
> try:
> iter(patterns)
> except TypeError:
> msg = (
> "The included URLconf '{name}' does not appear to have
any "
> "patterns in it. If you see valid patterns in the file
then "
> "the issue is probably caused by a circular import."
> )
> raise
ImproperlyConfigured(msg.format(name=self.urlconf_name))
> }}}
>
> If the urls.py code was evaluated at all then this code wouldn't be
reached and the exception not thrown. So, if this is not something
specific to the users environment then something very fishy is going on.
Ok, so to make sure it was not my environment configuration getting on the
way I tried to reproduce it on a docker container.
With Python 3.7.0 and Django 2.2.4 (the latest version from pip) it worked
just right.
But then I tried running it with Python 3.5.2 (same version as my local
machine where I have the issue) and I got the
`django.core.exceptions.ImproperlyConfigured` instead of my exception.
Apparently is a race condition (maybe?) because when I run it again it
worked as expected, it failed only sometimes.
I will attach a file of the system configuration and the output showing it
doesn't fail every time I run it. There's also the bundled project, can
you try and check if you can reproduce it using Python 3.5.2
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:8>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ariel Torti):
* Attachment "bug.zip" added.
Bundled PoC project
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ariel Torti):
* Attachment "output.log" added.
System information and run output, you can see it appears the issue has
something to do with race condition because it doesn't trigger everytime I
run it.
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Ariel Torti):
Everything works just fine on Python 3.5.7, apparently it's a bug with
python itself.
So the solution would be to update python, do I set the ticket as fixed ?
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:9>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tom Forbes):
Let’s keep it open for now, thank you for your detailed and very helpful
diagnostics
Likely the fix will be just “update Python”, but I want to do some digging
around why exactly this happens and document it here.
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:10>
Django
Reporter: Ariel Torti | Owner: nobody
Type: Bug | Status: closed
Component: Utilities | Version: 2.2
Severity: Release blocker | Resolution: duplicate
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tom Forbes):
* status: new => closed
* resolution: => duplicate
Comment:
I'm going to close this in favour of #30500 which is the same issue, and
has a lot more information there. I've posted my findings so far in
https://code.djangoproject.com/ticket/30500#comment:14.
--
Ticket URL: <https://code.djangoproject.com/ticket/30728#comment:11>