LiveServerTestCase, and override_settings(DEBUG=True)

[Harry Percival]

Django's test runner overrides your settings to force DEBUG to be True, which I understand the intention behind, but it is occasionally annoying.  One solution for those cases is to use `override_settings`, but that has very weird effects when using `LiveServerTestCase`.

Minimal repro:

    django-admin.py startproject myproj
    cd myproj
    vi tests.py

In tests.py:

from django.test import LiveServerTestCase
from selenium import webdriver
from django.test.utils import override_settings

@override_settings(DEBUG=True)
class MinimalTest(LiveServerTestCase):

    def setUp(self):
        self.browser = webdriver.Firefox()
        self.browser.implicitly_wait(3)

    def tearDown(self):
        self.browser.quit()


    def test_admin_site_is_there(self):
        self.browser.get(self.live_server_url + '/admin')
        self.assertIn('Django administration', self.browser.find_element_by_tag_name('body').text)

Change DEBUG=True to DEBUG=False and this passes.  But, as-is, you get:

  File "/tmp/myproj/test1.py", line 18, in test_admin_site_is_there
    self.assertIn('Django administration', self.browser.find_element_by_tag_name('body').text)
AssertionError: 'Django administration' not found in 'Page not found (404)\nRequest Method: GET\nRequest URL: http://localhost:8081/admin\n"admin" does not exist\nYou\'re seeing this error because you have DEBUG = True in your Django settings file. Change that to False, and Django will display a standard 404 page.'

I'm just using the admin URL because it's one that is activated by default.  Normal URLs, eg a home page view, also return 404s.  What's going on?

[Russell Keith-Magee]

On Wed, Dec 18, 2013 at 11:57 PM, Harry Percival <harry.p...@gmail.com> wrote:

Django's test runner overrides your settings to force DEBUG to be True, which I understand the intention behind, but it is occasionally annoying.  One solution for those cases is to use `override_settings`, but that has very weird effects when using `LiveServerTestCase`.

I presume this is just a typo, but for clarity - it's the other way around. TestRunner forces DEBUG=False, with the theory being that you should be testing "production" behaviour as much as possible.

 

Minimal repro:

    django-admin.py startproject myproj
    cd myproj
    vi tests.py

In tests.py:

from django.test import LiveServerTestCase
from selenium import webdriver
from django.test.utils import override_settings

@override_settings(DEBUG=True)
class MinimalTest(LiveServerTestCase):

    def setUp(self):
        self.browser = webdriver.Firefox()
        self.browser.implicitly_wait(3)

    def tearDown(self):
        self.browser.quit()


    def test_admin_site_is_there(self):
        self.browser.get(self.live_server_url + '/admin')
        self.assertIn('Django administration', self.browser.find_element_by_tag_name('body').text)

Change DEBUG=True to DEBUG=False and this passes.  But, as-is, you get:

  File "/tmp/myproj/test1.py", line 18, in test_admin_site_is_there
    self.assertIn('Django administration', self.browser.find_element_by_tag_name('body').text)
AssertionError: 'Django administration' not found in 'Page not found (404)\nRequest Method: GET\nRequest URL: http://localhost:8081/admin\n"admin" does not exist\nYou\'re seeing this error because you have DEBUG = True in your Django settings file. Change that to False, and Django will display a standard 404 page.'

I'm just using the admin URL because it's one that is activated by default.  Normal URLs, eg a home page view, also return 404s.  What's going on?

 

Ok - the reason for this is a little complicated.

The LiveTestServer is actually three media servers, layered over each other (like an onion. Or a parfait :-). The outer layer is the static files server, the middle layer is the media file server, and the inner layer is the actual WSGI server providing your URLs.

Starting from the outside and moving in, each layer does a check to see "should I serve this URL"; if it does, it returns the appropriate page/file.

However, if the file *doesn't* match, the behaviour of these layers is different depending on the value of DEBUG. If DEBUG=False, it *raises* a Http404; if DEBUG=True, it *returns* a technical 404 response. This is the page you're seeing during the test -- the technical 404 response that says "I couldn't find a file name 'admin'".

The internals of the server implementation catches Http404 exceptions and tries a deeper layer; but if a 404 is returned as a view response, it just gets returned.

So - why does this behaviour change matter?

Well, the default value for MEDIA_ROOT is ''. Which means it matches *all* URL requests. When DEBUG=False, this doesn't matter - it just means no files are ever matches by the middle media layer, a 404 is raised, and the next layer is tried; but when you force DEBUG=True, the media layer dutifully says "I can't find that file" - for every request it receives. It doesn't hit this problem for static files because STATIC_ROOT = '/static/' by default, and the requested URL doesn't match, so the static layer is never asked to response with file or 404.

You only see this on the LiveTestServer because runserver doesn't include the Media layer -- it only serve staticfiles and the main app. If have to manually add the media handler to your URLconf if you want media files to be served, and that handler will run *after* all the normal URLs have been tried. And you don't normally see the problem during testing because, as you've noted, DEBUG is forced to False by the test running infrastructure.

The upside - If you add MEDIA_ROOT='/media/' to settings.py, the problem goes away, because it gives the media layer something to match against that won't catch all files.

Yours,

Russ Magee %-)

[Aymeric Augustin]

On 19 déc. 2013, at 02:14, Russell Keith-Magee <rus...@keith-magee.com> wrote:

> The upside - If you add MEDIA_ROOT='/media/' to settings.py, the problem goes away, because it gives the media layer something to match against that won't catch all files.

I hit that issue a month ago and filed https://code.djangoproject.com/ticket/21451 — with pretty much the same analysis.

--
Aymeric.

[Harry Percival]

Thanks gang.

@Russell, I assume you meant set MEDIA_URL = "/media/"

That works if I do it in settings.py.  Interestingly, it doesn't work if I try and do it via override_settings?

[Russell Keith-Magee]

On Thu, Dec 19, 2013 at 7:36 PM, Harry Percival <harry.p...@gmail.com> wrote:

Thanks gang.

@Russell, I assume you meant set MEDIA_URL = "/media/"

Heh. Now we're even on really dumb typos :-)

 

That works if I do it in settings.py.  Interestingly, it doesn't work if I try and do it via override_settings?

I'd need to go digging to be sure, but I'm going to guess that this is just an ordering thing. The live server is set up fairly early in test setup, and the MEDIA_URL setting is baked into the Media handler at time of construction, not being read from settings on demand. However, the DEBUG setting *is* read on demand. @override_settings applies to the individual test, so the setup (or, at least, the bits that involve the live server setup) are probably being invoked in pre-override conditions.

Yours,

Russ Magee %-) 

[Ramiro Morales]

On Wed, Dec 18, 2013 at 12:57 PM, Harry Percival
<harry.p...@gmail.com> wrote:
> Django's test runner overrides your settings to force DEBUG to be True,
> which I understand the intention behind, but it is occasionally annoying.
> One solution for those cases is to use `override_settings`, but that has
> very weird effects when using `LiveServerTestCase`.

What version of Django are you using?

--
Ramiro Morales
@ramiromorales