delegating our static file serving

[Tim Graham]

Berker has worked on integrating gunicorn with runserver so that we might be able to deprecate our own homegrown webserver. Windows support for gunicorn is supposedly coming soon which may actually make the idea feasible. This way we provide a more secure solution out of the box (anecdotes indicate that runserver is widely used in production despite our documentation warnings against doing so).

On the pull request, Anssi had an idea to use dj-static to serve static/media files. My understanding is that we would basically integrate the code for dj-static into Django and then add a dependency on static. It could be an optional dependency since you might want to serve static files differently in production, but it would likely be more or less universally used in development. We could then say that django.views.static.serve (and its counterpart in staticfiles) is okay to use in production (and I guess people are already using them in production despite our warnings that they are not hardened for production use).

What do you think of this plan?

[Carl Meyer]

On 11/28/2014 07:15 AM, Tim Graham wrote:
> Berker has worked on integrating gunicorn with runserver

> <https://github.com/django/django/pull/3461> so that we might be able to

> deprecate our own homegrown webserver. Windows support for gunicorn is
> supposedly coming soon

> <https://github.com/benoitc/gunicorn/issues/524>which may actually make

> the idea feasible. This way we provide a more secure solution out of the
> box (anecdotes indicate that runserver is widely used in production
> despite our documentation warnings against doing so).
>
> On the pull request, Anssi had an idea to use dj-static

> <https://github.com/kennethreitz/dj-static> to serve static/media files.

> My understanding is that we would basically integrate the code for
> dj-static into Django and then add a dependency on static

> <https://pypi.python.org/pypi/static>. It could be an optional

> dependency since you might want to serve static files differently in
> production, but it would likely be more or less universally used in
> development. We could then say that django.views.static.serve (and its
> counterpart in staticfiles) is okay to use in production (and I guess
> people are already using them in production despite our warnings that
> they are not hardened for production use).
>
> What do you think of this plan?

I think it's a good plan.

Carl

[Jannis Leidel]

I don't think using dj-static is a good idea (after having fixed a few things myself in it). If we'd want to do some form of "simpler" file serving we should be going with whitenoise (http://whitenoise.evans.io/) because:

- it has a deeper integration with Django (e.g. knows about the manifest storage backend in staticfiles)
- has a more flexible API than the Cling based dj-static for custom paths like file uploads and more
- works great with CDN/load balancer backed deployments via far-future cache headers for content that doesn't change
- supports serving gzipped content when wanted

To me this is currently the one-stop-solution when using staticfiles and I would be stoked to see it or its techniques be adopted.

Jannis

[Collin Anderson]

Hi All,

I think doing something here is really good idea. I'm happy with any of the solutions mentioned so far.

My question is: what does static/dj-static do that our built-in code doesn't do? What makes it more secure? It seems to me we're only missing is wsgi.file_wrapper and maybe a few more security checks. Why don't we just make our own code secure and start supporting it?

Here's basic wsgi.file_wrapper support: https://github.com/django/django/pull/3650

We could then, over time, start supporting more extensions ourselves: ranges, pre-gziped files, urls with never-changing content, etc. That way we get very, very deep django integration. It seems to me this is a piece that a web framework should be able to support itself.

Collin

[David Evans]

It's worth flagging up that part of what makes WhiteNoise good for serving static files in production also makes it not quite as good for using in development.

Most file servers work by taking the requested URL, constructing a local path from it, and then checking whether a file exists at that particular path. WhiteNoise on the other hand, searches for all available static files on startup and then builds a dictionary of files which it assumes won't change until the code is reloaded. This has a couple of big advantages: one is security -- there's a whole class of path-traversal vulnerabilities that I don't have to worry about because I never have to generate a path from a user-supplied URL. The other is that WhiteNoise can serve files from arbitrary URLs, rather than just within a fixed prefix like `/static/`, without adding an extra hit on the filesystem for every single request that comes in.

While this works well in production, it breaks down in development where the static files change without causing a reload of the server. I've started getting bug reports from users who are trying to use WhiteNoise in development and are getting confused by this.

So, while I'd obviously be delighted to see some of WhiteNoise's features merged into core I think there will need to be a slightly different approach for serving files in development.

Dave

[Collin Anderson]

Hi All,

I'm pretty interested in getting secure and _somewhat_ efficient static file serving in Django.

Quick history:

2005 - Jacob commits #428: a "static pages" view.  Note that this view should only be used for testing!"

2010 - Jannis adds staticfiles. Serving via django is considered "grossly inefficient and probably insecure".

2011 - Graham Dumpleton adds wsgi.file_wrapper to Gunicorn.

2012 - Aymeric adds StreamingHttpResponse and now files are read in chunks rather than reading the entire file into memory. (No longer grossly inefficient IMHO.)

I propose:

- Deprecate the "show_indexes" parameter of static.serve() (unless people actually use it).

- Have people report security issues to secu...@djangoproject.com (like always)

- Audit the code and possibly add more security checks and tests.

- add wsgi.file_wrapper support to responses (5-line proof of concept: https://github.com/django/django/pull/3650 )

- support serving static files in production, but still recommend nginx/apache or a cdn for performance.

- make serving static files in production an opt-in, but put the view in project_template/project_name/urls.py

I think it's a huge win for low-traffic sites or sites in the "just trying to deploy and get something live" phase. You can always optimize later by serving via nginx or cdn.

We already have the views, api, and logic around for finding and serving the correct files.

We can be just as efficient and secure as static/dj-static without needing to make people install and configure wsgi middleware to the application.

We could have staticfiles classes implement more complicated features like giving cache recommendations, and serving pre-gzipped files.

Is this a good idea? I realize it's not totally thought through. I'm fine with waiting until 1.9 if needed.

Collin

[Aymeric Augustin]

Yes, I support this plan.

"Serve your files with nginx!" doesn't fly in the age of PaaS.

Serving static files with Django and having a CDN cache them is a reasonable setup as far as I know.

I don't know if the "probably insecure" argument still holds. Are there some specific security risks in serving files that don't exist in serving dynamic content? I'd say dynamic content is more difficult. This main problem I can imagine when serving files is directory traversal. We already have protections against this class of attacks in several features. (Usual disclaimer: my background is security is mostly theoretical.)

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/bfc1eb0d-8b69-4450-bfe0-7147ef729317%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Aymeric.

[Loïc Bistuer]

I'm +1 on this plan, "native" support for wsgi.file_wrapper makes a lot of sense.

--
Loïc

[Michael Manfre]

Requiring fewer moving parts for small sites is definitely an improvement. +1 to Collin's plan.

Regards,

Michael Manfre

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1603744A-A45A-4FDE-B5D6-3814CE589692%40gmail.com.

[David Evans]

+1 to this: I'm all in favour of supporting production-adequate static file handling in core. A couple of small points:

We already have the views, api, and logic around for finding and serving the correct files.

One question that needs to be thought through is the role of `collectstatic` and `STATIC_ROOT`.

The existing static.serve view just ignores `STATIC_ROOT` completely and uses the configured StaticFileFinders to locate files. This means if you use the ManifestStaticFilesStorage backend -- or anything that uses the `post_process` hook -- it won't find your processed files at all.

Probably the simplest thing is to switch on DEBUG, and serve files from `STATIC_ROOT` (falling back to using the finders) when DEBUG is False, and use the existing behaviour otherwise.

We could have staticfiles classes implement more complicated features like giving cache recommendations, and serving pre-gzipped files.

Definitely +1 to using classes (CBVs, presumably) to handle this. The existing views don't easily support customising behaviour.

Regards,

Dave

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/N0KbgDeLuUE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

[Carl Meyer]

On 12/04/2014 10:33 PM, Collin Anderson wrote:
> Hi All,
>

I also think this is a good plan. It certainly makes sense to look at
"static" and "whitenoise" for ideas and compare their code to ours to
see where we could be more efficient or secure, but it's much less churn
for Django users if we simply improve our existing code rather than pull
in something wholly new.

Carl

[David Evans]

Sorry to revive an old thread here, but I just wanted to add that v2.0 of WhiteNoise now supports serving development files, providing the same behaviour as runserver currently does in DEBUG mode. (There were enough people wanting to do their development using gunicorn rather than runserver to make this worthwhile.)

This means that WhiteNoise is now a one-stop-shop for static file handling in Django. If there's still an appetite for integrating it, or something equivalent, into core I'd be happy to help out.

Dave

[Tim Graham]

I'd like to work together with Dave to develop a proof of concept that integrates whitenoise into Django. I spent about an hour looking through whitenoise and our own static file serving code, and I think integrating whitenoise will yield a simpler user experience with about the same amount of code as have now.

Essentially, we'd recommend adding something like this to existing wsgi.py files (it would be in the default startproject template)

from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(application)
application.add_files(settings.MEDIA_ROOT, prefix=settings.MEDIA_URL)

which would have the benefit of working out of the box in production too, I think. Of course, you could disable that based on settings.DEBUG or some other toggle.

We could then deprecate:

* django/contrib/staticfiles/views.py
* django/contrib/staticfiles/management/commands/runserver.py
* django/contrib/staticfiles/handlers.py
* django/views/static.py

Any objections to doing further investigation in this area?

[David Evans]

I'd be very happy to help with this. Two things to mention:

1. When I first wrote WhiteNoise the FileResponse class didn't exist and so the only way I could access `wsgi.file_wrapper` was by wrapping the wsgi application directly and bypassing Django. Now we have the FileResponse class it would be possible to implement WhiteNoise as standard Django middleware without having to edit wsgi.py. I think this would be a much cleaner approach, and I don't think it will be too much work to implement. I should have time to look at this next week.

2. Serving media files is a slightly different case. WhiteNoise was designed around the assumption that it's serving a set of developer-supplied, public files which remain unchanged during the lifetime of the process. This simplifies a lot of performance and security concerns that come with serving mutable, user-supplied files. At present, if you use `add_files(settings.MEDIA_ROOT, prefix=settings.MEDIA_URL)` as you suggested then WhiteNoise won't pick up any files that were uploaded after the application was started -- at least, not unless you enable the "autorefresh" setting which I explicitly don't recommend for production.

Obviously it's possible to support this if we decide this is an important goal but it will need more thought and will definitely be more work than just supporting static files.

[Florian Apolloner]

On Tuesday, December 29, 2015 at 6:31:02 PM UTC+1, David Evans wrote:

2. Serving media files is a slightly different case. WhiteNoise was designed around the assumption that it's serving a set of developer-supplied, public files which remain unchanged during the lifetime of the process. This simplifies a lot of performance and security concerns that come with serving mutable, user-supplied files.

Speaking from a  security perspective: It is also important that WhiteNoise would serve Media files in a safe way, ie serving most files as attachments to prevent uploaded HTML files from starting an attack, preventing browser content type sniffing etc…

[Chris Cogdon]

Also consider the media files could number into the millions (or bajizillions?). Particularly for, say, a image hosting application. Clearly it would not be feasible to enumerate all the files, and there would clearly be regular additions. It might be that this use case is simply "beyond the scope" of a django-supplied static files server, and the developer should instead use nginx, lighttpd or something similar.

[Cristiano Coelho]

Just curious, about PaaS, I know AWS (Amazon) deploys python/django on apache (which is quite decent), and apache also serves static files decently. What would be wrong with this?
The idea of having Python serving static files (and potentially gziping it), when python is one of the slowest languages out there, I don't think it is really a good idea. You should be really serving static files with something implemented in C/C++ or similar high performant language (read nginx or apache). How wrong am I?

[Florian Apolloner]

You are free to use what your PaaS provider supplies, but if you are using heroku (and many others) you do not have a choice. If AWS allows you to use apache 2.4 in event mode, then by all means use it! But there are plenty of platforms out there where you cannot easily serve static files and/or performance does not matter too much. So in the end it boils down if we can remove code from Django, if yes it is a win, even if it is just used for the development server.

[Jannis Leidel]

> On 29 Dec 2015, at 01:36, Tim Graham <timog...@gmail.com> wrote:
>
> I'd like to work together with Dave to develop a proof of concept that integrates whitenoise into Django. I spent about an hour looking through whitenoise and our own static file serving code, and I think integrating whitenoise will yield a simpler user experience with about the same amount of code as have now.
>
> Essentially, we'd recommend adding something like this to existing wsgi.py files (it would be in the default startproject template)
>
> from whitenoise.django import DjangoWhiteNoise
> application = DjangoWhiteNoise(application)
> application.add_files(settings.MEDIA_ROOT, prefix=settings.MEDIA_URL)
>
> which would have the benefit of working out of the box in production too, I think. Of course, you could disable that based on settings.DEBUG or some other toggle.
>
> We could then deprecate:
>
> * django/contrib/staticfiles/views.py
> * django/contrib/staticfiles/management/commands/runserver.py
> * django/contrib/staticfiles/handlers.py
> * django/views/static.py
>
> Any objections to doing further investigation in this area?

None, this sounds like a great combination and a logical continuation of what I did back then with staticfiles.

The fact that staticfiles had all this custom logic to handle file serving in staticfiles was a necessary evil to stay backward compatible and lower the risk of staticfiles becoming a failure -- it wasn’t clear at all if it’d match the workflow for most of Django’s user base (and it still hasn’t completely I think). Whitenoise is agnostic enough about where the served files come from so this fits nicely.

> --
> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.

> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a2136cbb-2dfb-4269-8bb9-64a55f8c7b6a%40googlegroups.com.

[Anssi Kääriäinen]

In my opinion one of the most important goals for Django is to make
deploying tiny projects securely and easily a trivial matter. We
aren't there right now.

If you have a site where traffic is less than one hit per minute, you
shouldn't care about performance that much.

With Django, it is pretty easy to write a decent "one-off" application
in an hour. Unfortunately, proper (that is, secure and automated)
deployment takes a lot longer than that.

Maybe part of the problem is that core developers tend to work on
larger sites, where you likely want a customized deployment anyways,
and working on deployment scripts doesn't take such a large portion of
the total time.

Anything that improves the development experience for tiny sites gets
a big +1 from me.

- Anssi

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9395314F-C692-4F9E-A2B5-B6A0BF5610A1%40gmail.com.

[Guilherme Leal]

>>Anything that improves the development experience for tiny sites gets
>>a big +1 from me.

As a solo developer that usualy deploy a diferent application every 3 weeks or so, cant agree more.

Big +1 for me too.

[Aymeric Augustin]

2015-12-31 12:52 GMT+01:00 Anssi Kääriäinen <akaa...@gmail.com>:

In my opinion one of the most important goals for Django is to make
deploying tiny projects securely and easily a trivial matter. We
aren't there right now.

+1

I'm maintaining three such sites (my wife's, my consultancy's and mine). I'm

clearly not going to bother configuring a CDN. I deployed them with whitenoise

— which isn't nearly as well known as it should be. If I hadn't known about

it, I suspect I would have rolled a poor man's solution to serve static files

in Python, because that's the easiest when deploying to a PaaS.

Regarding the question of media files raised earlier in this thread, I think

we shouldn't attempt to cache all existing files on start-up because even

small sites could easily have tens of thousands e.g. a forum that stores

users' avatars. We should look up the file at each query, which will be

slightly less efficient, but functional and not subject to scalability issues.

For both static and media files, we care about simplicity and security. It

appears that whitenoise gives us great performance for static files as well,

which is good. If we can't have it for media files, it's still fine.

--

Aymeric.

[Collin Anderson]

I'm still +1 to django providing a good file solution, whether it be whitenoise, dj-static, or whatever.

Here's all I came up with the last time I looked into this. It basically involves passing insecure=True into our current code and sending cache headers. (I feel like the insecure flag could get renamed to inefficient or something like that.) I feel like the code we have really isn't all that bad. Using something like whitenoise or dj-static allows you to bypass middleware, which can really improve performance (if you have inefficient middleware installed), but on the other hand, some people may want to use the currently logged in user to determine whether or not to serve the file.

Collin

import re

from django.conf import settings
from django.conf.urls import url
from django.contrib.staticfiles.views import serve as static_serve
from django.core.exceptions import ImproperlyConfigured
from django.views.decorators.cache import cache_control
from django.views.static import serve


def static(prefix, view=serve, **kwargs):
    if not prefix:
        raise ImproperlyConfigured("Empty static prefix not permitted")
    if '://' in prefix:
        return []
    if not settings.DEBUG:
        view = cache_control(max_age=86400 * 30)(view)
    return [
        url(r'^%s(?P<path>.*)$' % re.escape(prefix.lstrip('/')), view, kwargs=kwargs),
    ]

urlpatterns = static(settings.STATIC_URL, view=static_serve, insecure=True)

if settings.MEDIA_URL and settings.MEDIA_ROOT:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

[Fabio Caritas Barrionuevo da Luz]

A question: are there any plans to also improve MEDIA files (user uploaded files) in any foreseeable future?

Perhaps this is outside the scope of Django, but I believe Django could provide by default, any option to get a little more control over who can and can not access the MEDIA files.

Most Django deployment tutorials with Nginx and Apache that saw out there do not say anything on the issue of data security.

I believe it is a common use case, that not every file sent by a User should be available and accessible to anyone on the web.

Out there, I found these two implementations:

https://github.com/benoitbryon/django-downloadview

https://github.com/johnsensible/django-sendfile


-- 

Fábio C. Barrionuevo da Luz
Palmas - Tocantins - Brasil - América do Sul

http://pythonclub.com.br/

[Curtis Maloney]

On 01/01/16 15:49, Fabio Caritas Barrionuevo da Luz wrote:
> A question: are there any plans to also improve MEDIA files (user
> uploaded files) in any foreseeable future?
>
> Perhaps this is outside the scope of Django, but I believe Django could
> provide by default, any option to get a little more control over who can
> and can not access the MEDIA files.

I certainly agree that at least some basic form of access control ought
be provided in core/contrib.

> Most Django deployment tutorials with Nginx and Apache that saw out
> there do not say anything on the issue of data security.
>
> I believe it is a common use case, that not every file sent by a User
> should be available and accessible to anyone on the web.

I was talking with Tom Eastman about a related topic at PyConAU this
year, and I believe the solution is multiple storage engines.

Basically, for each realm of access, have a separate file storage
instance with its own access policy.

We could then register each storage engine we want published with the
static/media server machinery, with its associated policy...

Hmm... I think I've just created myself a new project...

--
Curtis

[Aymeric Augustin]

Hello Collin,

> On 31 déc. 2015, at 23:44, Collin Anderson <cmawe...@gmail.com> wrote:
>
> I feel like the insecure flag could get renamed to inefficient or something like that.

This view is insecure for serving user-uploaded files, for reasons Florian hinted at.

Browsers “helpfully” get a lot things wrong for legacy reasons. As a consequence,
downloading a file is one of the most insecure things you can do in general.

I can’t say for sure if it’s insecure for serving static files. I believe Django has some
countermeasures against directory traversal attacks for example. However, since
the code wasn’t written with security in mind from the beginning, rewriting it from a
security perspective may be the most efficient way to make sure it’s secure.

Best regards,

--
Aymeric.

[Collin Anderson]

That makes sense. Thanks.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/db7e7fe6-c735-43bb-a3f7-2bc809707032%40googlegroups.com.

[David Evans]

I've just pushed a simple middleware wrapper that lets you install WhiteNoise by adding:

MIDDLEWARE_CLASSES = (

   ...

    'whitenoise.middleware.StaticFilesMiddleware',

   ...

)

See: https://github.com/evansd/whitenoise/blob/master/whitenoise/middleware.py

No messing with wsgi.py!

This approach will be marginally less efficient than handling things at the WSGI layer, but the integration is much cleaner.

[Gordon]

Apologies if this has already been mentioned but I read this thread in full awhile ago and now everything is collapsed.

If staticfiles serving is brought into django proper it would be nice if the storage api was used so that staticfiles can be stored remotely.  I just ran into this looking at whitenoise so I figured I would mention it here to be taken into consideration when putting an implementation together.

[Gordon]

I put together a simple django middleware based off the functionality of whitenoise except that it uses the staticfiles storage for finding/serving files.  I am sure there are lots of improvements to be made on it but it demonstrates using the filestorage api to accommodate remote static files.  It is roughly 100 lines.  https://gist.github.com/thenewguy/a92e05a3568fb6f1cc7c

[Curtis Maloney]

I should probably finish off my toy working on the same lines... but
introducing my earlier comments about per-source paths and permissions
mixins...

Very much a WIP, but I'm sure you'll see the idea:

https://gist.github.com/funkybob/a89c10e24716a4ce55cf

--
C

> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com

> <mailto:django-develop...@googlegroups.com>.

> To post to this group, send email to django-d...@googlegroups.com

> <mailto:django-d...@googlegroups.com>.

> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/c2b85443-4d50-425d-8b33-a05a048d3957%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/c2b85443-4d50-425d-8b33-a05a048d3957%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Thomas C]

Hello Curtis and Fabio, 

You can have a look at django- private-media, which implement a private storage engine :

https://github.com/RacingTadpole/django-private-media

Basic usage :

from private_media.storages import PrivateMediaStorage 

class Car(models.Model):
    photo = models.ImageField(storage=PrivateMediaStorage())

Regards,

Thomas.

[Aleksej Manaev]

Hello, I would like to work on this in the scope of my study project. Is someone already working on this?
I am fairly new to contributing to the framework, can you provide me more information on the current state of the issue, which problems need to be solved and which possible solutions there are available? Are there some resources I can read to educate myself in this topic?
I have already read the comments in this discussion and Django documentation

https://docs.djangoproject.com/en/1.10/howto/static-files/ and
https://docs.djangoproject.com/en/1.10/howto/static-files/deployment/.

[Tim Graham]

Hi, I don't know of any updates on this issue outside of this thread. I think you could read through the thread and summarize the problems and possible solutions just as well as I could. The solution might involve integrating whitenoise in Django, as well an adding some enhancements. The whitenoise author, David Evans, might have some more thoughts about specifics.

[Guilherme Leal]

I'm willing to cooperate on this one. Anyone is working on this and need help?

[David Evans]

I've been intending to work on this for a while (as the original author of WhiteNoise it would make the most sense for me to do the integration work) but as ever the problem has been finding the time. I've been hoping to make some progress on this over the Christmas period, but we'll see what happens!

[Guilherme Leal]

If you need some help, I'm as available as I can be.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9a6e143d-9bf9-4e26-89ca-cfc9944ba794%40googlegroups.com.

[Aleksej Manaev]

I started looking into this but have progressed slowely. At first I put the whitenoise project into the contrib folder of django.
Tim Graham suggested not to change much of the integration with the old WSGI interface, so I think the new middleware way would be alright too.

On Tuesday, 29 December 2015 00:36:06 UTC, Tim Graham wrote:

I'd like to work together with Dave to develop a proof of concept that integrates whitenoise into Django. I spent about an hour looking through whitenoise and our own static file serving code, and I think integrating whitenoise will yield a simpler user experience with about the same amount of code as have now.

Essentially, we'd recommend adding something like this to existing wsgi.py files (it would be in the default startproject template)

from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(application)
application.add_files(

settings.MEDIA_ROOT, prefix=settings.MEDIA_URL)

which would have the benefit of working out of the box in production too, I think. Of course, you could disable that based on settings.DEBUG or some other toggle.

We could then deprecate:

* django/contrib/staticfiles/views.py
* django/contrib/staticfiles/management/commands/runserver.py
* django/contrib/staticfiles/handlers.py
* django/views/static.py

Any objections to doing further investigation in this area?

As I tried to remove the suggested deprecated files I encoutered the dependency from StaticLiveServerTestCase to StaticFilesHandler.
StaticLiveServerTestCase allows the execution of tests without using collectstatic first which I think is not possible in Whitenoise. Do we add this functionality in Whitenoise or force the user to use collectstatic? Whitenoise depends on the staticfileapp and maybe a merge of the two components would be cleaner.
I wanted to do this as a project in my university and would like to offer my help.

[Aleksej Manaev]

Am I on the right track with the integration of WhiteNoise into Django? Can Someone lead me to the next steps that need to be done?

https://github.com/django/django/compare/master...AleksejManaev:ticket_27325?expand=1

I think "django/views/static.py" can't be deprecated because it is used for providing MEDIA_FILES during development and WhiteNoise is only responsible for STATIC_FILES.

[Florian Apolloner]

I think there is no need to vendor nowadays, we can just depend on whitenoise. (like we do for pytz)

[David Evans]

Hi Aleksej,

Thanks for your work on this, and sorry I haven't replied sooner -- just been busy with lots of other things ;)

My plan for integrating WhiteNoise was actually to merge it in as part of contrib.staticfiles and basically stop development on it as an independent project. When I first started WhiteNoise I went to quite a lot of effort to make the core framework-agnostic and to have all the Django specific code as an extension to the core. However this makes the code more complex than if it was written for Django alone. I'm also not sure it gets a lot of use outside of the Django world as other frameworks have their own way of doing things and the most useful features of WhiteNoise are Django only.

So the end-goal in my mind is to have a stripped down and simplified version of the WhiteNoise code bundled as part of Django itself, which I could then help maintain as part of the wider Django project. However, I don't think this process would be an ideal project for someone who is a new contributor to Django as it would require quite a detailed understanding of the design decisions and architectural tradeoffs behind WhiteNoise. Really this is the sort of thing I ought to be doing but it's been quite a busy year (new baby, work, etc etc) and I just haven't had the time.

If you're able to bundle WhiteNoise as it currently is with Django in a way that the core devs are happy with then that's great, and obviously I wouldn't stand in your way. But I can imagine that there might be some resistance to creating a hard dependency on WhiteNoise, while it's still a standalone project.

Again, thanks for the work you've put into this and sorry I don't have a more helpful response.

Dave