Hi, first of all, sorry for chiming in so late. It seems like this has been discussed in May, already, but I didn't follow the discussion back then.
Today the staticfiles contrib app was committed to trunk. This got me wondering: Wasn't there recently a discussion about how "contrib" is a bad idea and all apps should either be in core or live as separate projects? Is staticfiles really a core app? Even if it is a core app, does it do its job well enough to be in contrib?
I do agree that such an app is critical for practically all web projects and the documentation even says: """ For small projects, this [static files management] isn't a big deal, because you can just keep the media somewhere your web server can find it. However, in bigger projects -- especially those comprised of multiple apps -- dealing with the multiple sets of static files provided by each application starts to get tricky. """
So like the docs say, staticfiles seems to target "bigger projects". However, what staticfiles does has almost nothing to do with "bigger project" asset management. Just look at the features grid on djangopackages (disclaimer: I'm the author of django-mediagenerator and I maintain that grid): http://djangopackages.com/grids/g/asset-managers/ It's obvious that django-staticfiles has none of the critical features: * It doesn't combine your CSS/JS files. * It doesn't compress them. * It doesn't support CSS compilers like Sass or Less. * It doesn't support versioning of CSS, JS, and images. * Because of the lack of versioning the staticfiles view doesn't (and can't) support proper caching which means for every single visited page the browser will make unnecessary "if-modified-since" roundtrips to the very slow runserver. These unnecessary HTTP roundtrips make it very painful to work even on medium sized projects (unless combined with a local dev http server like Apache). * Finally, staticfiles doesn't auto-regenerate the assets when they are changed. This makes development extremely painful and error-prone because you have to "manage.py collectstatic" every time you make a little change to your JS or CSS code (and that's really easy to forget).
From that point of view, staticfiles is only useful for small hobby projects. So, why was this app added to Django? Most projects are better off using a different solution, anyway.
BTW, I noticed a bug in the staticfiles view: It checks for "if settings.DEBUG", but that should be "if not settings.DEBUG".
Also, staticfiles doesn't index "media" folders although the admin uses "media" instead of "static". Why not follow the existing scheme?
> However, what staticfiles does has almost nothing to do with "bigger
> project" asset management. Just look at the features grid on
> djangopackages (disclaimer: I'm the author of django-mediagenerator
> and I maintain that grid):http://djangopackages.com/grids/g/asset-managers/ > It's obvious that django-staticfiles has none of the critical features:
> * It doesn't combine your CSS/JS files.
> * It doesn't compress them.
> * It doesn't support CSS compilers like Sass or Less.
> * It doesn't support versioning of CSS, JS, and images.
> * Because of the lack of versioning the staticfiles view doesn't (and
> can't) support proper caching which means for every single visited
> page the browser will make unnecessary "if-modified-since" roundtrips
> to the very slow runserver. These unnecessary HTTP roundtrips make it
> very painful to work even on medium sized projects (unless combined
> with a local dev http server like Apache).
> * Finally, staticfiles doesn't auto-regenerate the assets when they
> are changed.
And the "lack" of all of these features is precisely why staticfiles
is well suited to contrib, whereas something like mediagenerator is
not (even though it may be very useful). Staticfiles has a very
specific, well-defined purpose (collecting media files from apps),
which fills a major hole in the Django "story" for reusable apps. It
does not come with a pile of additional features outside that scope,
adding much more code and more areas where people may reasonably
disagree on the best implementation.
Every feature you list here can easily be provided by a separate app
on top of staticfiles. I currently use staticfiles in combination with
django_compressor, which provides all these additional features
(compression, combining, versioning, auto-regenerating when changed),
and the combination works very well. I prefer having separable
concerns in separate apps whenever possible, and the tasks performed
by staticfiles and compressor are very much separable.
> This makes development extremely painful and error-prone
> because you have to "manage.py collectstatic" every time you make a
> little change to your JS or CSS code (and that's really easy to
> forget).
In development you generally would not run "manage.py collectstatic"
at all; you use the dynamic-serving view instead so changes are
reflected instantly.
> Staticfiles has a very specific, well-defined purpose (collecting media > files from apps), which fills a major hole in the Django "story" for > reusable apps.
IMHO contrib apps should have the following characteristics (and probably more): * Solves a problem that can be described in one short sentence. * Solves a fundamental problem in web development, including problems involving the reusable app paradigm. * Doesn't restrict the programmer from using other tools to solve the same problem. * Promotes a convention among developers.
Staticfiles solves the problem of reusable app media collection and doesn't restrict the developer from using something else. It also promotes the convention of storing media within a "static" directory, which will help adoption of new reusable apps. To me, adding staticfiles to Django was a logical move that will help the community. Just my two cents.
On Wed, Oct 20, 2010 at 8:55 PM, Carl Meyer <carl.j.me...@gmail.com> wrote: > Hi Waldemar,
> On Oct 20, 1:40 pm, Waldemar Kornewald <wkornew...@gmail.com> wrote: > [snip] >> However, what staticfiles does has almost nothing to do with "bigger >> project" asset management. Just look at the features grid on >> djangopackages (disclaimer: I'm the author of django-mediagenerator >> and I maintain that grid):http://djangopackages.com/grids/g/asset-managers/ >> It's obvious that django-staticfiles has none of the critical features: >> * It doesn't combine your CSS/JS files. >> * It doesn't compress them. >> * It doesn't support CSS compilers like Sass or Less. >> * It doesn't support versioning of CSS, JS, and images. >> * Because of the lack of versioning the staticfiles view doesn't (and >> can't) support proper caching which means for every single visited >> page the browser will make unnecessary "if-modified-since" roundtrips >> to the very slow runserver. These unnecessary HTTP roundtrips make it >> very painful to work even on medium sized projects (unless combined >> with a local dev http server like Apache). >> * Finally, staticfiles doesn't auto-regenerate the assets when they >> are changed.
> And the "lack" of all of these features is precisely why staticfiles > is well suited to contrib, whereas something like mediagenerator is > not (even though it may be very useful). Staticfiles has a very > specific, well-defined purpose (collecting media files from apps), > which fills a major hole in the Django "story" for reusable apps. It > does not come with a pile of additional features outside that scope, > adding much more code and more areas where people may reasonably > disagree on the best implementation.
In what way does staticfiles make writing reusable apps easier? It merely collects apps' "static" folders. The same thing could be achieved by defining a simple standard: "Put media files in the 'media' folder." and then making sure that all asset managers follow this very simple standard. Then the user is free to choose an asset manager that suits his needs and be happy.
The real problem is this: An app that gets shipped with Django needs to fulfill a need of a lot of people and it should be implemented with best practice in mind. If in the end almost every user will switch to a better solution (like you did with django_compressor) then I think it's valid to ask: Why should a tool that only few people want to use be shipped with Django? Isn't a simple standard for asset managers better?
> Every feature you list here can easily be provided by a separate app > on top of staticfiles. I currently use staticfiles in combination with > django_compressor, which provides all these additional features > (compression, combining, versioning, auto-regenerating when changed), > and the combination works very well. I prefer having separable > concerns in separate apps whenever possible, and the tasks performed > by staticfiles and compressor are very much separable.
That's a funny combination of tools. :) You don't really need django-staticfiles because in your case django_compressor takes care of collecting assets (i.e. it already does staticfiles's job) and keeping the media folder updated. All that django-staticfiles does is serve the files in your media folder. You get the same result with Django's serve() view. Try it. You'll see that django-staticfiles is in fact an unnecessary component in your project.
The separation makes everything more difficult (if not impossible) to implement properly. How do you want to connect the very flexible dynamic serving view of django-mediagenerator with the dynamic serving view of staticfiles? There's nothing to be gained from such a separation, especially since the code for collecting files is minimal, anyway.
>> This makes development extremely painful and error-prone >> because you have to "manage.py collectstatic" every time you make a >> little change to your JS or CSS code (and that's really easy to >> forget).
> In development you generally would not run "manage.py collectstatic" > at all; you use the dynamic-serving view instead so changes are > reflected instantly.
I wish that were the case. The staticfiles documentation says:
""" Remember to run :djadmin:`collectstatic` when your media changes; the view only serves static files that have been collected. """
You see, it's a mess. You just didn't notice the mess because you used django_compressor. This only supports my point that staticfiles fails at solving real end-user needs and thus gets replaced with better solutions, anyway.
On Wed, Oct 20, 2010 at 3:04 PM, Waldemar Kornewald
<wkornew...@gmail.com> wrote: > I wish that were the case. The staticfiles documentation says:
> """ > Remember to run :djadmin:`collectstatic` when your media changes; > the view only serves static files that have been collected. > """
I actually wrote that without checking that it's true, and looking at the code I think it's not. It looks like the view does indeed serve files out of the static locations directly. I'll double-check and remove that line from the docs accordingly.
On Oct 20, 4:04 pm, Waldemar Kornewald <wkornew...@gmail.com> wrote:
> In what way does staticfiles make writing reusable apps easier? It
> merely collects apps' "static" folders. The same thing could be
> achieved by defining a simple standard:
> "Put media files in the 'media' folder."
> and then making sure that all asset managers follow this very simple
> standard. Then the user is free to choose an asset manager that suits
> his needs and be happy.
The standard you are asking for is there now, and it's spelled
"static." And Django also now provides a functional and useful tool to
collect media from that standard location, both dynamically in
development and statically in production. If you want to use a
different tool for this, you're free to do so.
> The real problem is this: An app that gets shipped with Django needs
> to fulfill a need of a lot of people and it should be implemented with
> best practice in mind. If in the end almost every user will switch to
> a better solution (like you did with django_compressor) then I think
I did not switch, I used both together.
> That's a funny combination of tools. :)
> You don't really need django-staticfiles because in your case
> django_compressor takes care of collecting assets (i.e. it already
> does staticfiles's job) and keeping the media folder updated. All that
That is simply not true. django_compressor does not collect media from
reusable app media directories. It expects to find all media in
COMPRESS_ROOT, a single directory. django-staticfiles solves the
separate problem of collecting media from app directories and putting
it in that single location.
> django-staticfiles does is serve the files in your media folder. You
> get the same result with Django's serve() view. Try it. You'll see
> that django-staticfiles is in fact an unnecessary component in your
> project.
No, django-staticfiles serves (in development) or just collects (in
production) the files in my project-level static folder, _and all of
my app static folders_, and implements an overlay mechanism similar to
the app-directories template loader so my project-level static folder
can easily override app-provided static files. This is a simple,
obvious, and useful mechanism: it is most certainly not an
"unnecessary component."
> I wish that were the case. The staticfiles documentation says:
> """
> Remember to run :djadmin:`collectstatic` when your media changes;
> the view only serves static files that have been collected.
> """
The docs are what's wrong here, and should be fixed.
> You see, it's a mess. You just didn't notice the mess because you used
> django_compressor. This only supports my point that staticfiles fails
> at solving real end-user needs and thus gets replaced with better
> solutions, anyway.
This is a completely false characterization of the situation. I
suggest you review again the different and non-overlapping purposes
served by staticfiles and a media-compressor.
On Wed, Oct 20, 2010 at 10:53 PM, Carl Meyer <carl.j.me...@gmail.com> wrote: > Hi Waldemar,
> On Oct 20, 4:04 pm, Waldemar Kornewald <wkornew...@gmail.com> wrote: >> That's a funny combination of tools. :) >> You don't really need django-staticfiles because in your case >> django_compressor takes care of collecting assets (i.e. it already >> does staticfiles's job) and keeping the media folder updated. All that
> That is simply not true. django_compressor does not collect media from > reusable app media directories. It expects to find all media in > COMPRESS_ROOT, a single directory. django-staticfiles solves the > separate problem of collecting media from app directories and putting > it in that single location.
Ah sorry, my assumption was based on the staticfiles documentation which seems to be broken. Also, you're right, the compressor doesn't search app folders. However, quite a few other asset managers do. My primary concern is that with those asset managers you just throw away staticfiles, so unlike django.contrib.sessions which is "totally useful" this app feels more like "maybe useful".
I mean, if the real goal is merely to allow for writing reusable apps then staticfiles is not much better than having an official standard and linking to djangopackages, so users can select a standards-compliant asset manager that suits their needs.
On Wed, Oct 20, 2010 at 10:13 PM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote: > On Wed, Oct 20, 2010 at 3:04 PM, Waldemar Kornewald > <wkornew...@gmail.com> wrote: >> I wish that were the case. The staticfiles documentation says:
>> """ >> Remember to run :djadmin:`collectstatic` when your media changes; >> the view only serves static files that have been collected. >> """
> I actually wrote that without checking that it's true, and looking at > the code I think it's not. It looks like the view does indeed serve > files out of the static locations directly. I'll double-check and > remove that line from the docs accordingly.
On Thu, Oct 21, 2010 at 2:40 AM, Waldemar Kornewald <wkornew...@gmail.com>wrote:
> Today the staticfiles contrib app was committed to trunk. This got me > wondering: Wasn't there recently a discussion about how "contrib" is a > bad idea and all apps should either be in core or live as separate > projects? Is staticfiles really a core app? Even if it is a core app, > does it do its job well enough to be in contrib?
I thought about this too and had a long thread on Twitter with jezdez about staticfiles. It occurred to me that adding more apps to contrib was kind of a bad idea. I know "everyone" uses admin etc. but I was of the understanding that contrib apps are optional apps for django but staticfiles doesn't seem to be that way. The core 'django.views.static.serve' and 'django.core.context_processors.media' are deprecated in favor of the staticfiles equivalents in contrib. Is the idea that the contrib app is a stepping stone to providing core functionality? i.e. It was put in contrib to maintain backwards compatibility for at least enough time to satisfy the deprecation policy? I'm also getting worried in general about adding more and more functionality to Django that doesn't really *need* to be there.
On a seperate topic, it occurred to me that if you wanted to use the same backend for your STATIC_STORAGE as you use for DEFAULT_STORAGE then you'll run into problems if you want to use different options for each. The docs mention django_storages but you'll be hard pressed if you want to store your static and user media in different buckets. It's one of the reasons I added options to the constructor for the s3boto backend (I'm the current maintainer) in django_storages and added a STATIC_STORAGE_KWARGS in my branch of staticfiles on bitbucket. Not sure it was the best solution, but it works.
> I thought about this too and had a long thread on Twitter with jezdez about staticfiles. It occurred to me that adding more apps to contrib was kind of a bad idea. I know "everyone" uses admin etc. but I was of the understanding that contrib apps are optional apps for django but staticfiles doesn't seem to be that way.
Repeating what I said briefly on Twitter, the reason to extend this ability was to give a better answer to "How does Django help me to manage and deploy static files?" In that sense, the contrib staticfiles app is really just an extension of the tools that Django already provides, the media context processor, the static view and the WSGI Middleware for runserver -- plus a few optional helpers like a template tag, management commands and extensible file finders that work similar to template loaders. All of which are not required to run a Django site.
> The core 'django.views.static.serve' and 'django.core.context_processors.media' are deprecated in favor of the staticfiles equivalents in contrib. Is the idea that the contrib app is a stepping stone to providing core functionality? i.e. It was put in contrib to maintain backwards compatibility for at least enough time to satisfy the deprecation policy? I'm also getting worried in general about adding more and more functionality to Django that doesn't really *need* to be there.
Moving pieces of code arund is part of the development process and hasn't really been done just for the sake of it. Personally, I don't share your worry about added functionality, as long as it solves real world problems, which staticfiles certainly did for me, as well as for the hundreds of other users, too.
> On a seperate topic, it occurred to me that if you wanted to use the same backend for your STATIC_STORAGE as you use for DEFAULT_STORAGE then you'll run into problems if you want to use different options for each. The docs mention django_storages but you'll be hard pressed if you want to store your static and user media in different buckets. It's one of the reasons I added options to the constructor for the s3boto backend (I'm the current maintainer) in django_storages and added a STATIC_STORAGE_KWARGS in my branch of staticfiles on bitbucket. Not sure it was the best solution, but it works.
I would like to stress that using the same storage backend for static files and user generated files is considered bad practice, which is also why I haven't applied your changes in the staticfiles app. It was a distinct design decission to have both kinds of files in different locations and to provide the STATICFILES_STORAGE next to the DEFAUL_STORAGE setting for further extension. I can't comment on the backends in django-storages much since I'm not very familiar with the code, but I assume that it's possible to subclass those storage backends to change the options, as needed. An extended example on djangosnippets.org or in the django-storages docs would cetainly be useful to have of course. If you have any suggestions regarding the staticfiles docs with regard to using other storage backends, I'd appreciate your help.
On Thu, Oct 21, 2010 at 12:25 PM, Jannis Leidel <jan...@leidel.info> wrote: > Ian,
> > I thought about this too and had a long thread on Twitter with jezdez > about staticfiles. It occurred to me that adding more apps to contrib was > kind of a bad idea. I know "everyone" uses admin etc. but I was of the > understanding that contrib apps are optional apps for django but staticfiles > doesn't seem to be that way.
> Repeating what I said briefly on Twitter, the reason to extend this ability > was to give a better answer to "How does Django help me to manage and deploy > static files?" In that sense, the contrib staticfiles app is really just an > extension of the tools that Django already provides, the media context > processor, the static view and the WSGI Middleware for runserver -- plus a > few optional helpers like a template tag, management commands and extensible > file finders that work similar to template loaders. All of which are not > required to run a Django site.
> > The core 'django.views.static.serve' and > 'django.core.context_processors.media' are deprecated in favor of the > staticfiles equivalents in contrib. Is the idea that the contrib app is a > stepping stone to providing core functionality? i.e. It was put in contrib > to maintain backwards compatibility for at least enough time to satisfy the > deprecation policy? I'm also getting worried in general about adding more > and more functionality to Django that doesn't really *need* to be there.
> Moving pieces of code arund is part of the development process and hasn't > really been done just for the sake of it. Personally, I don't share your > worry about added functionality, as long as it solves real world problems, > which staticfiles certainly did for me, as well as for the hundreds of other > users, too.
I certainly agree it's useful. I use it for my projects. I just don't think it needs to be in core. Anyone that needed this functionality could simply install and use django-staticfiles. For me, adding everything and the kitchen sink to Django with no clear scope is worrysome. Anyway, this part of the conversation at least isn't productive so it's the last you'll hear from me on it.
> > On a seperate topic, it occurred to me that if you wanted to use the same > backend for your STATIC_STORAGE as you use for DEFAULT_STORAGE then you'll > run into problems if you want to use different options for each. The docs > mention django_storages but you'll be hard pressed if you want to store your > static and user media in different buckets. It's one of the reasons I added > options to the constructor for the s3boto backend (I'm the current > maintainer) in django_storages and added a STATIC_STORAGE_KWARGS in my > branch of staticfiles on bitbucket. Not sure it was the best solution, but > it works.
> I would like to stress that using the same storage backend for static files > and user generated files is considered bad practice, which is also why I > haven't applied your changes in the staticfiles app. It was a distinct > design decission to have both kinds of files in different locations and to > provide the STATICFILES_STORAGE next to the DEFAUL_STORAGE setting for > further extension.
I can imagine situations where you would want to store uploaded data and static files in S3 (but in different buckets or directories) to take advantage of the Cloudfront CDN. That just one example but I'm sure there are others such as storing static media in a normal directory and uploaded media on a shared NFS mount (some people do that). I suppose subclassing the backend simply to provide different options would be doable but didn't seem as nice at I need to support it in code rather than just changing some settings. That said, the StaticFilesStorage seems to do exactly what you described so if it's discouraged for some reason (which I'm still not sure I understand) then I'll respect your API.
> I can't comment on the backends in django-storages much since I'm not very > familiar with the code, but I assume that it's possible to subclass those > storage backends to change the options, as needed. An extended example on > djangosnippets.org or in the django-storages docs would cetainly be useful > to have of course. If you have any suggestions regarding the staticfiles > docs with regard to using other storage backends, I'd appreciate your help.
For the S3BotoBackend at least, you would just need to subclass and override the __init__() method to call super() with the options you need, which would mostly likely be environment specific and set in the settings.py anyway.
I think that staticfiles in contrib is a good idea because with it
authors of reusable apps have an answer for the "How should user
install app's static files?" question. They already know how to make
python code and django templates available, but static files was a bit
of pain before. Now the puzzle is assembled. So I think staticfiles is
a better fit for contrib than most of existing contrib apps. It
doesn't provide much functionality but it is a foundation that
reusable apps can rely on.
On 21 ÏËÔ, 12:23, Ian Lewis <ianmle...@gmail.com> wrote:
> On Thu, Oct 21, 2010 at 12:25 PM, Jannis Leidel <jan...@leidel.info> wrote:
> > Ian,
> > > I thought about this too and had a long thread on Twitter with jezdez
> > about staticfiles. It occurred to me that adding more apps to contrib was
> > kind of a bad idea. I know "everyone" uses admin etc. but I was of the
> > understanding that contrib apps are optional apps for django but staticfiles
> > doesn't seem to be that way.
> > Repeating what I said briefly on Twitter, the reason to extend this ability
> > was to give a better answer to "How does Django help me to manage and deploy
> > static files?" In that sense, the contrib staticfiles app is really just an
> > extension of the tools that Django already provides, the media context
> > processor, the static view and the WSGI Middleware for runserver -- plus a
> > few optional helpers like a template tag, management commands and extensible
> > file finders that work similar to template loaders. All of which are not
> > required to run a Django site.
> > > The core 'django.views.static.serve' and
> > 'django.core.context_processors.media' are deprecated in favor of the
> > staticfiles equivalents in contrib. Is the idea that the contrib app is a
> > stepping stone to providing core functionality? i.e. It was put in contrib
> > to maintain backwards compatibility for at least enough time to satisfy the
> > deprecation policy? I'm also getting worried in general about adding more
> > and more functionality to Django that doesn't really *need* to be there.
> > Moving pieces of code arund is part of the development process and hasn't
> > really been done just for the sake of it. Personally, I don't share your
> > worry about added functionality, as long as it solves real world problems,
> > which staticfiles certainly did for me, as well as for the hundreds of other
> > users, too.
> I certainly agree it's useful. I use it for my projects. I just don't think
> it needs to be in core. Anyone that needed this functionality could simply
> install and use django-staticfiles. For me, adding everything and the
> kitchen sink to Django with no clear scope is worrysome. Anyway, this part
> of the conversation at least isn't productive so it's the last you'll hear
> from me on it.
> > > On a seperate topic, it occurred to me that if you wanted to use the same
> > backend for your STATIC_STORAGE as you use for DEFAULT_STORAGE then you'll
> > run into problems if you want to use different options for each. The docs
> > mention django_storages but you'll be hard pressed if you want to store your
> > static and user media in different buckets. It's one of the reasons I added
> > options to the constructor for the s3boto backend (I'm the current
> > maintainer) in django_storages and added a STATIC_STORAGE_KWARGS in my
> > branch of staticfiles on bitbucket. šNot sure it was the best solution, but
> > it works.
> > I would like to stress that using the same storage backend for static files
> > and user generated files is considered bad practice, which is also why I
> > haven't applied your changes in the staticfiles app. It was a distinct
> > design decission to have both kinds of files in different locations and to
> > provide the STATICFILES_STORAGE next to the DEFAUL_STORAGE setting for
> > further extension.
> I can imagine situations where you would want to store uploaded data and
> static files in S3 (but in different buckets or directories) to take
> advantage of the Cloudfront CDN. That just one example but I'm sure there
> are others such as storing static media in a normal directory and uploaded
> media on a shared NFS mount (some people do that). I suppose subclassing the
> backend simply to provide different options would be doable but didn't seem
> as nice at I need to support it in code rather than just changing some
> settings. That said, the StaticFilesStorage seems to do exactly what you
> described so if it's discouraged for some reason (which I'm still not sure I
> understand) then I'll respect your API.
> > I can't comment on the backends in django-storages much since I'm not very
> > familiar with the code, but I assume that it's possible to subclass those
> > storage backends to change the options, as needed. An extended example on
> > djangosnippets.org or in the django-storages docs would cetainly be useful
> > to have of course. If you have any suggestions regarding the staticfiles
> > docs with regard to using other storage backends, I'd appreciate your help.
> For the S3BotoBackend at least, you would just need to subclass and override
> the __init__() method to call super() with the options you need, which would
> mostly likely be environment specific and set in the settings.py anyway.
On Thu, Oct 21, 2010 at 5:25 AM, Jannis Leidel <jan...@leidel.info> wrote: >> The core 'django.views.static.serve' and 'django.core.context_processors.media' are deprecated in favor of the staticfiles equivalents in contrib. Is the idea that the contrib app is a stepping stone to providing core functionality? i.e. It was put in contrib to maintain backwards compatibility for at least enough time to satisfy the deprecation policy? I'm also getting worried in general about adding more and more functionality to Django that doesn't really *need* to be there.
> Moving pieces of code arund is part of the development process and hasn't really been done just for the sake of it. Personally, I don't share your worry about added functionality, as long as it solves real world problems, which staticfiles certainly did for me, as well as for the hundreds of other users, too.
With this reasoning we could as well add django-debug-toolbar, South, django-registration and many other popular apps. What makes staticfiles different? Seriously, I don't see it.
It's not even future-proof. We're heading towards larger client-side web apps which means there will be HTML5 offline manifests and apps consisting of more than 50 files. Even the combination of django_compressor with staticfiles can't handle such apps. In a few years staticfiles will be like the other "badteries" in contrib.
Well, looks like the Django core team doesn't care. :(
On Thu, Oct 21, 2010 at 3:50 AM, Waldemar Kornewald
<wkornew...@gmail.com> wrote: > It's not even future-proof. We're heading towards larger client-side > web apps which means there will be HTML5 offline manifests and apps > consisting of more than 50 files. Even the combination of > django_compressor with staticfiles can't handle such apps. In a few > years staticfiles will be like the other "badteries" in contrib.
> Well, looks like the Django core team doesn't care. :(
Multiple committers have replied to you, explaining their points of view on this. You disagree with their reasoning; that's unfortunate, but it's also the nature of community-driven development. We work by rough consensus, not by unanimous consent. None of which implies that the people who've taken the time to present their arguments to you "don't care" -- the fact that they're doing this shows that they do care and are paying plenty of attention to criticisms.
So in the future, lay off the incendiary/insulting commentary, OK?
As for the main topic of the thread: personally, I think staticfiles is fine in contrib. "How do I gather up the static files from all these apps into one place where I can manage their deployment" is a genuine problem people face when using Django. This app solves that problem. It may not always be a perfect solution for everyone, but neither are most of the apps in contrib; stuff like sessions, auth, the admin, etc. all exist to solve common problems and provide a solid baseline of supplementary functionality to go with the core framework, and they all do a good job at that. But none of them are or ever can be one-size-fits-all solutions which will work for every developer in every situation, and it's unreasonable to expect them to suit every use case. Such is life.
In the future, it may well turn out that people will commonly need much more than what the current iteration of staticfiles provides. But we can't cross that bridge until we come to it; for now, staticfiles solves the common version of the problem (let's face it: *very* few people are doing CDNs or offline HTML5 apps or any of the other stuff brought up in this thread, as compared to gathering a bunch of files and serving them up).
-- "Bureaucrat Conrad, you are technically correct -- the best kind of correct."
On Thu, Oct 21, 2010 at 5:27 AM, James Bennett <ubernost...@gmail.com>wrote:
> In the future, it may well turn out that people will commonly need > much more than what the current iteration of staticfiles provides. But > we can't cross that bridge until we come to it; for now, staticfiles > solves the common version of the problem (let's face it: *very* few > people are doing CDNs or offline HTML5 apps or any of the other stuff > brought up in this thread, as compared to gathering a bunch of files > and serving them up).
For the sake of posterity (including me since I seem to be arriving rather late to the game), is there another mailing list thread or wiki page somewhere that follows or summarizes the decisions that went into django.contrib.staticfiles? I think pointing to the fact that such a process has already happened would also resolve a lot of the concerns in this thread.
On 21 October 2010 13:41, Tobias McNulty <tob...@caktusgroup.com> wrote:
> is there another mailing list thread or wiki page somewhere that follows > or summarizes the decisions that went into django.contrib.staticfiles? I > think pointing to the fact that such a process has already happened would > also resolve a lot of the concerns in this thread.
On Thu, Oct 21, 2010 at 8:54 AM, Dougal Matthews <douga...@gmail.com> wrote: > On 21 October 2010 13:41, Tobias McNulty <tob...@caktusgroup.com> wrote:
>> is there another mailing list thread or wiki page somewhere that follows >> or summarizes the decisions that went into django.contrib.staticfiles? I >> think pointing to the fact that such a process has already happened would >> also resolve a lot of the concerns in this thread.
> I'm not sure if its documented or summarised anywhere.
That thread's pretty old and doesn't really end on anything conclusive other than "work has started". I do see that the patch was updated numerous times on the ticket [1] this month, but was there an associated review on the mailing list? Searching the group for "staticfiles" [2] or "12323" [3] doesn't turn up much, but maybe I'm missing something.
On Thu, Oct 21, 2010 at 8:19 AM, Tobias McNulty <tob...@caktusgroup.com> wrote: > That thread's pretty old and doesn't really end on anything conclusive other > than "work has started". I do see that the patch was updated numerous times > on the ticket [1] this month, but was there an associated review on the > mailing list? Searching the group for "staticfiles" [2] or "12323" [3] > doesn't turn up much, but maybe I'm missing something.
It looks like most of the review and discussion took place on the ticket tracker and on IRC. I'll take a mea culpa for that: I should have brought things back here.
I'd like to try again to actually move this forward: does anyone have an alternate proposal? I'm hearing a bit of complaining, but no alternate proposals.
On Thu, Oct 21, 2010 at 3:50 AM, Waldemar Kornewald
<wkornew...@gmail.com> wrote: > With this reasoning we could as well add django-debug-toolbar, South, > django-registration and many other popular apps. What makes > staticfiles different? Seriously, I don't see it.
Jannis proposed that we add static files in. Brian concurred. So did I. No core developers objected.
You're completely right about DDT, South, and -registration. What makes them different?
* Nobody's asked that DDT be included in Django. Last I spoke to Rob about it, he felt it needed a bit more time as a third-party app to stabilize the API and get more robust. I'd happily support moving DDT into contrib if there's sufficient interest.
* We've (at least, Russ and I have) talked a lot with Andrew about how South might fit into contrib. The consensus was that there's still a good deal of exploring to be done in the schema evolution space -- there's at least a couple-three tools that could easily be considered "good enough". However, we agreed that we should push some of the parts of schema migration down into Django, particularly DDL generation, migration discovery, and perhaps migration tracking. Andrew was offered a commit bit partially to help make this work happen more smoothly.
* Like DDT, nobody's proposed that -registration be included in Django. I'm not sure that James would be interested, either: he's got a different opinion on contrib versus external apps than I do. Like DDT, I'd go along if there was sufficient interest and if James was willing.
> On Thu, Oct 21, 2010 at 3:50 AM, Waldemar Kornewald > <wkornew...@gmail.com> wrote: >> With this reasoning we could as well add django-debug-toolbar, South, >> django-registration and many other popular apps. What makes >> staticfiles different? Seriously, I don't see it.
> Jannis proposed that we add static files in. Brian concurred. So did > I. No core developers objected.
> You're completely right about DDT, South, and -registration. What > makes them different?
> * Nobody's asked that DDT be included in Django. Last I spoke to Rob > about it, he felt it needed a bit more time as a third-party app to > stabilize the API and get more robust. I'd happily support moving DDT > into contrib if there's sufficient interest.
> * We've (at least, Russ and I have) talked a lot with Andrew about how > South might fit into contrib. The consensus was that there's still a > good deal of exploring to be done in the schema evolution space -- > there's at least a couple-three tools that could easily be considered > "good enough". However, we agreed that we should push some of the > parts of schema migration down into Django, particularly DDL > generation, migration discovery, and perhaps migration tracking. > Andrew was offered a commit bit partially to help make this work > happen more smoothly.
> * Like DDT, nobody's proposed that -registration be included in > Django. I'm not sure that James would be interested, either: he's got > a different opinion on contrib versus external apps than I do. Like > DDT, I'd go along if there was sufficient interest and if James was > willing.
> Jacob
I think South is the best candidate here, because it could improve the third party app ecosystem to have a stable and consistent migration framework battery included. DDT and django-registration both stand alone pretty well, so I see less benefit (though I don't have any particular objections, either).
Under that rubric, I agree that staticfiles is a good candidate, as well.
On Thu, Oct 21, 2010 at 11:01 AM, Jacob Kaplan-Moss <ja...@jacobian.org>wrote:
> On Thu, Oct 21, 2010 at 8:19 AM, Tobias McNulty <tob...@caktusgroup.com> > wrote: > > That thread's pretty old and doesn't really end on anything conclusive > other > > than "work has started". I do see that the patch was updated numerous > times > > on the ticket [1] this month, but was there an associated review on the > > mailing list? Searching the group for "staticfiles" [2] or "12323" [3] > > doesn't turn up much, but maybe I'm missing something.
> It looks like most of the review and discussion took place on the > ticket tracker and on IRC. I'll take a mea culpa for that: I should > have brought things back here.
No worries. I certainly haven't been as involved as I could have been, so it's quite likely I'd know more about what's going on if I'd been paying more attention. :-)
I'd like to try again to actually move this forward: does anyone have
> an alternate proposal? I'm hearing a bit of complaining, but no > alternate proposals.
I think the issue is that the commit has already been made, and that doesn't feel like the right time to anyone to submit an alternate proposal.
That said, I'm not complaining, and I think having a standard way to bundle static media with reusable apps is a Really Good Thing (tm).
As far as constructive feedback goes, something I would like to see is making "compilestatic" more configurable: One might prefer to serve media directly from the app static directories (via Aliases in apache or locations in nginx), instead of copying it manually every time during a deployment. Some hooks that would let you generate a customized configuration, instead of making copying the de facto standard implementation, would be great IMHO.
Another concern I have is about naming conflicts: how does staticfiles address the case where multiple apps supply files with the same names? This is something that might also be addressed by being more configurable: I can see the case where it'd make more sense to expose app-specific URLs for media to the browser, instead of consolidating everything under a single global namespace. This could be implemented by a template tag, like so:
On Thu, Oct 21, 2010 at 10:44 AM, Tobias McNulty <tob...@caktusgroup.com> wrote: > I think the issue is that the commit has already been made, and that doesn't > feel like the right time to anyone to submit an alternate proposal.
Well, that's why we use revision control: if there's a rough consensus that a commit was a mistake, we can revert it. I'm hearing a few voices in this thread that I *think* would like to see the change backed out -- though nobody's actually suggested that explicit yet, so I'm not sure. OTOH, I'm hearing a lot of "whoohoos" from users, so, yes, we couldn't just pull the rug out: we'd need something better to replace it with.
> As far as constructive feedback goes, something I would like to see is > making "compilestatic" more configurable:
Yes, indeed -- my main concern pre-commit was configurability, and the main focus of my code review was in making sure that we didn't do anything that precluded adding better more powerful features later. I didn't, however, want to hold off adding the feature waiting for some ill-defined set of improvements. I wanted to see a minimalistic addition that we can grow on later. That's how we roll.
I think realistically "later" likely means 1.4, but I'd happily review patches adding more hooks at every time.
I really don't want staticfiles to be the tool that tries to do everything, though. The idea is to provide the lowest common set of functionality needed to handle media in reusable apps successfully, and then to let other more complicated features (compression, etc.) build on top of that. So yes: more hooks!
> One might prefer to serve media > directly from the app static directories (via Aliases in apache or locations > in nginx), instead of copying it manually every time during a deployment.
Hm, I certainly wouldn't... but it would be nice to make something like that possible, yes. BTW, you can right now run `collectstatic --links` to use symlinks instead of actually copying the files. Not quite the same thing, but close right now.
> Another concern I have is about naming conflicts: how does staticfiles > address the case where multiple apps supply files with the same names?
> This > is something that might also be addressed by being more configurable: I can > see the case where it'd make more sense to expose app-specific URLs for > media to the browser, instead of consolidating everything under a single > global namespace. This could be implemented by a template tag, like so: > {% staticfile 'appname' 'path/to/file.png' %}
I'm not quite sure I get the use case here, but like I said above: I would be thrilled to see patches making this more configurable now.
It seems like everyone's talking about two different things here:
1) Should Django add more apps to contrib?
2) Does staticfiles solve a real need that users of Django have
today? Is it the de-facto implementation of a common pattern?
Let's just drop the discussion of the first point for the purposes of
this thread. At this point it's clear that contrib isn't going
anywhere soon, and certainly not without some concrete proposal by
someone and a lot of discussion.
So that leaves us with staticfiles, specifically. I've heard
countless times from people at conferences, at local meetup groups,
and elsewhere who are upset that Django doesn't give them any
direction on what to do with their static files. Django is fairly
opinionated software, but as soon as you start dealing with static
files Django suddenly has no opinion. That's confusing and
frustrating for users, to say the least.
I think staticfiles is an *excellent* contribution, and if there is
some advanced use case that it doesn't do, then let's do our best to
make it extensible enough to make that use case possible. Easy things
(like bundling media with your app) should be easy, but hard things
(like minification and consolidation of js) should be possible.
On Thu, Oct 21, 2010 at 5:10 PM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote: > On Thu, Oct 21, 2010 at 3:50 AM, Waldemar Kornewald > <wkornew...@gmail.com> wrote: >> With this reasoning we could as well add django-debug-toolbar, South, >> django-registration and many other popular apps. What makes >> staticfiles different? Seriously, I don't see it.
> Jannis proposed that we add static files in. Brian concurred. So did > I. No core developers objected.
> You're completely right about DDT, South, and -registration. What > makes them different?
> * Nobody's asked that DDT be included in Django. Last I spoke to Rob > about it, he felt it needed a bit more time as a third-party app to > stabilize the API and get more robust. I'd happily support moving DDT > into contrib if there's sufficient interest.
> * We've (at least, Russ and I have) talked a lot with Andrew about how > South might fit into contrib. The consensus was that there's still a > good deal of exploring to be done in the schema evolution space -- > there's at least a couple-three tools that could easily be considered > "good enough". However, we agreed that we should push some of the > parts of schema migration down into Django, particularly DDL > generation, migration discovery, and perhaps migration tracking. > Andrew was offered a commit bit partially to help make this work > happen more smoothly.
> * Like DDT, nobody's proposed that -registration be included in > Django. I'm not sure that James would be interested, either: he's got > a different opinion on contrib versus external apps than I do. Like > DDT, I'd go along if there was sufficient interest and if James was > willing.
Thanks a lot for the clarification. So, then the "bad batteries" part in Eric's talk "Why Django sucks and how we can fix it" doesn't receive much agreement within the Django core team?
My proposal would've been to not add staticfiles in the first place, but it seems to be too late, now. From what I can see, only the finder/storage API looks fully reusable. More advanced asset managers will use custom templatetags, manage.py commands, and sometimes even their own view in order to support CSS compilers and other advanced features. So, is staticfiles' goal to only provide a reusable finder API or are 3rd-party asset managers supposed to reuse anything else?
> On Thu, Oct 21, 2010 at 10:44 AM, Tobias McNulty <tob...@caktusgroup.com> > wrote: > > I think the issue is that the commit has already been made, and that > doesn't > > feel like the right time to anyone to submit an alternate proposal.
> Well, that's why we use revision control: if there's a rough consensus > that a commit was a mistake, we can revert it. I'm hearing a few > voices in this thread that I *think* would like to see the change > backed out -- though nobody's actually suggested that explicit yet, so > I'm not sure. OTOH, I'm hearing a lot of "whoohoos" from users, so, > yes, we couldn't just pull the rug out: we'd need something better to > replace it with.
> > As far as constructive feedback goes, something I would like to see is > > making "compilestatic" more configurable:
> Yes, indeed -- my main concern pre-commit was configurability, and the > main focus of my code review was in making sure that we didn't do > anything that precluded adding better more powerful features later. I > didn't, however, want to hold off adding the feature waiting for some > ill-defined set of improvements. I wanted to see a minimalistic > addition that we can grow on later. That's how we roll.
> I think realistically "later" likely means 1.4, but I'd happily review > patches adding more hooks at every time.
> I really don't want staticfiles to be the tool that tries to do > everything, though. The idea is to provide the lowest common set of > functionality needed to handle media in reusable apps successfully, > and then to let other more complicated features (compression, etc.) > build on top of that. So yes: more hooks!
> > One might prefer to serve media > > directly from the app static directories (via Aliases in apache or > locations > > in nginx), instead of copying it manually every time during a deployment.
> Hm, I certainly wouldn't... but it would be nice to make something > like that possible, yes. BTW, you can right now run `collectstatic > --links` to use symlinks instead of actually copying the files. Not > quite the same thing, but close right now.
What's the argument against serving the original files directly? Symlinks are pretty horrible; I'd raise a bigger fit if that was the recommended approach. ;-)
> > Another concern I have is about naming conflicts: how does staticfiles > > address the case where multiple apps supply files with the same names?
Ah, so realistically we should put all our media in 'static/<appname>', like for templates, if we want to avoid conflicts with other apps. Would that be worth mentioning as a convention in the docs so we don't end up with a bunch of reusable apps that aren't at all reusable?
> This > > is something that might also be addressed by being more configurable: I > can > > see the case where it'd make more sense to expose app-specific URLs for > > media to the browser, instead of consolidating everything under a single > > global namespace. This could be implemented by a template tag, like so: > > {% staticfile 'appname' 'path/to/file.png' %}
> I'm not quite sure I get the use case here, but like I said above: I > would be thrilled to see patches making this more configurable now.
Adding something to the docs about avoiding name conflicts would make my concerns about such conflicts (and the template tag suggested above) obsolete.
