contrib.staticfiles app concerns
Waldemar Kornewald
first of all, sorry for chiming in so late. It seems like this has
been discussed in May, already, but I didn't follow the discussion
back then.
Today the staticfiles contrib app was committed to trunk. This got me
wondering: Wasn't there recently a discussion about how "contrib" is a
bad idea and all apps should either be in core or live as separate
projects? Is staticfiles really a core app? Even if it is a core app,
does it do its job well enough to be in contrib?
I do agree that such an app is critical for practically all web
projects and the documentation even says:
"""
For small projects, this [static files management] isn't a big deal,
because you can just keep the media
somewhere your web server can find it. However, in bigger projects -- especially
those comprised of multiple apps -- dealing with the multiple sets of static
files provided by each application starts to get tricky.
"""
So like the docs say, staticfiles seems to target "bigger projects".
However, what staticfiles does has almost nothing to do with "bigger
project" asset management. Just look at the features grid on
djangopackages (disclaimer: I'm the author of django-mediagenerator
and I maintain that grid):
http://djangopackages.com/grids/g/asset-managers/
It's obvious that django-staticfiles has none of the critical features:
* It doesn't combine your CSS/JS files.
* It doesn't compress them.
* It doesn't support CSS compilers like Sass or Less.
* It doesn't support versioning of CSS, JS, and images.
* Because of the lack of versioning the staticfiles view doesn't (and
can't) support proper caching which means for every single visited
page the browser will make unnecessary "if-modified-since" roundtrips
to the very slow runserver. These unnecessary HTTP roundtrips make it
very painful to work even on medium sized projects (unless combined
with a local dev http server like Apache).
* Finally, staticfiles doesn't auto-regenerate the assets when they
are changed. This makes development extremely painful and error-prone
because you have to "manage.py collectstatic" every time you make a
little change to your JS or CSS code (and that's really easy to
forget).
From that point of view, staticfiles is only useful for small hobby
projects. So, why was this app added to Django? Most projects are
better off using a different solution, anyway.
BTW, I noticed a bug in the staticfiles view: It checks for "if
settings.DEBUG", but that should be "if not settings.DEBUG".
Also, staticfiles doesn't index "media" folders although the admin
uses "media" instead of "static". Why not follow the existing scheme?
Bye,
Waldemar Kornewald
--
Django on App Engine, MongoDB, ...? Browser-side Python? It's open-source:
http://www.allbuttonspressed.com/blog/django
Carl Meyer
On Oct 20, 1:40 pm, Waldemar Kornewald <wkornew...@gmail.com> wrote:
[snip]
> project" asset management. Just look at the features grid on
> djangopackages (disclaimer: I'm the author of django-mediagenerator
> and I maintain that grid):http://djangopackages.com/grids/g/asset-managers/
> It's obvious that django-staticfiles has none of the critical features:
> * It doesn't combine your CSS/JS files.
> * It doesn't compress them.
> * It doesn't support CSS compilers like Sass or Less.
> * It doesn't support versioning of CSS, JS, and images.
> * Because of the lack of versioning the staticfiles view doesn't (and
> can't) support proper caching which means for every single visited
> page the browser will make unnecessary "if-modified-since" roundtrips
> to the very slow runserver. These unnecessary HTTP roundtrips make it
> very painful to work even on medium sized projects (unless combined
> with a local dev http server like Apache).
> * Finally, staticfiles doesn't auto-regenerate the assets when they
> are changed.
is well suited to contrib, whereas something like mediagenerator is
not (even though it may be very useful). Staticfiles has a very
specific, well-defined purpose (collecting media files from apps),
which fills a major hole in the Django "story" for reusable apps. It
does not come with a pile of additional features outside that scope,
adding much more code and more areas where people may reasonably
disagree on the best implementation.
Every feature you list here can easily be provided by a separate app
on top of staticfiles. I currently use staticfiles in combination with
django_compressor, which provides all these additional features
(compression, combining, versioning, auto-regenerating when changed),
and the combination works very well. I prefer having separable
concerns in separate apps whenever possible, and the tasks performed
by staticfiles and compressor are very much separable.
> This makes development extremely painful and error-prone
> because you have to "manage.py collectstatic" every time you make a
> little change to your JS or CSS code (and that's really easy to
> forget).
at all; you use the dynamic-serving view instead so changes are
reflected instantly.
Carl
Sean Bleier
Staticfiles has a very specific, well-defined purpose (collecting media files from apps), which fills a major hole in the Django "story" for reusable apps.
Waldemar Kornewald
In what way does staticfiles make writing reusable apps easier? It
merely collects apps' "static" folders. The same thing could be
achieved by defining a simple standard:
"Put media files in the 'media' folder."
and then making sure that all asset managers follow this very simple
standard. Then the user is free to choose an asset manager that suits
his needs and be happy.
The real problem is this: An app that gets shipped with Django needs
to fulfill a need of a lot of people and it should be implemented with
best practice in mind. If in the end almost every user will switch to
a better solution (like you did with django_compressor) then I think
it's valid to ask: Why should a tool that only few people want to use
be shipped with Django? Isn't a simple standard for asset managers
better?
> Every feature you list here can easily be provided by a separate app
> on top of staticfiles. I currently use staticfiles in combination with
> django_compressor, which provides all these additional features
> (compression, combining, versioning, auto-regenerating when changed),
> and the combination works very well. I prefer having separable
> concerns in separate apps whenever possible, and the tasks performed
> by staticfiles and compressor are very much separable.
That's a funny combination of tools. :)
You don't really need django-staticfiles because in your case
django_compressor takes care of collecting assets (i.e. it already
does staticfiles's job) and keeping the media folder updated. All that
django-staticfiles does is serve the files in your media folder. You
get the same result with Django's serve() view. Try it. You'll see
that django-staticfiles is in fact an unnecessary component in your
project.
The separation makes everything more difficult (if not impossible) to
implement properly. How do you want to connect the very flexible
dynamic serving view of django-mediagenerator with the dynamic serving
view of staticfiles? There's nothing to be gained from such a
separation, especially since the code for collecting files is minimal,
anyway.
>> This makes development extremely painful and error-prone
>> because you have to "manage.py collectstatic" every time you make a
>> little change to your JS or CSS code (and that's really easy to
>> forget).
>
> In development you generally would not run "manage.py collectstatic"
> at all; you use the dynamic-serving view instead so changes are
> reflected instantly.
I wish that were the case. The staticfiles documentation says:
"""
Remember to run :djadmin:`collectstatic` when your media changes;
the view only serves static files that have been collected.
"""
You see, it's a mess. You just didn't notice the mess because you used
django_compressor. This only supports my point that staticfiles fails
at solving real end-user needs and thus gets replaced with better
solutions, anyway.
Jacob Kaplan-Moss
<wkorn...@gmail.com> wrote:
> I wish that were the case. The staticfiles documentation says:
>
> """
> Remember to run :djadmin:`collectstatic` when your media changes;
> the view only serves static files that have been collected.
> """
I actually wrote that without checking that it's true, and looking at
the code I think it's not. It looks like the view does indeed serve
files out of the static locations directly. I'll double-check and
remove that line from the docs accordingly.
> You see, it's a mess.
Do you have an alternate proposal?
Jacob
Carl Meyer
On Oct 20, 4:04 pm, Waldemar Kornewald <wkornew...@gmail.com> wrote:
> In what way does staticfiles make writing reusable apps easier? It
> merely collects apps' "static" folders. The same thing could be
> achieved by defining a simple standard:
> "Put media files in the 'media' folder."
> and then making sure that all asset managers follow this very simple
> standard. Then the user is free to choose an asset manager that suits
> his needs and be happy.
"static." And Django also now provides a functional and useful tool to
collect media from that standard location, both dynamically in
development and statically in production. If you want to use a
different tool for this, you're free to do so.
> The real problem is this: An app that gets shipped with Django needs
> to fulfill a need of a lot of people and it should be implemented with
> best practice in mind. If in the end almost every user will switch to
> a better solution (like you did with django_compressor) then I think
> That's a funny combination of tools. :)
> You don't really need django-staticfiles because in your case
> django_compressor takes care of collecting assets (i.e. it already
> does staticfiles's job) and keeping the media folder updated. All that
reusable app media directories. It expects to find all media in
COMPRESS_ROOT, a single directory. django-staticfiles solves the
separate problem of collecting media from app directories and putting
it in that single location.
> django-staticfiles does is serve the files in your media folder. You
> get the same result with Django's serve() view. Try it. You'll see
> that django-staticfiles is in fact an unnecessary component in your
> project.
production) the files in my project-level static folder, _and all of
my app static folders_, and implements an overlay mechanism similar to
the app-directories template loader so my project-level static folder
can easily override app-provided static files. This is a simple,
obvious, and useful mechanism: it is most certainly not an
"unnecessary component."
> I wish that were the case. The staticfiles documentation says:
>
> """
> Remember to run :djadmin:`collectstatic` when your media changes;
> the view only serves static files that have been collected.
> """
> You see, it's a mess. You just didn't notice the mess because you used
> django_compressor. This only supports my point that staticfiles fails
> at solving real end-user needs and thus gets replaced with better
> solutions, anyway.
suggest you review again the different and non-overlapping purposes
served by staticfiles and a media-compressor.
Carl
Waldemar Kornewald
On Wed, Oct 20, 2010 at 10:53 PM, Carl Meyer <carl.j...@gmail.com> wrote:
> Hi Waldemar,
>
> On Oct 20, 4:04 pm, Waldemar Kornewald <wkornew...@gmail.com> wrote:
>> That's a funny combination of tools. :)
>> You don't really need django-staticfiles because in your case
>> django_compressor takes care of collecting assets (i.e. it already
>> does staticfiles's job) and keeping the media folder updated. All that
>
> That is simply not true. django_compressor does not collect media from
> reusable app media directories. It expects to find all media in
> COMPRESS_ROOT, a single directory. django-staticfiles solves the
> separate problem of collecting media from app directories and putting
> it in that single location.
Ah sorry, my assumption was based on the staticfiles documentation
which seems to be broken. Also, you're right, the compressor doesn't
search app folders. However, quite a few other asset managers do. My
primary concern is that with those asset managers you just throw away
staticfiles, so unlike django.contrib.sessions which is "totally
useful" this app feels more like "maybe useful".
I mean, if the real goal is merely to allow for writing reusable apps
then staticfiles is not much better than having an official standard
and linking to djangopackages, so users can select a
standards-compliant asset manager that suits their needs.
Waldemar Kornewald
> On Wed, Oct 20, 2010 at 3:04 PM, Waldemar Kornewald
> <wkorn...@gmail.com> wrote:
>> I wish that were the case. The staticfiles documentation says:
>>
>> """
>> Remember to run :djadmin:`collectstatic` when your media changes;
>> the view only serves static files that have been collected.
>> """
>
> I actually wrote that without checking that it's true, and looking at
> the code I think it's not. It looks like the view does indeed serve
> files out of the static locations directly. I'll double-check and
> remove that line from the docs accordingly.
Indeed, the documentation is broken.
Bye,
Waldemar
Ian Lewis
Today the staticfiles contrib app was committed to trunk. This got me
wondering: Wasn't there recently a discussion about how "contrib" is a
bad idea and all apps should either be in core or live as separate
projects? Is staticfiles really a core app? Even if it is a core app,
does it do its job well enough to be in contrib?
Jannis Leidel
> I thought about this too and had a long thread on Twitter with jezdez about staticfiles. It occurred to me that adding more apps to contrib was kind of a bad idea. I know "everyone" uses admin etc. but I was of the understanding that contrib apps are optional apps for django but staticfiles doesn't seem to be that way.
Repeating what I said briefly on Twitter, the reason to extend this ability was to give a better answer to "How does Django help me to manage and deploy static files?" In that sense, the contrib staticfiles app is really just an extension of the tools that Django already provides, the media context processor, the static view and the WSGI Middleware for runserver -- plus a few optional helpers like a template tag, management commands and extensible file finders that work similar to template loaders. All of which are not required to run a Django site.
> The core 'django.views.static.serve' and 'django.core.context_processors.media' are deprecated in favor of the staticfiles equivalents in contrib. Is the idea that the contrib app is a stepping stone to providing core functionality? i.e. It was put in contrib to maintain backwards compatibility for at least enough time to satisfy the deprecation policy? I'm also getting worried in general about adding more and more functionality to Django that doesn't really *need* to be there.
Moving pieces of code arund is part of the development process and hasn't really been done just for the sake of it. Personally, I don't share your worry about added functionality, as long as it solves real world problems, which staticfiles certainly did for me, as well as for the hundreds of other users, too.
> On a seperate topic, it occurred to me that if you wanted to use the same backend for your STATIC_STORAGE as you use for DEFAULT_STORAGE then you'll run into problems if you want to use different options for each. The docs mention django_storages but you'll be hard pressed if you want to store your static and user media in different buckets. It's one of the reasons I added options to the constructor for the s3boto backend (I'm the current maintainer) in django_storages and added a STATIC_STORAGE_KWARGS in my branch of staticfiles on bitbucket. Not sure it was the best solution, but it works.
I would like to stress that using the same storage backend for static files and user generated files is considered bad practice, which is also why I haven't applied your changes in the staticfiles app. It was a distinct design decission to have both kinds of files in different locations and to provide the STATICFILES_STORAGE next to the DEFAUL_STORAGE setting for further extension. I can't comment on the backends in django-storages much since I'm not very familiar with the code, but I assume that it's possible to subclass those storage backends to change the options, as needed. An extended example on djangosnippets.org or in the django-storages docs would cetainly be useful to have of course. If you have any suggestions regarding the staticfiles docs with regard to using other storage backends, I'd appreciate your help.
Best,
Jannis
Ian Lewis
Ian,
Repeating what I said briefly on Twitter, the reason to extend this ability was to give a better answer to "How does Django help me to manage and deploy static files?" In that sense, the contrib staticfiles app is really just an extension of the tools that Django already provides, the media context processor, the static view and the WSGI Middleware for runserver -- plus a few optional helpers like a template tag, management commands and extensible file finders that work similar to template loaders. All of which are not required to run a Django site.
> I thought about this too and had a long thread on Twitter with jezdez about staticfiles. It occurred to me that adding more apps to contrib was kind of a bad idea. I know "everyone" uses admin etc. but I was of the understanding that contrib apps are optional apps for django but staticfiles doesn't seem to be that way.
Moving pieces of code arund is part of the development process and hasn't really been done just for the sake of it. Personally, I don't share your worry about added functionality, as long as it solves real world problems, which staticfiles certainly did for me, as well as for the hundreds of other users, too.
> The core 'django.views.static.serve' and 'django.core.context_processors.media' are deprecated in favor of the staticfiles equivalents in contrib. Is the idea that the contrib app is a stepping stone to providing core functionality? i.e. It was put in contrib to maintain backwards compatibility for at least enough time to satisfy the deprecation policy? I'm also getting worried in general about adding more and more functionality to Django that doesn't really *need* to be there.
> On a seperate topic, it occurred to me that if you wanted to use the same backend for your STATIC_STORAGE as you use for DEFAULT_STORAGE then you'll run into problems if you want to use different options for each. The docs mention django_storages but you'll be hard pressed if you want to store your static and user media in different buckets. It's one of the reasons I added options to the constructor for the s3boto backend (I'm the current maintainer) in django_storages and added a STATIC_STORAGE_KWARGS in my branch of staticfiles on bitbucket. Not sure it was the best solution, but it works.I would like to stress that using the same storage backend for static files and user generated files is considered bad practice, which is also why I haven't applied your changes in the staticfiles app. It was a distinct design decission to have both kinds of files in different locations and to provide the STATICFILES_STORAGE next to the DEFAUL_STORAGE setting for further extension.
I can't comment on the backends in django-storages much since I'm not very familiar with the code, but I assume that it's possible to subclass those storage backends to change the options, as needed. An extended example on djangosnippets.org or in the django-storages docs would cetainly be useful to have of course. If you have any suggestions regarding the staticfiles docs with regard to using other storage backends, I'd appreciate your help.
Ian
http://www.ianlewis.org/
Mikhail Korobov
authors of reusable apps have an answer for the "How should user
install app's static files?" question. They already know how to make
python code and django templates available, but static files was a bit
of pain before. Now the puzzle is assembled. So I think staticfiles is
a better fit for contrib than most of existing contrib apps. It
doesn't provide much functionality but it is a foundation that
reusable apps can rely on.
Waldemar Kornewald
>> The core 'django.views.static.serve' and 'django.core.context_processors.media' are deprecated in favor of the staticfiles equivalents in contrib. Is the idea that the contrib app is a stepping stone to providing core functionality? i.e. It was put in contrib to maintain backwards compatibility for at least enough time to satisfy the deprecation policy? I'm also getting worried in general about adding more and more functionality to Django that doesn't really *need* to be there.
>
> Moving pieces of code arund is part of the development process and hasn't really been done just for the sake of it. Personally, I don't share your worry about added functionality, as long as it solves real world problems, which staticfiles certainly did for me, as well as for the hundreds of other users, too.
>
With this reasoning we could as well add django-debug-toolbar, South,
django-registration and many other popular apps. What makes
staticfiles different? Seriously, I don't see it.
It's not even future-proof. We're heading towards larger client-side
web apps which means there will be HTML5 offline manifests and apps
consisting of more than 50 files. Even the combination of
django_compressor with staticfiles can't handle such apps. In a few
years staticfiles will be like the other "badteries" in contrib.
Well, looks like the Django core team doesn't care. :(
Bye,
Waldemar
James Bennett
<wkorn...@gmail.com> wrote:
> It's not even future-proof. We're heading towards larger client-side
> web apps which means there will be HTML5 offline manifests and apps
> consisting of more than 50 files. Even the combination of
> django_compressor with staticfiles can't handle such apps. In a few
> years staticfiles will be like the other "badteries" in contrib.
>
> Well, looks like the Django core team doesn't care. :(
Multiple committers have replied to you, explaining their points of
view on this. You disagree with their reasoning; that's unfortunate,
but it's also the nature of community-driven development. We work by
rough consensus, not by unanimous consent. None of which implies that
the people who've taken the time to present their arguments to you
"don't care" -- the fact that they're doing this shows that they do
care and are paying plenty of attention to criticisms.
So in the future, lay off the incendiary/insulting commentary, OK?
As for the main topic of the thread: personally, I think staticfiles
is fine in contrib. "How do I gather up the static files from all
these apps into one place where I can manage their deployment" is a
genuine problem people face when using Django. This app solves that
problem. It may not always be a perfect solution for everyone, but
neither are most of the apps in contrib; stuff like sessions, auth,
the admin, etc. all exist to solve common problems and provide a solid
baseline of supplementary functionality to go with the core framework,
and they all do a good job at that. But none of them are or ever can
be one-size-fits-all solutions which will work for every developer in
every situation, and it's unreasonable to expect them to suit every
use case. Such is life.
In the future, it may well turn out that people will commonly need
much more than what the current iteration of staticfiles provides. But
we can't cross that bridge until we come to it; for now, staticfiles
solves the common version of the problem (let's face it: *very* few
people are doing CDNs or offline HTML5 apps or any of the other stuff
brought up in this thread, as compared to gathering a bunch of files
and serving them up).
--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
Tobias McNulty
In the future, it may well turn out that people will commonly needmuch more than what the current iteration of staticfiles provides. But
we can't cross that bridge until we come to it; for now, staticfiles
solves the common version of the problem (let's face it: *very* few
people are doing CDNs or offline HTML5 apps or any of the other stuff
brought up in this thread, as compared to gathering a bunch of files
and serving them up).
Dougal Matthews
is there another mailing list thread or wiki page somewhere that follows or summarizes the decisions that went into django.contrib.staticfiles? I think pointing to the fact that such a process has already happened would also resolve a lot of the concerns in this thread.
Tobias McNulty
Jacob Kaplan-Moss
> That thread's pretty old and doesn't really end on anything conclusive other
> than "work has started". I do see that the patch was updated numerous times
> on the ticket [1] this month, but was there an associated review on the
> mailing list? Searching the group for "staticfiles" [2] or "12323" [3]
> doesn't turn up much, but maybe I'm missing something.
It looks like most of the review and discussion took place on the
ticket tracker and on IRC. I'll take a mea culpa for that: I should
have brought things back here.
I'd like to try again to actually move this forward: does anyone have
an alternate proposal? I'm hearing a bit of complaining, but no
alternate proposals.
Jacob
Jacob Kaplan-Moss
<wkorn...@gmail.com> wrote:
> With this reasoning we could as well add django-debug-toolbar, South,
> django-registration and many other popular apps. What makes
> staticfiles different? Seriously, I don't see it.
Jannis proposed that we add static files in. Brian concurred. So did
I. No core developers objected.
You're completely right about DDT, South, and -registration. What
makes them different?
* Nobody's asked that DDT be included in Django. Last I spoke to Rob
about it, he felt it needed a bit more time as a third-party app to
stabilize the API and get more robust. I'd happily support moving DDT
into contrib if there's sufficient interest.
* We've (at least, Russ and I have) talked a lot with Andrew about how
South might fit into contrib. The consensus was that there's still a
good deal of exploring to be done in the schema evolution space --
there's at least a couple-three tools that could easily be considered
"good enough". However, we agreed that we should push some of the
parts of schema migration down into Django, particularly DDL
generation, migration discovery, and perhaps migration tracking.
Andrew was offered a commit bit partially to help make this work
happen more smoothly.
* Like DDT, nobody's proposed that -registration be included in
Django. I'm not sure that James would be interested, either: he's got
a different opinion on contrib versus external apps than I do. Like
DDT, I'd go along if there was sufficient interest and if James was
willing.
Jacob
Peter Landry
I think South is the best candidate here, because it could improve the third
party app ecosystem to have a stable and consistent migration framework
battery included. DDT and django-registration both stand alone pretty well,
so I see less benefit (though I don't have any particular objections,
either).
Under that rubric, I agree that staticfiles is a good candidate, as well.
Peter
Tobias McNulty
On Thu, Oct 21, 2010 at 8:19 AM, Tobias McNulty <tob...@caktusgroup.com> wrote:It looks like most of the review and discussion took place on the
> That thread's pretty old and doesn't really end on anything conclusive other
> than "work has started". I do see that the patch was updated numerous times
> on the ticket [1] this month, but was there an associated review on the
> mailing list? Searching the group for "staticfiles" [2] or "12323" [3]
> doesn't turn up much, but maybe I'm missing something.
ticket tracker and on IRC. I'll take a mea culpa for that: I should
have brought things back here.
I'd like to try again to actually move this forward: does anyone have
an alternate proposal? I'm hearing a bit of complaining, but no
alternate proposals.
Jacob Kaplan-Moss
> I think the issue is that the commit has already been made, and that doesn't
> feel like the right time to anyone to submit an alternate proposal.
Well, that's why we use revision control: if there's a rough consensus
that a commit was a mistake, we can revert it. I'm hearing a few
voices in this thread that I *think* would like to see the change
backed out -- though nobody's actually suggested that explicit yet, so
I'm not sure. OTOH, I'm hearing a lot of "whoohoos" from users, so,
yes, we couldn't just pull the rug out: we'd need something better to
replace it with.
> As far as constructive feedback goes, something I would like to see is
> making "compilestatic" more configurable:
Yes, indeed -- my main concern pre-commit was configurability, and the
main focus of my code review was in making sure that we didn't do
anything that precluded adding better more powerful features later. I
didn't, however, want to hold off adding the feature waiting for some
ill-defined set of improvements. I wanted to see a minimalistic
addition that we can grow on later. That's how we roll.
I think realistically "later" likely means 1.4, but I'd happily review
patches adding more hooks at every time.
I really don't want staticfiles to be the tool that tries to do
everything, though. The idea is to provide the lowest common set of
functionality needed to handle media in reusable apps successfully,
and then to let other more complicated features (compression, etc.)
build on top of that. So yes: more hooks!
> One might prefer to serve media
> directly from the app static directories (via Aliases in apache or locations
> in nginx), instead of copying it manually every time during a deployment.
Hm, I certainly wouldn't... but it would be nice to make something
like that possible, yes. BTW, you can right now run `collectstatic
--links` to use symlinks instead of actually copying the files. Not
quite the same thing, but close right now.
> Another concern I have is about naming conflicts: how does staticfiles
> address the case where multiple apps supply files with the same names?
Just like templates: last one wins. See
http://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#collectstatic,
and also see the `findstatic` command that you can use to debug which
file shows up on which name.
> This
> is something that might also be addressed by being more configurable: I can
> see the case where it'd make more sense to expose app-specific URLs for
> media to the browser, instead of consolidating everything under a single
> global namespace. This could be implemented by a template tag, like so:
> {% staticfile 'appname' 'path/to/file.png' %}
I'm not quite sure I get the use case here, but like I said above: I
would be thrilled to see patches making this more configurable now.
Jacob
flo...@gmail.com
1) Should Django add more apps to contrib?
2) Does staticfiles solve a real need that users of Django have
today? Is it the de-facto implementation of a common pattern?
Let's just drop the discussion of the first point for the purposes of
this thread. At this point it's clear that contrib isn't going
anywhere soon, and certainly not without some concrete proposal by
someone and a lot of discussion.
So that leaves us with staticfiles, specifically. I've heard
countless times from people at conferences, at local meetup groups,
and elsewhere who are upset that Django doesn't give them any
direction on what to do with their static files. Django is fairly
opinionated software, but as soon as you start dealing with static
files Django suddenly has no opinion. That's confusing and
frustrating for users, to say the least.
I think staticfiles is an *excellent* contribution, and if there is
some advanced use case that it doesn't do, then let's do our best to
make it extensible enough to make that use case possible. Easy things
(like bundling media with your app) should be easy, but hard things
(like minification and consolidation of js) should be possible.
Anyway, that's my 2c.
Thanks,
Eric Florenzano
Waldemar Kornewald
Thanks a lot for the clarification. So, then the "bad batteries" part
in Eric's talk "Why Django sucks and how we can fix it" doesn't
receive much agreement within the Django core team?
My proposal would've been to not add staticfiles in the first place,
but it seems to be too late, now. From what I can see, only the
finder/storage API looks fully reusable. More advanced asset managers
will use custom templatetags, manage.py commands, and sometimes even
their own view in order to support CSS compilers and other advanced
features. So, is staticfiles' goal to only provide a reusable finder
API or are 3rd-party asset managers supposed to reuse anything else?
Bye,
Waldemar Kornewald
Tobias McNulty
On Thu, Oct 21, 2010 at 10:44 AM, Tobias McNulty <tob...@caktusgroup.com> wrote:Well, that's why we use revision control: if there's a rough consensus
> I think the issue is that the commit has already been made, and that doesn't
> feel like the right time to anyone to submit an alternate proposal.
that a commit was a mistake, we can revert it. I'm hearing a few
voices in this thread that I *think* would like to see the change
backed out -- though nobody's actually suggested that explicit yet, so
I'm not sure. OTOH, I'm hearing a lot of "whoohoos" from users, so,
yes, we couldn't just pull the rug out: we'd need something better to
replace it with.
Yes, indeed -- my main concern pre-commit was configurability, and the
> As far as constructive feedback goes, something I would like to see is
> making "compilestatic" more configurable:
main focus of my code review was in making sure that we didn't do
anything that precluded adding better more powerful features later. I
didn't, however, want to hold off adding the feature waiting for some
ill-defined set of improvements. I wanted to see a minimalistic
addition that we can grow on later. That's how we roll.
I think realistically "later" likely means 1.4, but I'd happily review
patches adding more hooks at every time.
I really don't want staticfiles to be the tool that tries to do
everything, though. The idea is to provide the lowest common set of
functionality needed to handle media in reusable apps successfully,
and then to let other more complicated features (compression, etc.)
build on top of that. So yes: more hooks!
Hm, I certainly wouldn't... but it would be nice to make something
> One might prefer to serve media
> directly from the app static directories (via Aliases in apache or locations
> in nginx), instead of copying it manually every time during a deployment.
like that possible, yes. BTW, you can right now run `collectstatic
--links` to use symlinks instead of actually copying the files. Not
quite the same thing, but close right now.
> Another concern I have is about naming conflicts: how does staticfilesJust like templates: last one wins. See
> address the case where multiple apps supply files with the same names?
http://docs.djangoproject.com/en/dev/ref/contrib/staticfiles/#collectstatic,
and also see the `findstatic` command that you can use to debug which
file shows up on which name.
> ThisI'm not quite sure I get the use case here, but like I said above: I
> is something that might also be addressed by being more configurable: I can
> see the case where it'd make more sense to expose app-specific URLs for
> media to the browser, instead of consolidating everything under a single
> global namespace. This could be implemented by a template tag, like so:
> {% staticfile 'appname' 'path/to/file.png' %}
would be thrilled to see patches making this more configurable now.
Jacob Kaplan-Moss
<wkorn...@gmail.com> wrote:
> Thanks a lot for the clarification. So, then the "bad batteries" part
> in Eric's talk "Why Django sucks and how we can fix it" doesn't
> receive much agreement within the Django core team?
I think that's kind of irrelevant: people complaining about "bad
batteries" are arguing that the solution should be "good batteries",
not "no batteries".
And look, the core team rarely has a single opinion -- there's 20 of
us with full commit now, so I don't see how we'd find a single thing
that all 20 agreed 100% on.
> My proposal would've been to not add staticfiles in the first place,
> but it seems to be too late, now.
If you think reverting the commit's a good idea, please feel free to
propose it and/or call for a vote. Me? I use django-staticfiles
already, so whether it's in contrib or not doesn't matter one iota.
Personally, if I got to start Django over from scratch I wouldn't have
a contrib. But I'm in the minority there. More importantly, the *vast*
majority of Django users I speak to love contrib and want to see it
grow, not shrink.
You have to remember that the voices on this list are actually a
pretty poor representation of our user community. People here are the
tip of the iceberg, the folks who have the time and inclination to
pitch in and help out. There are about 6,000 members on this mailing
list. django-users has three times as many members(18,000), and the
website gets something on the order of 50-70,000 uniques a month. If I
had to guess, I'd say that django-dev membership represents only 3-5%
of users worldwide.
I'm lucky enough to get to spend lots of my time talking to that other
95%. I'm by no means claiming to be at all scientific or unbiased or
anything here, but I would like just a bit of credit for trying to do
what I think will be best for our users. As Eric mentioned, static
file handling is actually a pretty big pain point right now.
> More advanced asset managers
> will use custom templatetags, manage.py commands, and sometimes even
> their own view in order to support CSS compilers and other advanced
> features.
To me that sounds like a feature, not a bug.
If staticfiles somehow gets in the way of any of that, then we'll fix it.
> So, is staticfiles' goal to only provide a reusable finder
> API or are 3rd-party asset managers supposed to reuse anything else?
I don't know -- let's find out. Patches welcome! If there's something
staticfiles can do to make -mediagenerator better/easier/awesomer then
I would be thrilled to review a patch.
Jacob
Jacob Kaplan-Moss
> What's the argument against serving the original files directly?
It means that when you add new apps you have to update your
Apache/Nginx/whatever config. Back in the day (pre-open-source), every
time you added an app you had to edit your Apache config, and we
*never* remembered that step because it's just such a totally
non-intuitive thing to do. Ideally, your web server should know as
little about your app as possible. That is, deployment tends to be
easier to closer to a black box your app becomes.
Anyway, though, we're getting off topic here -- if there's a way to
make staticfiles pluggable to the point that this sort of deployment
is possible, that's great and I'd love to see some ideas.
> Ah, so realistically we should put all our media in 'static/<appname>', like
> for templates, if we want to avoid conflicts with other apps. Would that be
> worth mentioning as a convention in the docs so we don't end up with a bunch
> of reusable apps that aren't at all reusable?
Dang it, I'd put this point in my outline and *thought* I put it in
the docs. But I didn't. Yes, a note to that effect (and/or an example)
would be the right thing to do. I'll add it when I get a chance.
Jacob
Waldemar Kornewald
> I think staticfiles is an *excellent* contribution, and if there is
> some advanced use case that it doesn't do, then let's do our best to
> make it extensible enough to make that use case possible. Easy things
> (like bundling media with your app) should be easy, but hard things
> (like minification and consolidation of js) should be possible.
OK, I just went through django-mediagenerator to check if there's
anything else needed by staticfiles and I noticed that we need to have
a standard for URLs in CSS files (e.g., url(image.png)).
Since we don't know the STATICFILES_URL in our CSS code we have to use
relative paths when referencing images. When CSS files get combined
the resulting file is often in a different folder than its source
files. This will break relative paths. Now we have these solutions:
1. keep URLs untouched when combining
2. rewrite URLs relative to source files
3. rewrite URLs relative to the combined file's path
4. rewrite URLs relative to STATICFILES_URL
URL rewriting would convert relative paths to absolute URLs beginning
with STATICFILES_URL.
(1) is the easiest solution and it might be used by very simple asset
managers. This approach is also used by quite a few CSS and JS
frameworks (e.g. jQuery UI). Disadvantage: It only works if the
combined CSS file is in the same location as its source files.
(2) is very intuitive because it behaves exactly like with uncombined
files. Disadvantage: URL rewriting is inconsistent if you also support
CSS compilers like Sass because the URLs used in imported files would
be relative to the master file. This means imported files have to make
assumptions about the master file's location which is very confusing
and bad for reusability.
(3) is too confusing and makes too many assumptions about the combined
file's location which is confusing and bad for reusability.
(4) is the most consistent solution. No matter whether you use a CSS
compiler or normal CSS files or something else, URLs always behave the
same. Disadvantage: At first you might expect different behavior, so
it's quite not as intuitive as (2), but it's very easy to understand
once you know what it does. Also, it doesn't have any of the
disadvantages of (2).
IMHO, (4) is the best solution because it's consistent. (2) is the
second best solution. It should be easy to adapt the code in
django-mediagenerator and make a little patch for staticfiles, so it
behaves like (4). What do you think?
In any case, staticfiles would need to rewrite URLs in its view, too.
Otherwise we can't provide a consistent solution across all asset
managers.
Bye,
Waldemar Kornewald
Waldemar Kornewald
Is staticfiles supposed to put "app/static/style.css" into
"<output>/style.css" or "<output>/app/style.css"? Currently it behaves
like the latter, but if it should behave like Django's templates we
need to fix the code.
Also, why is STATICFILES_DIRS split into a prefix and a root folder?
The prefix can easily be emulated via a subfolder within the root
folder. Is this really necessary or can we change this to a simple
list of strings? I'd happily provide the patch. :)
Luke Plant
> My proposal would've been to not add staticfiles in the first place,
> but it seems to be too late, now. From what I can see, only the
> finder/storage API looks fully reusable. More advanced asset managers
> will use custom templatetags, manage.py commands, and sometimes even
> their own view in order to support CSS compilers and other advanced
> features. So, is staticfiles' goal to only provide a reusable finder
> API or are 3rd-party asset managers supposed to reuse anything else?
As I see it, staticfiles primary goal is to fix tickets #12323 and
#11582, which it does nicely. Having user media and developer media
mixed together is a significant pain point for almost any project due to
the way that Django handled file uploads etc., and manually collecting
the media provided by re-usable apps was also a pain point for anyone
who used more than one app.
Back when we were discussing this, I had an alternative proposal which
only really addressed the first one (and mine was the only alternative
proposal as I remember). When I saw that Jannis was working on a fuller
solution, I happily dropped mine.
To fully support one of the other assets managers you mention, we would
need the admin and all contrib apps to get on board and use the template
tags etc. This would be a much more controversial change, and it would
probably stall for quite a while. Getting staticfiles in is much more
manageable, and a starting point for more advanced functionality.
Luke
--
If you can't answer a man's arguments, all is not lost; you can
still call him vile names. (Elbert Hubbard)
Luke Plant || http://lukeplant.me.uk/
Jannis Leidel
> "<output>/style.css" or "<output>/app/style.css"? Currently it behaves
> like the latter, but if it should behave like Django's templates we
> need to fix the code.
The latter is correct. E.g. files from apps (<app>/static/*) will be collected at the same relative path in the output directory, e.g. a file at ``/path/to/blog/static/blog/css/entries.css`` will be collected at ``/path/to/staticfiles_root/blog/css/entries.css``.
In your templates, you can refer to it with {% get_staticfiles_prefix %}blog/css/entries.css
> Also, why is STATICFILES_DIRS split into a prefix and a root folder?
> The prefix can easily be emulated via a subfolder within the root
> folder. Is this really necessary or can we change this to a simple
> list of strings? I'd happily provide the patch. :)
I believe this is a good point where the documentation is misleading and needs to be updated.
The *default* is to not have a prefix at all and to follow the same namespace idea as the app files finder (as above).
In fact it's perfectly fine to omit the prefix at all since we check what kind of value there is [1].
In other words, this:
STATICFILES_DIRS = (
('', '/home/special.polls.com/polls/media'),
('', '/home/polls.com/polls/media'),
('stuff', '/opt/webfiles/common'),
)
.. is the same as:
STATICFILES_DIRS = (
'/home/special.polls.com/polls/media',
'/home/polls.com/polls/media',
('stuff', '/opt/webfiles/common'),
)
You can *optionally* provide a prefix to namespace a bunch of files, e.g because they aren't in a sensible named directory. The following configuration would collect the zip files in a "downloads" directory and the files in MEDIA_ROOT in an "uploads" directory of the output directory.
STATICFILES_DIRS = (
("downloads", "/path/to/zip/files"),
("uploads", MEDIA_ROOT),
)
Jannis
Waldemar Kornewald
> On Thu, Oct 21, 2010 at 1:25 PM, Waldemar Kornewald
> <wkorn...@gmail.com> wrote:
>> My proposal would've been to not add staticfiles in the first place,
>> but it seems to be too late, now.
>
> If you think reverting the commit's a good idea, please feel free to
> propose it and/or call for a vote. Me? I use django-staticfiles
> already, so whether it's in contrib or not doesn't matter one iota.
Honestly, if enough other people wanted to see the commit reverted
we'd probably be done by now. ;)
> Personally, if I got to start Django over from scratch I wouldn't have
> a contrib. But I'm in the minority there. More importantly, the *vast*
> majority of Django users I speak to love contrib and want to see it
> grow, not shrink.
Quoting Henry Ford (sorry, it just fits so nicely :):
If I had asked people what they wanted, they would have said faster horses.
The majority of users don't think about what is better for Django on
the long run, but what is better for their own project right now.
Listening to users is good, but being controlled by them is bad.
>> More advanced asset managers
>> will use custom templatetags, manage.py commands, and sometimes even
>> their own view in order to support CSS compilers and other advanced
>> features.
>
> To me that sounds like a feature, not a bug.
>
> If staticfiles somehow gets in the way of any of that, then we'll fix it.
It does somewhat get in the way because apps that want to include
media files in their templates can't do it in a reusable way. Luckily
this is a rare case because you normally add the media in the base
template. The admin UI is an exception, though. In the end it's
probably an acceptable compromise to not have any official template
tag.
Bye,
Waldemar
Waldemar Kornewald
>> Is staticfiles supposed to put "app/static/style.css" into
>> "<output>/style.css" or "<output>/app/style.css"? Currently it behaves
>> like the latter, but if it should behave like Django's templates we
>> need to fix the code.
>
> The latter is correct. E.g. files from apps (<app>/static/*) will be collected at the same relative path in the output directory, e.g. a file at ``/path/to/blog/static/blog/css/entries.css`` will be collected at ``/path/to/staticfiles_root/blog/css/entries.css``.
Oh yeah. I got confused by the admin. It seems to be an exception to
this rule. Why don't we move the admin media and get rid of the
exception? Does this little change have to go through a deprecation
cycle?
Waldemar Kornewald
I doubt that we will ever get the admin to support more powerful asset
managers. There are just too many incompatible approaches out there
and like you said it'll be difficult to agree on a compromise. It
might be impossible to solve this problem without requiring an asset
manager that can combine your CSS and JS.
Bye,
Waldemar
Tom Evans
> On Thu, Oct 21, 2010 at 1:25 PM, Waldemar Kornewald
> <wkorn...@gmail.com> wrote:
>> Thanks a lot for the clarification. So, then the "bad batteries" part
>> in Eric's talk "Why Django sucks and how we can fix it" doesn't
>> receive much agreement within the Django core team?
>
> I think that's kind of irrelevant: people complaining about "bad
> batteries" are arguing that the solution should be "good batteries",
> not "no batteries".
>
(I don't particularly have an objection to contrib.staticfiles)
I think that things should only be in contrib if they are universally
'The One True Way'. For instance, there are not other admin suites out
there clamouring to be django's admin, nor is there anyone out there
thinking that generic keys should be implemented in a different manner
or that pagination isn't doing the right thing.
On the other hand, schema migration has a number of approaches (I
personally use South), there are many resource bundlers available. I
personally don't think that things should be going into contrib just
because they fill a hole, but because they truly are the 'correct'
implementation.
Cheers
Tom
Waldemar Kornewald
Since this is pretty important and it would be great to have in the
alpha release, what do you need to be able to make a good decision,
here?
Bye,
Waldemar
Mikhail Korobov
Could you explain why is this should belong to django staticfiles app?
This app has nothing to do with combining css files. It has one view
(django.contrib.staticfiles.views.serve) in order to serve files in
development. This is only a helper view used in development and I
don't see why it is necessary to support css-combining solutions in
it. If some app combines/compresses css/js files it should handle the
path rewriting etc. and provide a way to insert the media into
templates (e.g. via templatetag as django-compress do) because all
these stuff can be handled in different ways.
django.contrib.staticfiles (and it's serve view) doesn't stand in the
way here and I think it's the best it could do. If
django.contrib.staticfiles prevents something to be implemented then
it's an another story and this should be fixed.
Waldemar Kornewald
On Wed, Oct 27, 2010 at 1:14 PM, Mikhail Korobov <kmi...@googlemail.com> wrote:
> Hi Waldemar,
>
> Could you explain why is this should belong to django staticfiles app?
> This app has nothing to do with combining css files. It has one view
> (django.contrib.staticfiles.views.serve) in order to serve files in
> development. This is only a helper view used in development and I
> don't see why it is necessary to support css-combining solutions in
> it. If some app combines/compresses css/js files it should handle the
> path rewriting etc. and provide a way to insert the media into
> templates (e.g. via templatetag as django-compress do) because all
> these stuff can be handled in different ways.
> django.contrib.staticfiles (and it's serve view) doesn't stand in the
> way here and I think it's the best it could do. If
> django.contrib.staticfiles prevents something to be implemented then
> it's an another story and this should be fixed.
While it's true that staticfiles doesn't handle CSS-combining, many
people will want to combine their files and thus use a different asset
manager (potentially building on top of staticfiles; doesn't matter).
If staticfiles doesn't set a standard here then different apps will
depend on different (incompatible) URL rewriting schemes and this is
bad for reusability. Staticfiles is supposed to prevent this. So, we
need a standard here. Depending on which standard we choose it can
have an influence on staticfiles and in order to be compliant with the
standard we might have to rewrite URLs in staticfiles, too.
Bye,
Waldemar
Mikhail Korobov
different assets bundlers?
E.g. if one uses django-compress he should list all files he wants to
combine in settings.py and then output them into template as {%
compressed_css 'my_css' %}. If one is using django-compressor then he
list all files he want to combine in template inside some other other
template tag. Then django-compress and django-compressor combine
files, store combined variants somewhere and provide correct urls for
stored files.
Why should one care if these files are stored and prepared the same
way in django-mediagenerator, django-compress and django-compressor or
not? Why should django.contrib.staticfiles care about this? There are
a lot of bundling apps now and they all already provide different
answers for this problem.
Please excuse me if I don't understand some obvious but I can't see a
point. As I can see, the purpose of django.contrib.staticfiles is to
collect media files from reusable applications in one place so that
developers of reusable application can rely on it and don't require
users to manually copy and update files from app source releases. It
also solves a problem of media files customizing - where to put
customized media files so that they can survive app upgrade. And
that's all. Why do you see django.contrib.staticfiles as a framework
for css/js combiners?
Waldemar Kornewald
> Why isn't it fine to have different URL rewriting schemes for
> different assets bundlers?
OK, sorry for not having explained it well. What I mean is this:
Imagine this code snippet in a reusable app's CSS file:
/* myapp/style.css */
div.icon {
background-image: url(img/icon.png);
}
Now this file gets combined into "css/main.css". Depending on which
asset manager and URL rewriting scheme you use the URL can be:
1. unmodified: "img/icon.png"
2. relative to the current file: "/static/myapp/img/icon.png"
3. relative to the combined file: "/static/css/img/icon.png"
4. relative to STATICFILES_URL "/static/img/icon.png"
If one developer who uses an asset manager based on (2) publishes a
reusable app and another developer who uses an asset manager based on
(4) publishes a reusable app there is no way to put both apps into
your project and have both work correctly with the same asset manager.
Note, in practice, (1) and (3) have pretty much the same result.
Anyway, (1) and (3) would be very bad ideas if you want to support
combined files, so let's leave them out of this discussion.
(2) is what everyone is used to, but it's inconsistent if you use Sass
or other CSS compilers because they allow importing other files which
might also contain URLs and there's no way for the asset manager to
rewrite URLs relative to the imported file's path (the asset manager
only gets one big result file which already contains all imported
files). This inconsistency is annoying e.g. when you develop both with
CSS and Sass.
(4) is fully consistent and easy to understand, but obviously behaves
differently from what most people are used to when developing .
However, this way all URLs look the same, no matter if you use Sass or
CSS or both.
(1), (2), and (3) would already be compatible with the current
staticfiles implementation. (4) would require a little modification.
I've seen (1), (2), and (4) used in practice when combining CSS files.
Which behavior should be the standard for all asset managers?
bur...@gmail.com
(4) is no good because you might have file name conflicts when used
with multiple reusable applications.
Why haven't you mentioned this problem?
> (1), (2), and (3) would already be compatible with the current
> staticfiles implementation. (4) would require a little modification.
> I've seen (1), (2), and (4) used in practice when combining CSS files.
>
> Which behavior should be the standard for all asset managers?
I think, in a compressor, you shouldn't rely on STATICFILES_URL but on your own
DJANGO_MEDIAGENERATOR_FILES_URL and DJANGO_MEDIAGENERATOR_FILES_ROOT,
which can be set up to implement necessary strategy from (1)-(4).
--
Best regards, Yuri V. Baburov, ICQ# 99934676, Skype: yuri.baburov,
MSN: bu...@live.com
Waldemar Kornewald
Which conflict do you mean? If you want to refer to some app's image
from "app/css/style.css" you'd use "app/img/icon.png" with (4) and
"../img/icon.png" with (2). Each path with (4) maps to exactly one
path with (2) and the other way around, so any conflict that applies
to (4) also applies to (2).
>> (1), (2), and (3) would already be compatible with the current
>> staticfiles implementation. (4) would require a little modification.
>> I've seen (1), (2), and (4) used in practice when combining CSS files.
>>
>> Which behavior should be the standard for all asset managers?
>
> I think, in a compressor, you shouldn't rely on STATICFILES_URL but on your own
> DJANGO_MEDIAGENERATOR_FILES_URL and DJANGO_MEDIAGENERATOR_FILES_ROOT,
> which can be set up to implement necessary strategy from (1)-(4).
Whether URLs are rewritten with STATICFILES_URL or
DJANGO_MEDIAGENERATOR_FILES_URL doesn't make a real difference for
this discussion. I'm fine with either setting, but let's first focus
on the general URL rewriting solution, please.
bur...@gmail.com
it seems I just don't get what does "relative to" mean in your (1)-(4).
Could you please explain better, what do you mean by that?
Starting with what path does your "css/main.css" have?
I think you've missed some important bits in your explanations, or
just not covering every possible choice.
I presumed that with making css/main.css you're just copying that
image over to your new location like staticfiles does (or symlinking
images folder).
On Thu, Oct 28, 2010 at 1:29 PM, Waldemar Kornewald
<wkorn...@gmail.com> wrote:
> Hi Yuri,
>
> On Thu, Oct 28, 2010 at 6:19 AM, bur...@gmail.com <bur...@gmail.com> wrote:
>> Hi Waldemar,
>>
>> On Wed, Oct 27, 2010 at 9:05 PM, Waldemar Kornewald
>> <wkorn...@gmail.com> wrote:
>>> 2010/10/27 Mikhail Korobov <kmi...@googlemail.com>:
>>>> Why isn't it fine to have different URL rewriting schemes for
>>>> different assets bundlers?
>>>
>>> OK, sorry for not having explained it well. What I mean is this:
>>> Imagine this code snippet in a reusable app's CSS file:
>>>
>>> /* myapp/style.css */
>>> div.icon {
>>> background-image: url(img/icon.png);
>>> }
>>>
>>> Now this file gets combined into "css/main.css". Depending on which
>>> asset manager and URL rewriting scheme you use the URL can be:
>>> 1. unmodified: "img/icon.png"
>>> 2. relative to the current file: "/static/myapp/img/icon.png"
>>> 3. relative to the combined file: "/static/css/img/icon.png"
>>> 4. relative to STATICFILES_URL "/static/img/icon.png"
>
> Which conflict do you mean? If you want to refer to some app's image
> from "app/css/style.css" you'd use "app/img/icon.png" with (4) and
> "../img/icon.png" with (2). Each path with (4) maps to exactly one
> path with (2) and the other way around, so any conflict that applies
> to (4) also applies to (2).
>
>>> (1), (2), and (3) would already be compatible with the current
>>> staticfiles implementation. (4) would require a little modification.
>>> I've seen (1), (2), and (4) used in practice when combining CSS files.
>>>
>>> Which behavior should be the standard for all asset managers?
>>
>> I think, in a compressor, you shouldn't rely on STATICFILES_URL but on your own
>> DJANGO_MEDIAGENERATOR_FILES_URL and DJANGO_MEDIAGENERATOR_FILES_ROOT,
>> which can be set up to implement necessary strategy from (1)-(4).
>
> Whether URLs are rewritten with STATICFILES_URL or
> DJANGO_MEDIAGENERATOR_FILES_URL doesn't make a real difference for
> this discussion. I'm fine with either setting, but let's first focus
> on the general URL rewriting solution, please.
>
> Bye,
> Waldemar
>
> --
> Django on App Engine, MongoDB, ...? Browser-side Python? It's open-source:
> http://www.allbuttonspressed.com/blog/django
>
> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Waldemar Kornewald
On Thu, Oct 28, 2010 at 3:18 PM, bur...@gmail.com <bur...@gmail.com> wrote:
> Hi Waldemar,
>
> it seems I just don't get what does "relative to" mean in your (1)-(4).
> Could you please explain better, what do you mean by that?
> Starting with what path does your "css/main.css" have?
> I think you've missed some important bits in your explanations, or
> just not covering every possible choice.
> I presumed that with making css/main.css you're just copying that
> image over to your new location like staticfiles does (or symlinking
> images folder).
The image is just copied over. That's not the problem. The problem is
when you combine/concat CSS files. Indeed, staticfiles doesn't support
this, but it's still a real problem in practice because many people
use an asset manager that does combine files.
The source files might be
* app_a/static/app_a/css/style.css
* app_a/static/app_a/img/icon.png
* app_b/static/app_b/style.css
* app_b/static/app_b/icon.png
* ...
The combined CSS file might be written to "static_root/css/main.css",
so you'd have this result after generating your assets:
* static_root/css/main.css
* static_root/app_a/img/icon.png
* static_root/app_b/icon.png
Imagine that the input CSS files reference images via url() (e.g.:
"background-image: url(....)"). The big question is: What happens to
those url() paths when the files are combined?
Imagine you have this in app_a/css/style.css:
# app_a/static/app_a/css/style.css
div.app_a_icon {
background-image: url(../img/icon.png);
}
What will this snippet look like in main.css after the files are combined?
With solution (1) we'd have no change:
# static_root/css/main.css
div.app_a_icon {
background-image: url(../img/icon.png);
}
However, this won't work from main.css's location. The correct path
would be url(../app_a/img/icon.png). This means that everyone who uses
an asset manager based on method (1) needs to use a different path in
style.css:
# app_a/static/app_a/css/style.css
div.app_a_icon {
background-image: url(../app_a/img/icon.png);
}
Now imagine that someone else uses an asset manager with method (2).
In his case URLs are rewritten relative to the source file. This means
he has to write:
url(../img/icon.png) in app_a/css/style.css
That's because the source folder is "static_root/app_a/css/" and if
you chdir into that folder then "../img/icon.png" refers to the file
"static_root/app_a/img/icon.png". That's what I mean with "relative
to" the source file.
With method (4) the same file would be referenced via
url(app_a/img/icon.png) in app_a/css/style.css
That's because here all paths are relative to the root media folder.
If you chdir into the "static_root" folder then "app_a/img/icon.png"
refers to "static_root/app_a/img/icon.png".
What's the problem with all of this? Code written for (1) is
incompatible with code written for (2) which is incompatible with code
written for (4). The asset managers listed on djangopackages use any
of those three methods. There is no agreement and none of those
methods seems to be preferred:
* django-compressor: (2)
* django-mediasync: (1)
* django-compress: (1)
* django-mediagenerator: (4)
* django-static: (4)
* django-static-media-manager: (1)
* django-assets: (2)
Solution (1) is useless when writing reusable code because (as you've
seen above) the source files would have to know the location of the
combined output file. This leaves (2) and (4) as possible options.
The problem with (2) is that it won't work well with Sass and other
CSS compilers. In Sass when you use @import the imported file is
copied into the master. If you have
# app_a/static/app_a/x.sass
@import subfolder/y
# app_a/static/app_a/subfolder/y.sass
... url(... what comes here? ...)
You want to refer to "static_root/app_a/img/icon.png" from y.sass.
Should the URL be "../img/icon.png" or "img/icon.png" or
"app_a/img/icon.png"? With solution (2) only the second URL works
because y.sass gets copied literally into x.sass by the Sass
executable. This means that y.sass would have to know the location of
x.sass. This problem is similar to solution (1) and makes writing
reusable code too difficult or even impossible.
What would this look like with (4)?
# app_a/static/app_a/subfolder/y.sass
... url(app_a/img/icon.png)
And what would the URL look like in app_a/css/style.css, again?
# app_a/static/app_a/css/style.css
... url(app_a/img/icon.png);
Look closely. It's *exactly* the same. It absolutely doesn't matter
where the source file is placed and it also doesn't matter where the
combined file is placed. You always use the same URL to refer to an
image. It's never ambiguous. BTW, method (4) has the same behavior as
Django's templates: {% extends %} and {% include %} are relative to
the root template folder, not the source file.
The only advantage of (2) is that it works more like what most
developers are used to when working without any asset manager. Since
(4) allows for much more consistent URLs I'd say method (4) is still
better than (2), though.
Bye,
Waldemar Kornewald
Waldemar Kornewald
<wkorn...@gmail.com> wrote:
> What's the problem with all of this? Code written for (1) is
> incompatible with code written for (2) which is incompatible with code
> written for (4). The asset managers listed on djangopackages use any
> of those three methods. There is no agreement and none of those
> methods seems to be preferred:
> * django-compressor: (2)
> * django-mediasync: (1)
> * django-compress: (1)
> * django-mediagenerator: (4)
> * django-static: (4)
> * django-static-media-manager: (1)
> * django-assets: (2)
BTW, if the combined output CSS file is always placed into the root
folder (static_root/main.css) then code written for method (1) is
compatible with method (4).
Bye,
Waldemar
Carl Meyer
Thanks for putting so much thought into this issue, and outlining
these options in detail. However, I am not convinced that this
something Django core should be concerned with. I think we need to
maintain a clearer conceptual separation between the various layers of
functionality here.
As I read it, your option 4 means putting URLs into CSS files that
will not resolve correctly if static files are served directly,
unmodified, from their source locations (after being collected from
apps) under STATICFILES_URL (because the URLs you give don't begin
with a slash, so they will be interpreted as relative to the current
location; and if they did begin with a slash, that would break anytime
STATICFILES_URL is not a path-less domain). In other words, you are
proposing to write CSS files that _depend_ on being run through some
kind of combiner/compressor/filter that will intelligently rewrite all
their URLs relative to STATICFILES_URL. As a proposal for a
"standard," this is a non-starter for several reasons:
1. Not all projects want or need to combine/compress their media at
all, and there is no reason they should have to run their CSS through
a URL-rewriting filter.
2. Even in projects that do combine/compress media, there are good
reasons to want to serve the individual files directly in development
(to aid in front-end debugging), and only do the combining/compressing
step on deployment to staging/production.
3. Designers and front-end developers working on Django projects
should not have to learn some odd way of linking in their CSS, that
appears to be wrong according to everything they know about how things
usually work. The links they write should make sense to them, not just
to some Python developer who invented a new "standard" for them.
4. Django does not provide any built-in tools to combine, compress, or
rewrite links in media, so it is certainly not going to adopt a
"standard" that requires the use of such a tool.
In other words, the only reason a "standard" would even be needed here
is because you want to establish a new standard which makes the source
CSS files unusable in the absence of a filter which implements the
standard!
I think the problem here is a desire to conflate multiple issues which
are clearer when considered separately. To wit:
1. Links in static files should be written in such a way they work as-
is, without knowing what root URL they will be served under (this
means relative internal links).
2. Any third-party tool that combines/compresses CSS should be able to
take working input files and generate working output files, handling
whatever internal links it finds correctly.
Both of these seem almost so obvious that they shouldn't require
stating at all; you'll also note that they keep the question of
internal linkage neatly separated from the presence or absence of a
hypothetical compressor/combiner.
The only argument you present against this approach is that it's
problematic for Sass; considering all of my projects use Sass, I
disagree. It simply requires the Sass author to understand the tool
they are using and know how imports and paths work (or use a Sass
framework such as Compass, which already handles the problem neatly).
This is the nature of Sass, is second-nature to any experienced Sass
author, and is in no way specific to Django; there's no reason for
Django to be providing a solution for it.
Carl
Waldemar Kornewald
On Fri, Oct 29, 2010 at 5:18 AM, Carl Meyer <carl.j...@gmail.com> wrote:
> Hi Waldemar,
>
> Thanks for putting so much thought into this issue, and outlining
> these options in detail. However, I am not convinced that this
> something Django core should be concerned with. I think we need to
> maintain a clearer conceptual separation between the various layers of
> functionality here.
Just to clarify: We are in fact talking about two questions here:
1. Do we need a standard for URL handling?
This is the most important question and my answer is "yes" (I'll
explain this in the rest of this mail).
2. I'm suggesting that (4) should be the standard.
This is where you focused your reply and from this you somehow
concluded that the answer to the first question is "no". The code
examples in my previous mail show that we do need a standard
(independent of which method becomes the standard).
> As I read it, your option 4 means putting URLs into CSS files that
> will not resolve correctly if static files are served directly,
> unmodified, from their source locations (after being collected from
> apps) under STATICFILES_URL (because the URLs you give don't begin
> with a slash, so they will be interpreted as relative to the current
> location; and if they did begin with a slash, that would break anytime
> STATICFILES_URL is not a path-less domain). In other words, you are
> proposing to write CSS files that _depend_ on being run through some
> kind of combiner/compressor/filter that will intelligently rewrite all
> their URLs relative to STATICFILES_URL. As a proposal for a
> "standard," this is a non-starter for several reasons:
>
> 1. Not all projects want or need to combine/compress their media at
> all, and there is no reason they should have to run their CSS through
> a URL-rewriting filter.
When your CSS files are split into different locations via apps and
generated via manage.py collectstatic and served via the staticfiles
view it makes no real difference *from a tools perspective*. You
already depend on those extra tools/filters, anyway. Your point 3
below is about the human perspective which is the real problem here.
> 2. Even in projects that do combine/compress media, there are good
> reasons to want to serve the individual files directly in development
> (to aid in front-end debugging), and only do the combining/compressing
> step on deployment to staging/production.
django-mediagenerator and several other asset managers have a view
which takes care of serving media uncombined and uncompressed during
development for exactly that reason (easier debugging).
If you're not yet using a professional asset manager you really try
one. It can provide a noticeable speed boost (beyond just
combining/compressing media) by utilizing the browser's cache
perfectly and it can also help you during development.
> 3. Designers and front-end developers working on Django projects
> should not have to learn some odd way of linking in their CSS, that
> appears to be wrong according to everything they know about how things
> usually work. The links they write should make sense to them, not just
> to some Python developer who invented a new "standard" for them.
I see the point you're trying to make and I don't like that solution
(4) breaks with an existing convention, either. There's still the
other option of going with (2) for CSS and with (4) for Sass/etc. It's
just inconsistent and this becomes even more confusing if you use some
language with CSS-like syntax (e.g., SCSS).
> 4. Django does not provide any built-in tools to combine, compress, or
> rewrite links in media, so it is certainly not going to adopt a
> "standard" that requires the use of such a tool.
While staticfiles doesn't combine media it definitely is a bulit-in
tool and it could easily be extended to rewrite URLs (for the purpose
of having a standard).
> In other words, the only reason a "standard" would even be needed here
> is because you want to establish a new standard which makes the source
> CSS files unusable in the absence of a filter which implements the
> standard!
>
> I think the problem here is a desire to conflate multiple issues which
> are clearer when considered separately. To wit:
>
> 1. Links in static files should be written in such a way they work as-
> is, without knowing what root URL they will be served under (this
> means relative internal links).
> 2. Any third-party tool that combines/compresses CSS should be able to
> take working input files and generate working output files, handling
> whatever internal links it finds correctly.
>
> Both of these seem almost so obvious that they shouldn't require
> stating at all; you'll also note that they keep the question of
> internal linkage neatly separated from the presence or absence of a
> hypothetical compressor/combiner.
I agree with 1. I also fully agree that 2 *should* work the way you
described. However, without a standard it can't and in fact this very
moment it doesn't. Depending on your asset manager you might use
method (1) or (2) or (4) and thus use different URL paths than some
other developer. We already have this problem right now: CSS files
created for django-compress are incompatible with django-compressor
and CSS files created for django-mediagenerator are incompatible with
django-compress. Your point 2 doesn't reflect reality. It's a nice
dream and an official standard is the first step needed for it to come
true. That's what I'm fighting for, here.
> The only argument you present against this approach is that it's
> problematic for Sass; considering all of my projects use Sass, I
> disagree. It simply requires the Sass author to understand the tool
> they are using and know how imports and paths work (or use a Sass
> framework such as Compass, which already handles the problem neatly).
> This is the nature of Sass, is second-nature to any experienced Sass
> author, and is in no way specific to Django; there's no reason for
> Django to be providing a solution for it.
Compass solves this by adding a custom image_url() directive and the
paths are pretty much like with method (4), i.e.: they're relative to
the root images folder. I'd say this fact supports my suggestion that
method (4) is the right one for projects with Sass files. Any other
solution would be difficult to understand and bad for reusability
because the imported files (e.g., a/_x.sass) would have to know the
output file's location (e.g., app/main.css or css/main.css or just
main.css or ...). Only (4) is a practical solution for Sass.
So, should we
* use the same method (4) for CSS and Sass/etc.? (+consistency, -familiarity)
* use method (2) for CSS and method (4) for Sass/etc.? (-consistency,
+familiarity)
Bye,
Waldemar
bur...@gmail.com
On Fri, Oct 29, 2010 at 2:05 PM, Waldemar Kornewald
<wkorn...@gmail.com> wrote:
> Hi Carl,
>
>> As I read it, your option 4 means putting URLs into CSS files that
>> will not resolve correctly if static files are served directly,
>> unmodified, from their source locations (after being collected from
>> apps) under STATICFILES_URL (because the URLs you give don't begin
>> with a slash, so they will be interpreted as relative to the current
>> location; and if they did begin with a slash, that would break anytime
>> STATICFILES_URL is not a path-less domain). In other words, you are
>> proposing to write CSS files that _depend_ on being run through some
>> kind of combiner/compressor/filter that will intelligently rewrite all
>> their URLs relative to STATICFILES_URL. As a proposal for a
>> "standard," this is a non-starter for several reasons:
Do you agree with that "you are proposing to write CSS files that
_depend_ on being run through some kind of combiner/compressor/filter
that will intelligently rewrite all their URLs relative to
STATICFILES_URL"?
If you do, then you understand that you're trying to impose a standard
for django users that is really unnecessary for them?
You're free to allow such perversion with your app, but please don't
try to put this into django.
Why can't you read css files, transform their paths to absolute, merge
files, and then write paths back as relative to merged files?
Waldemar Kornewald
On Fri, Oct 29, 2010 at 10:37 AM, bur...@gmail.com <bur...@gmail.com> wrote:
> Hi Waldemar,
>
> On Fri, Oct 29, 2010 at 2:05 PM, Waldemar Kornewald
> <wkorn...@gmail.com> wrote:
>> Hi Carl,
>>
>>> As I read it, your option 4 means putting URLs into CSS files that
>>> will not resolve correctly if static files are served directly,
>>> unmodified, from their source locations (after being collected from
>>> apps) under STATICFILES_URL (because the URLs you give don't begin
>>> with a slash, so they will be interpreted as relative to the current
>>> location; and if they did begin with a slash, that would break anytime
>>> STATICFILES_URL is not a path-less domain). In other words, you are
>>> proposing to write CSS files that _depend_ on being run through some
>>> kind of combiner/compressor/filter that will intelligently rewrite all
>>> their URLs relative to STATICFILES_URL. As a proposal for a
>>> "standard," this is a non-starter for several reasons:
> Do you agree with that "you are proposing to write CSS files that
> _depend_ on being run through some kind of combiner/compressor/filter
> that will intelligently rewrite all their URLs relative to
> STATICFILES_URL"?
I keep getting misunderstood, so I'll just simplify it. Forget all my
proposals. My primary proposal is:
"Let's have exactly one official standard, no matter if it's (2) or (4)."
What I don't want is (1), (2), and (4) living side-by-side as they do now.
Do you agree that we should have a standard?
If the answer is "yes", then which one should it be?
A: (4) for CSS and Sass/etc.
B: (2) for CSS, (4) for Sass/etc.
> If you do, then you understand that you're trying to impose a standard
> for django users that is really unnecessary for them?
> You're free to allow such perversion with your app, but please don't
> try to put this into django.
>
> Why can't you read css files, transform their paths to absolute, merge
> files, and then write paths back as relative to merged files?
That's method (2) which only works well for CSS.
bur...@gmail.com
On Fri, Oct 29, 2010 at 4:19 PM, Waldemar Kornewald
Just a standard? Or standard for some problem related to django?
> If the answer is "yes", then which one should it be?
> A: (4) for CSS and Sass/etc.
> B: (2) for CSS, (4) for Sass/etc.
>
>> If you do, then you understand that you're trying to impose a standard
>> for django users that is really unnecessary for them?
>> You're free to allow such perversion with your app, but please don't
>> try to put this into django.
>>
>> Why can't you read css files, transform their paths to absolute, merge
>> files, and then write paths back as relative to merged files?
>
> That's method (2) which only works well for CSS.
Django doesn't support Sass.
Django doesn't support media compression.
Why django should have this stupid standard?
Related to your media compressor, i'd prefer not (2) or (4), i prefer
this one I described above:
1. Your compressor starts after media is gathered with staticfiles
into a single place. (All relative paths are valid at this moment!)
2. You read css files, transform their url paths to absolute, merge
files, and then write back into user specified directory with paths
rewritten as relative to merged files location.
3. If you're copying images, you can put images to whatever folder you
want, but they should still work.
Your compressor should make it possible to work after your compression
if you'll just put that folder to media server.
That way, every compressor is compatible with each other if their
output is set to compressor-specific folder.
This doesn't impose any standard of writing urls in their files on
django users -- they do what they did before they have any staticfiles
support.
Waldemar Kornewald
A standard for a problem related to Django: How can we have reusable
apps that come with media files? How is that possible if everyone uses
a different asset manager?
Unless I've misunderstood something, the Django team added
django.contrib.staticfiles exactly because this app is supposed to
solve the reusable-apps-with-media-files problem. We need a standard
URL rewriting scheme for this.
>> If the answer is "yes", then which one should it be?
>> A: (4) for CSS and Sass/etc.
>> B: (2) for CSS, (4) for Sass/etc.
>>
>>> If you do, then you understand that you're trying to impose a standard
>>> for django users that is really unnecessary for them?
>>> You're free to allow such perversion with your app, but please don't
>>> try to put this into django.
>>>
>>> Why can't you read css files, transform their paths to absolute, merge
>>> files, and then write paths back as relative to merged files?
>>
>> That's method (2) which only works well for CSS.
> Django doesn't support Sass.
> Django doesn't support media compression.
> Why django should have this stupid standard?
Because practically all asset managers for Django (except for
"staticfiles") do support media combining, and a standard is required
to make those asset managers compatible with each other, so we can
write reusable apps. No standard => no reusable apps.
You also can't ignore the growing number of web developers (I've seen
a poll saying 20%) who use CSS compilers like Sass. The number is
large enough to matter even for Django.
> Related to your media compressor, i'd prefer not (2) or (4), i prefer
> this one I described above:
> 1. Your compressor starts after media is gathered with staticfiles
> into a single place. (All relative paths are valid at this moment!)
> 2. You read css files, transform their url paths to absolute, merge
> files, and then write back into user specified directory with paths
> rewritten as relative to merged files location.
> 3. If you're copying images, you can put images to whatever folder you
> want, but they should still work.
> Your compressor should make it possible to work after your compression
> if you'll just put that folder to media server.
This is method (2). :)
It rewrites the URLs relative to the source files before merging
(which is your step 2).
> That way, every compressor is compatible with each other if their
> output is set to compressor-specific folder.
> This doesn't impose any standard of writing urls in their files on
> django users -- they do what they did before they have any staticfiles
> support.
You say that you don't want a standard, but what you describe in your
mail is a standard. :) What else would you call your three
instructions above? They describe how to rewrite URLs according to
(2).
Please take a look at the existing asset managers:
http://djangopackages.com/grids/g/asset-managers/
Only two of them use (2). All the others use (1) and (4). Where do you
see the compatibility here?
If you still don't agree then see it for yourself: Try to use
*exactly* the same reusable app with *exactly* the same CSS files with
three different asset managers which combine those app CSS files into
a single file. Tell me if it works. You'll see it's impossible.
bur...@gmail.com
> A standard for a problem related to Django: How can we have reusable
> apps that come with media files? How is that possible if everyone uses
> a different asset manager?
Ok, this is a problem related to Django, but it shouldn't be solved
with scope of Django because nothing in Django depends on this
decision. Now it is better?
> Unless I've misunderstood something, the Django team added
> django.contrib.staticfiles exactly because this app is supposed to
> solve the reusable-apps-with-media-files problem. We need a standard
> URL rewriting scheme for this.
>> Related to your media compressor, i'd prefer not (2) or (4), i prefer
>> this one I described above:
>> 1. Your compressor starts after media is gathered with staticfiles
>> into a single place. (All relative paths are valid at this moment!)
>> 2. You read css files, transform their url paths to absolute, merge
>> files, and then write back into user specified directory with paths
>> rewritten as relative to merged files location.
>> 3. If you're copying images, you can put images to whatever folder you
>> want, but they should still work.
>> Your compressor should make it possible to work after your compression
>> if you'll just put that folder to media server.
>
> This is method (2). :)
> It rewrites the URLs relative to the source files before merging
> (which is your step 2).
>
>> That way, every compressor is compatible with each other if their
>> output is set to compressor-specific folder.
>> This doesn't impose any standard of writing urls in their files on
>> django users -- they do what they did before they have any staticfiles
>> support.
>
> You say that you don't want a standard, but what you describe in your
> mail is a standard. :) What else would you call your three
> instructions above? They describe how to rewrite URLs according to
> (2).
I clearly said it's my preference.
> If you still don't agree then see it for yourself: Try to use
> *exactly* the same reusable app with *exactly* the same CSS files with
> three different asset managers which combine those app CSS files into
> a single file. Tell me if it works. You'll see it's impossible.
Who said this should work?
I understand you wants to get everyone happy, by taking a decision
instead of users.
Harro
up to the app/tool that does this to fix relative paths and other
issues that arrise when compressing/combining files.
It has nothing to do with the serving of the files.
>
>
>
> <wkornew...@gmail.com> wrote:
> > Hi Yuri,
>
Jacob Kaplan-Moss
Like a few in this thread, I'm really having trouble understanding
exactly what you're proposing. I think the best thing, then, would be
if you could write some code to do whatever it is you'd like.
Just a proof-of-concept is totally fine -- I don't mean to ask you to
invest a lot more time than you already have -- but I think I need to
actually see a patch (and perhaps an example of how the functionality
would be used?) to understand what it is you'd like to see happen
here.
Thanks,
Jacob
Waldemar Kornewald
OK, so you are thinking in terms of code. A standard doesn't have to
be in code. The standard I'm talking about is primarily about
communication done officially by Django, not code that lives in Django
core. In order to explain it better I guess I have to send a patch, so
here it goes (see attachment which "implements" method (2) :). I see
no better way to make it clear what I mean with having a standard for
reusable open-source apps. Take a look at the patch, please. I'm sure
you're shocked, now, and I hope that my whole intention becomes clear:
We need a standard to write reusable open-source apps, so the
community can share more code easily.
If we ignore the standard, the real issue will become apparent when
the first developers start to publish reusable open-source apps and
others start reporting bugs about incompatible CSS files. At that
point it will be clear that just putting staticfiles into Django is
not enough to guarantee that reusable apps are truly reusable (without
extra forking and maintenance effort).
Now, if my intention has become clear, these questions need to be answered:
1. How should URLs be rewritten according to the standard? Using
method (2) or (4)?
2. Should we have a standard for Sass/etc., so reusable open-source
apps can use those, too?
3. Is it a good idea to have conflicting conventions for CSS vs
Sass/etc. (method (2) vs (4), respectively) or is it better to have
just one convention for everything (method (4))?
These questions probably don't affect internally-used apps, but I
believe they are very important for the Django open-source community.
Bye,
Waldemar
Carl Meyer
On Oct 29, 3:05 am, Waldemar Kornewald <wkornew...@gmail.com> wrote:
> Just to clarify: We are in fact talking about two questions here:
> 1. Do we need a standard for URL handling?
>
> This is the most important question and my answer is "yes" (I'll
> explain this in the rest of this mail).
that tries to cover every possible use case (including Sass, SCSS,
etc). Rather than astronaut-architecting an entire hypothetical asset-
handling system for all Django users, I think we need to separate the
individual potential steps in the asset-handling toolchain and be
clear about which of those steps Django is opinionated about, and
which it is not. The only conclusion of this that seems reasonable to
me does look like your option (2); I'm fine with calling that a
standard. I don't believe this standard actually requires documenting
(as in your patch), because it's obvious: if your reusable app
provides CSS files and images, they should work out of the box as
provided, and not rely on any other third-party tool (unless, of
course, you document that reliance, which reduces the reusable appeal
of your app, but is your choice).
> > 1. Not all projects want or need to combine/compress their media at
> > all, and there is no reason they should have to run their CSS through
> > a URL-rewriting filter.
>
> When your CSS files are split into different locations via apps and
> generated via manage.py collectstatic and served via the staticfiles
> view it makes no real difference *from a tools perspective*. You
> already depend on those extra tools/filters, anyway. Your point 3
> below is about the human perspective which is the real problem here.
into a single tree, and a tool that looks inside the files and munges
their contents. The latter dramatically increases the surface area for
something to go wrong. While I personally use tools that do both, I
think it is appropriate for Django itself to provide a tool that does
the former, and strictly avoid the latter. Standardizing on your
option (4) would require adding url-munging to staticfiles, or else
requiring the use of a third-party tool; I am strongly -1 to either of
those.
> > 2. Even in projects that do combine/compress media, there are good
> > reasons to want to serve the individual files directly in development
> > (to aid in front-end debugging), and only do the combining/compressing
> > step on deployment to staging/production.
>
> django-mediagenerator and several other asset managers have a view
> which takes care of serving media uncombined and uncompressed during
> development for exactly that reason (easier debugging).
using any tool outside of Django itself (and if you use a combining/
compressing tool, you can simply disable it in development if you
want, rather than still needing it to rewrite your internal links for
you).
> If you're not yet using a professional asset manager you really try
> one. It can provide a noticeable speed boost (beyond just
> combining/compressing media) by utilizing the browser's cache
> perfectly and it can also help you during development.
forward.
> > 3. Designers and front-end developers working on Django projects
> > should not have to learn some odd way of linking in their CSS, that
> > appears to be wrong according to everything they know about how things
> > usually work. The links they write should make sense to them, not just
> > to some Python developer who invented a new "standard" for them.
>
> I see the point you're trying to make and I don't like that solution
> (4) breaks with an existing convention, either. There's still the
> other option of going with (2) for CSS and with (4) for Sass/etc. It's
> just inconsistent and this becomes even more confusing if you use some
> language with CSS-like syntax (e.g., SCSS).
anyone using Sass/SCSS simply needs to figure out how to use that tool
such that it generates working CSS.
> > 1. Links in static files should be written in such a way they work as-
> > is, without knowing what root URL they will be served under (this
> > means relative internal links).
> > 2. Any third-party tool that combines/compresses CSS should be able to
> > take working input files and generate working output files, handling
> > whatever internal links it finds correctly.
>
> described. However, without a standard it can't and in fact this very
> moment it doesn't. Depending on your asset manager you might use
> method (1) or (2) or (4) and thus use different URL paths than some
> other developer. We already have this problem right now: CSS files
> created for django-compress are incompatible with django-compressor
> and CSS files created for django-mediagenerator are incompatible with
> django-compress. Your point 2 doesn't reflect reality. It's a nice
> dream and an official standard is the first step needed for it to come
> true. That's what I'm fighting for, here.
description of the minimum that I would expect from a well-behaved
media compressor. The problem here is that neither django-compress nor
django-mediagenerator meet that standard. Django-compress takes
working CSS and, depending on your configuration and path layout, may
generate broken CSS. Django-mediagenerator requires broken CSS input
in order to generate working CSS output. I wouldn't choose to use a
tool that doesn't meet the standard of "working input; working
output." Others are free to choose as they wish, but I don't support
asking reusable apps to standardize on writing broken CSS so they can
require the presence of an asset manager to fix it for them.
> So, should we
> * use the same method (4) for CSS and Sass/etc.? (+consistency, -familiarity)
> * use method (2) for CSS and method (4) for Sass/etc.? (-consistency,
> +familiarity)
perspective:
It's a basic architectural principle that in order to build a working
large and complex system, you break it down into smaller parts, and
ensure that each part of the system takes care of its own
responsibility, and needs to know as little as possible about the
other components of the system in order to do so. This reduces
interdependence and increases modular reusability. Here are the parts
that have been touched on in this thread, and the responsibility of
each one as I see it, arranged such that each piece depends only on
knowledge of a single other piece:
1. Reusable apps should put their media in static/appname/ and expect
it to be available at STATICFILES_URL/appname/. This is implicitly
endorsed by Django due to the inclusion of staticfiles, though of
course it is not required.
2. If reusable apps include CSS that links to images or other media,
it should work as provided, given only the assumption in (1). (In
practice this means relative links, since we don't know the value of
STATICFILES_URL in advance). This is also implicitly endorsed by
Django, simply because it does not do any link-rewriting: if your
provided CSS is broken, Django doesn't fix it for you.
3. A media compressor/combiner should take working input CSS (2) and
generate working output CSS.
4. A Sass/SCSS compiler should generate working CSS (2). How it
chooses to do so, and what that means for how the Sass/SCSS itself is
written, is an internal concern of this part of the system that should
not spill over into other parts of the system.
I believe in practice this does correspond to what you are calling
"option 2 for CSS, option 4 for Sass." When I look at the
responsibilities of each part of the system independently, I can't
honestly see any other conclusion that makes sense. Parts 3 and 4 are
entirely external to Django; if an app wants to do things differently,
it's free to do so, and should document its requirements. But if
you're asking me about establishing a standard expectation for those
areas, this is what I would expect from apps that I would choose to
use.
Carl
Waldemar Kornewald
On Fri, Oct 29, 2010 at 7:41 PM, Carl Meyer <carl.j...@gmail.com> wrote:
> Hi Waldemar,
>
> On Oct 29, 3:05 am, Waldemar Kornewald <wkornew...@gmail.com> wrote:
>> Just to clarify: We are in fact talking about two questions here:
>> 1. Do we need a standard for URL handling?
>>
>> This is the most important question and my answer is "yes" (I'll
>> explain this in the rest of this mail).
>
> I don't think we need a single end-to-end, all-encompassing standard
> that tries to cover every possible use case (including Sass, SCSS,
> etc). Rather than astronaut-architecting an entire hypothetical asset-
> handling system for all Django users, I think we need to separate the
> individual potential steps in the asset-handling toolchain and be
> clear about which of those steps Django is opinionated about, and
> which it is not. The only conclusion of this that seems reasonable to
> me does look like your option (2); I'm fine with calling that a
> standard. I don't believe this standard actually requires documenting
> (as in your patch), because it's obvious: if your reusable app
> provides CSS files and images, they should work out of the box as
> provided, and not rely on any other third-party tool (unless, of
> course, you document that reliance, which reduces the reusable appeal
> of your app, but is your choice).
I believe that the standard requires documenting because, as mentioned
in a previous mail, only two of the seven combining asset managers on
djangopackages.com comply with option (2). IOW, more than 70% of the
listed asset managers use an incompatible method! So, it doesn't seem
to be as obvious as you might suggest. We won't get the desired result
without documenting and promoting this standard.
Yes, that's method 2 for CSS and method 4 for Sass. I'm fine with
using that. At least, the larger user group still only uses CSS and
it's probably more important to comply with their expectations (which
is method 2).
Like I said above, I think that at least part 3 must be actively dealt with.
Part 4 would be a nice bonus since it allows easily reusable
open-source apps with Sass/etc. files. Currently, django-compressor
uses method (2) for LESS. I'm not sure about django-static, but it
might use method (2), too (at least it does for normal CSS files). I'm
sure you agree that method (2) doesn't make sense for CSS compilers
(in particular with @import). In this case method (4) is the better
option. Let's please not ignore this problem and instead promote
method (4) as the official standard for Sass/LESS/etc. Any other
standard for relative url() expressions doesn't make sense, anyway.
What do the other core developers think about the whole standards issue?
Jacob Kaplan-Moss
<wkorn...@gmail.com> wrote:
> (see attachment which "implements" method (2) :).
OK.
For the record, I really don't want to be a dick here, but I think
what follows comes across that way. I'm quite sorry for not being able
to communicate myself in a less jerk-y way; hopefully you won't be too
offended.
There's no way I'm adding text like that to the staticfiles
documentation. Not in a million years. It's confusing to me, and
*I've* been following this discussion. Can you imagine how confusing
that's going to be to people who *haven't*? The vast majority of users
aren't going to know what the heck an "asset manager" is, what this
"standards-complient" business is, or why they should care. They're
reading this document because they want to get a CSS file up onto a
production server somewhere. That's it.
You're proposing that we hit them over the head with this need to
evaluate asset managers (whatever those are) for standards compliance
(whatever that means) and you're chastising them for not understanding
what's going on. Then you mention an "official Django standard"
without mentioning what that standard is. Worse, we've never had
"official Django standards" before, and I'm sure not going to start
with anything that high-falutin' now.
As far as I can understand from the vast reams you've written so far,
what you're basically saying is "if you use url() in a CSS file it
needs to point to an URL that actually exists." I'm pretty sure most
people are smart enough to figure that out.
At this point I'm checking out of this conversation until I see an
*actual* bug report from an *actual user. If you'd like to propose a
standard for asset managers go right ahead -- you've got a blog and
plenty of clout. If it catches on, maybe we'll add something to the
docs. Later.
Jacob
Waldemar Kornewald
> There's no way I'm adding text like that to the staticfiles
> documentation. Not in a million years. It's confusing to me, and
> *I've* been following this discussion. Can you imagine how confusing
> that's going to be to people who *haven't*? The vast majority of users
> aren't going to know what the heck an "asset manager" is, what this
> "standards-complient" business is, or why they should care. They're
> reading this document because they want to get a CSS file up onto a
> production server somewhere. That's it.
Why do you think was the patch named "ridiculous-patch.diff"? I'm
trying to communicate the *problem* because you haven't understood
that, yet. But for some reason you expect me to send the *solution*
(that's why you ask for a patch). That makes no sense at all because
you don't know what the *problem* is.
> As far as I can understand from the vast reams you've written so far,
> what you're basically saying is "if you use url() in a CSS file it
> needs to point to an URL that actually exists." I'm pretty sure most
> people are smart enough to figure that out.
No, you're looking at this from the wrong perspective. When people use
django.contrib.staticfiles then of course they will use URLs that
exist. Anything else is ridiculous. Why would anyone write URLs that
don't work? :)
But is it so difficult to understand that a lot of developers use some
*other* asset manager? More than 70% of those other asset managers
*force* you to write *broken* URLs. Most asset managers are broken!
The problem is not staticfiles, but the large number of broken
3rd-party asset managers.
Why are they broken? Because they combine your files and once your
files are combined the URLs suddenly work. IOW, those asset managers
convert *broken* URLs into *working* URLs. That's their convention and
their standard. Is this stupid? Yes. It's still reality. A very large
number of developers use this stupid broken convention.
Now what do you think, how likely is it that open-source developers
will use such a *broken* asset manager (not staticfiles) and release
*broken* CSS code that is *incompatible* with
django.contrib.staticfiles? It's far too likely for Django to ignore
this.
This is the last time I try to explain this. As you said, let's just
wait for the first users to report a bug. I hope that then you'll
realize that Django's *3rd-party* asset managers have to be fixed, not
staticfiles.
I'll do my part and at least convert django-mediagenerator to a
staticfiles-compatible URL scheme.
bur...@gmail.com
So, we agreed, it's not a problem with django, it's problem with those
3rd-party apps.
Perhaps, you can write emails to their authors now explaining your position?
Actually, it's not a problem that 70% of those apps are broken -- in
other areas, the percent of "wrong" solutions can be up to 90-100% ;)
> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
>
>
--
Waldemar Kornewald
On Sat, Oct 30, 2010 at 1:22 PM, bur...@gmail.com <bur...@gmail.com> wrote:
> Hi Waldemar,
>
> So, we agreed, it's not a problem with django, it's problem with those
> 3rd-party apps.
Yes, exactly! That's why I wanted an official standard that all
3rd-party apps can follow.
> Perhaps, you can write emails to their authors now explaining your position?
I thought about doing that, but it would be a lot easier if Django
officially promoted this as a standard. Otherwise I'm just some random
guy trying to convince them that they should fix their asset managers.
This might even mean that they have to migrate all their current CSS
code (or at least add a settings switch for the old behavior). Who
would invest this effort just because some random guy suggests it? It
would be so much easier with the Django core team leading this effort.
Mikhail Korobov
The problem was really hard to understand for me because I was
assuming you're trying to describe django.contrib.staticfiles flaw
while you were describing the problem with third-party asset managers.
I think you have described a real problem here but (as you said) this
problem is not a problem of Django itself and should be reported to
asset apps authors and users via bug tracker and blog posts, not to
django-developers list. I don't see how the incompatibility you
describe relates to staticfiles at all - it is a flaw in asset
managers and it is related to how css/html works, not to the way
django works.
Waldemar Kornewald
On Sat, Oct 30, 2010 at 1:22 PM, Mikhail Korobov <kmi...@googlemail.com> wrote:
> Hi Waldemar,
>
> The problem was really hard to understand for me because I was
> assuming you're trying to describe django.contrib.staticfiles flaw
> while you were describing the problem with third-party asset managers.
It seems like most people here had this assumption. :(
> I think you have described a real problem here but (as you said) this
> problem is not a problem of Django itself and should be reported to
> asset apps authors and users via bug tracker and blog posts, not to
> django-developers list. I don't see how the incompatibility you
> describe relates to staticfiles at all - it is a flaw in asset
> managers and it is related to how css/html works, not to the way
> django works.
I went to django-developers in order to get some official Django
backing. I think this can be seen as a Django problem: It's Django's
community that will experience the problem and it's Django's task to
take care of its own community. If we tell asset manager authors "Fix
your asset manager." they might just respond "No. Your way is wrong.
Mine is right.". But if we can point to the Django documentation and
say "Hey, you don't follow the specification." it's a whole different
story. It's not a matter of taste, anymore. It becomes a hard fact.
They can still decide to not fix their code, but then they know they
are breaking the specification.