Proposal: installmedia command - A story for distributing media with apps

47 views
Skip to first unread message

Brian Beck

unread,
Sep 16, 2008, 12:26:09 PM9/16/08
to Django developers
Distributing media with apps could be a lot easier. Currently this
requires copying or linking files manually (possibly each time the app
is updated), and this encourages developers to put CSS and JavaScript
inline in their templates. I propose a management command to make
this easier:

$ python manage.py installmedia appname [appname ...]

The command is "dumb." It goes like this:

- Read MEDIA_ROOT from settings.py.
- Look for a 'media' directory in the app.
- Copy or link everything in there to MEDIA_ROOT.

This simple command saves you quite a bit of typing. Some
possibilities...

- Switches like --copy, --link, --overwrite, etc.
- Just like with templates, it could become best practice for apps to
have their media structured like so:
media/
media/appname/
media/appname/css/
...
- Or in the simple case:
media/
media/css/appname.css
media/css/appname.js

This should make it easier for apps to make assumptions about where
their media is located relative to MEDIA_URL (this is currently ad
hoc). Thoughts?

Rajeev J Sebastian

unread,
Sep 16, 2008, 2:37:46 PM9/16/08
to django-d...@googlegroups.com
On Tue, Sep 16, 2008 at 9:56 PM, Brian Beck <exo...@gmail.com> wrote:
>
> Distributing media with apps could be a lot easier. Currently this
> requires copying or linking files manually (possibly each time the app
> is updated), and this encourages developers to put CSS and JavaScript
> inline in their templates. I propose a management command to make
> this easier:
>
> $ python manage.py installmedia appname [appname ...]

I have already implemented this in our in-house django-based
framework. If interested, I can provide it. As you said, it really
does make things very simple.

Regards
Rajeev J Sebastian

Erik Allik

unread,
Sep 16, 2008, 2:45:50 PM9/16/08
to django-d...@googlegroups.com
> I have already implemented this in our in-house django-based
> framework. If interested, I can provide it. As you said, it really
> does make things very simple.
>

Please do, I'd be interested even if it doesn't make it to the trunk.

Erik

Ivan Sagalaev

unread,
Sep 16, 2008, 4:42:42 PM9/16/08
to django-d...@googlegroups.com
Brian Beck wrote:
> Distributing media with apps could be a lot easier. Currently this
> requires copying or linking files manually (possibly each time the app
> is updated), and this encourages developers to put CSS and JavaScript
> inline in their templates. I propose a management command to make
> this easier:
>
> $ python manage.py installmedia appname [appname ...]

Oh, yes! I was trying to invent some way to distribute media (apart from
documenting copying it in a README) but with no luck. Yet your solution
is so simple and beautiful!

Don Spaulding

unread,
Sep 16, 2008, 5:09:06 PM9/16/08
to django-d...@googlegroups.com
On Tue, Sep 16, 2008 at 11:26 AM, Brian Beck <exo...@gmail.com> wrote:

$ python manage.py installmedia appname [appname ...]

The command is "dumb."......

Could it be a little smarter, and in the absence of specific apps, iterate over all INSTALLED_APPS, and install their media automagically?


Julien Phalip

unread,
Sep 16, 2008, 6:29:45 PM9/16/08
to Django developers
+1 for some sort of media.autodiscover().
It would have to be optional though, as that might break backwards-
compatibility.

Also, there's an issue about the way files are served. For example, in
my projects I make Apache serve 'admin_media' files independently.
What would be an approach to make that serving configuration happen
for custom apps?

Brian Beck

unread,
Sep 16, 2008, 6:45:29 PM9/16/08
to Django developers
On Sep 16, 6:29 pm, Julien Phalip <jpha...@gmail.com> wrote:
> > Could it be a little smarter, and in the absence of specific apps, iterate
> > over all INSTALLED_APPS, and install their media automagically?
>
> +1 for some sort of media.autodiscover().
> It would have to be optional though, as that might break backwards-
> compatibility.

I think he meant making "python manage.py installmedia" (no app names
provided) install media for all INSTALLED_APPS, not something that
will happen without running any commands -- so no backwards-
compatibility will be affected. And I agree with Don's idea.

> Also, there's an issue about the way files are served. For example, in
> my projects I make Apache serve 'admin_media' files independently.
> What would be an approach to make that serving configuration happen
> for custom apps?

Serving is totally orthogonal -- everyone is already serving up
MEDIA_ROOT in their projects somehow anyway, and this just copies
files to MEDIA_ROOT.

Brian Beck

unread,
Sep 16, 2008, 6:49:59 PM9/16/08
to Django developers
On Sep 16, 6:45 pm, Brian Beck <exo...@gmail.com> wrote:
> Serving is totally orthogonal -- everyone is already serving up
> MEDIA_ROOT in their projects somehow anyway, and this just copies
> files to MEDIA_ROOT.

Sorry, I guess that's not totally true -- everyone who uses more than
just the admin app has set up static file serving for MEDIA_ROOT.

Julien Phalip

unread,
Sep 16, 2008, 7:22:18 PM9/16/08
to Django developers
I really like the idea. However, I think there should also be a way to
configure it to not copy to MEDIA_ROOT but to somewhere else.

It would also be good to get that deployment system play nicely with
custom file storage (if you're hosting your media on S3 for example).

Also, instead of just copying the media files, it should also do some
cleanup. Say, if you're tracking trunk for a given app, when you SVN
update that app you want stale media files to be removed.

Just some thoughts.

Brian Beck

unread,
Sep 16, 2008, 7:31:41 PM9/16/08
to Django developers
On Sep 16, 7:22 pm, Julien Phalip <jpha...@gmail.com> wrote:
> I really like the idea. However, I think there should also be a way to
> configure it to not copy to MEDIA_ROOT but to somewhere else.

Sure - a --destination flag, defaulting to MEDIA_ROOT, would work.

> Also, instead of just copying the media files, it should also do some
> cleanup. Say, if you're tracking trunk for a given app, when you SVN
> update that app you want stale media files to be removed.

This is getting into package management system territory ;) -- this
problem is mostly solved by linking a directory instead of copying all
the files, which I think should be the default behavior (linking, not
copying).

Justin Lilly

unread,
Sep 16, 2008, 7:33:00 PM9/16/08
to django-d...@googlegroups.com
Not sure if you guys are aware, but this seems a very likely candidate
for django-exensions which extends manage.py like functionality.

http://code.google.com/p/django-command-extensions/

-justin

--
Justin Lilly
Web Developer/Designer
http://justinlilly.com

sciyoshi

unread,
Sep 16, 2008, 10:17:07 PM9/16/08
to Django developers
On Sep 16, 7:31 pm, Brian Beck <exo...@gmail.com> wrote:
> On Sep 16, 7:22 pm, Julien Phalip <jpha...@gmail.com> wrote:
>
> > I really like the idea. However, I think there should also be a way to
> > configure it to not copy to MEDIA_ROOT but to somewhere else.
>
> Sure - a --destination flag, defaulting to MEDIA_ROOT, would work.

There's also the issue of the application knowing where its media will
be installed,
so that it can access it from templates. Let's say I just did

manage.py installmedia myapp

and all of myapp's media got copied into MEDIA_ROOT/myapp/{css,js}/.
Should I assume
in my templates that the files are available at {{ MEDIA_URL }}/myapp/
{css,js}/? Or maybe
there should be a settings variable, something like

APP_MEDIA_URLS = {
'myapp': 'some/other/path',
...
}

so that the app can use {{ APP_MEDIA_URLS.myapp }}/myapp/{css,js}/.
This doesn't seem like
a great solution though, because there's no real "standard" for which
{% blocks %} to override in
an app to add css/js; maybe this configuration step should just be
left to the person using the app.

Rajeev J Sebastian

unread,
Sep 16, 2008, 10:49:10 PM9/16/08
to django-d...@googlegroups.com
On Wed, Sep 17, 2008 at 7:47 AM, sciyoshi <sciy...@gmail.com> wrote:
>
> On Sep 16, 7:31 pm, Brian Beck <exo...@gmail.com> wrote:
>> On Sep 16, 7:22 pm, Julien Phalip <jpha...@gmail.com> wrote:
>>
>> > I really like the idea. However, I think there should also be a way to
>> > configure it to not copy to MEDIA_ROOT but to somewhere else.
>>
>> Sure - a --destination flag, defaulting to MEDIA_ROOT, would work.
>
> There's also the issue of the application knowing where its media will
> be installed,
> so that it can access it from templates. Let's say I just did

This is not really a problem, since the media will always be under
<MEDIA_ROOT>/<appname>/..

A problem would be referencing images, and media in css/js files.
Currently, in all our projects we use /site_media/ as the MEDIA_URL,
so this not a problem. But for a true solution, there should be a
preprocessor for css/js to use the MEDIA_URL within it.

Another problem which we havent solved, is overrides. E.g., when we
use a "plugabble"/reusable app with its own media, and we want to
override this in our project, how should the collect media command
work ?

Anyway, I've attached our media app (with the command, etc). If you
want any changes to be included, please do tell me.

Regards
Rajeev J Sebastian

dynamo_media.tar.bz2

Brian Beck

unread,
Sep 16, 2008, 11:05:37 PM9/16/08
to Django developers
On Sep 16, 10:49 pm, "Rajeev J Sebastian" <rajeev.sebast...@gmail.com>
wrote:
> A problem would be referencing images, and media in css/js files.
> Currently, in all our projects we use /site_media/ as the MEDIA_URL,
> so this not a problem. But for a true solution, there should be a
> preprocessor for css/js to use the MEDIA_URL within it.

CSS can refer to images by their path relative to the CSS file, so if
your media tree looks like:

media/css/base.css
media/images/icon.png

Then base.css can do: background-image('../images/icon.png')

...so no preprocessing needed there. But referring to *another app's*
images is another story. And JavaScript... well, don't reference
images in JavaScript.

> Another problem which we havent solved, is overrides. E.g., when we
> use a "plugabble"/reusable app with its own media, and we want to
> override this in our project, how should the collect media command
> work ?

I imagined this working just like templates. By *convention* apps can
lay out their media directories like media/appname. So there's
nothing stopping your app from including, say, media/anotherapp/css/
base.css, and we can say that whichever app is in INSTALLED_APPS first
wins, just like with templates.

Samuel Cormier-Iijima

unread,
Sep 17, 2008, 1:20:22 AM9/17/08
to Django developers
On Sep 16, 11:05 pm, Brian Beck <exo...@gmail.com> wrote:
> On Sep 16, 10:49 pm, "Rajeev J Sebastian" <rajeev.sebast...@gmail.com>
> wrote:
>
> > A problem would be referencing images, and media in css/js files.
> > Currently, in all our projects we use /site_media/ as the MEDIA_URL,
> > so this not a problem. But for a true solution, there should be a
> > preprocessor for css/js to use the MEDIA_URL within it.
>
> CSS can refer to images by their path relative to the CSS file, so if
> your media tree looks like:
>
> media/css/base.css
> media/images/icon.png
>
> Then base.css can do: background-image('../images/icon.png')
>
> ...so no preprocessing needed there.  But referring to *another app's*
> images is another story.  And JavaScript... well, don't reference
> images in JavaScript.

We use the convention that there should be a Javascript variable
called MEDIA_URL defined somewhere in a base template that lets
Javascript access it; seems like this might be a useful convention to
adopt.

> > Another problem which we havent solved, is overrides. E.g., when we
> > use a "plugabble"/reusable app with its own media, and we want to
> > override this in our project, how should the collect media command
> > work ?
>
> I imagined this working just like templates.  By *convention* apps can
> lay out their media directories like media/appname.  So there's
> nothing stopping your app from including, say, media/anotherapp/css/
> base.css, and we can say that whichever app is in INSTALLED_APPS first
> wins, just like with templates.

For CSS, there's a couple ways that this might possible. Let's say
myapp wants to distribute some default CSS; then it could actually
provide two separate files:

myapp/media/myapp/css/style_base.css:

#myapp div.something {
/* custom styles for the app */
}

myapp/media/myapp/css/style.css:

@import url(./style_base.css);

This way, the project using the app can customize the CSS or replace/
exclude it completely (by excluding the import line), without having
to change anything inside myapp proper. django-compress might be
useful in this situation; it could automatically parse the @imports in
CSS and generate a compressed version including the right files.

I'm not sure though that all of this is worth it; it seems like we're
trying to optimize for the case where somebody a user simply drops an
app into their project, runs installmedia, and never customizes
anything beyond that. In reality, the templates and CSS are most
likely going to be customized (although JS is another story), so maybe
this sort of "inheritance" isn't really needed.

Which reminds me - I think it would be really nice if there was some
sort of script/style manager in Django (similar to the django.forms
media):

{% extends 'base.html' %}

{% css 'myapp/css/stuff.css' 'myapp/css/morestuff.css' %}
{% js 'myapp/js/stuff.js' %}

{% block content %}
...
{% endblock %}

and then in the base template

<html>
<head>
{% css %}
{% js %}
</head>
<body>
{% block content %}{% endblock %}
</body>
</html>

Django could then even check in a list of directories to find the
appropriate files (similar to the way template lookups work now) and
include the first one found.

Brian Beck

unread,
Sep 19, 2008, 4:26:27 PM9/19/08
to Django developers
I noticed that elwaywitvac just posted this management command here:
http://www.djangosnippets.org/snippets/1066/

I also have an implementation that I'll post when I get home.

Brian Beck

unread,
Sep 19, 2008, 9:19:36 PM9/19/08
to Django developers
On Sep 19, 4:26 pm, Brian Beck <exo...@gmail.com> wrote:
> I also have an implementation that I'll post when I get home.

I just posted my collectmedia (I liked the name Rajeev used) command
here: http://www.djangosnippets.org/snippets/1068/

It's a long snippet because it includes an interactive mode and sets
permissions (see below). For testing, I recommend running with -i and
-n (interactive mode and dry run mode, respectively). I've tested it,
but you should probably back up your media first.

Features:

If multiple apps provide a media file of the same name, use the file
provided by the app listed first in INSTALLED_APPS - this mimics the
template loader behavior. In interactive mode (-i), you may specify
which app to select from for each such file. With this command, best
practice would be to put media files at app/media/appname/... - just
like templates.

It attempts to be "smart" about permissions by using MEDIA_ROOT's
permissions and ownership on all the files it creates. So, if
MEDIA_ROOT is owned by apache/www-data for instance, the media files
should, too.

You may provide a directory other than MEDIA_ROOT to copy to. You may
also use symbolic links instead of copying, but this doesn't work on
Windows, so copying is the default behavior. See the code for the
other options...

Thoughts?

Erik Allik

unread,
Sep 20, 2008, 6:58:17 AM9/20/08
to django-d...@googlegroups.com
> If multiple apps provide a media file of the same name, use the file
> provided by the app listed first in INSTALLED_APPS - this mimics the
> template loader behavior. In interactive mode (-i), you may specify
> which app to select from for each such file. With this command, best
> practice would be to put media files at app/media/appname/... - just
> like templates.

Has anyone got some good use cases where the template-loaded mimicking
behavior would be desired? Otherwise it's just needless complexity in
my opinion.

I love the whole installmedia command idea though.

Erik

Brian Beck

unread,
Sep 20, 2008, 9:12:51 AM9/20/08
to Django developers
On Sep 20, 6:58 am, Erik Allik <race...@gmail.com> wrote:
> Has anyone got some good use cases where the template-loaded mimicking  
> behavior would be desired? Otherwise it's just needless complexity in  
> my opinion.

Well, first of all, it's not really complexity at all. If it were to
just copy all the media files found, the last app in INSTALLED_APPS
would overwrite the others with the same name. This way, the first
app in INSTALLED_APPS wins instead of the last one. That's the only
difference.

And the use case is of course to customize the style of a reusable app
without modifying the actual files distributed with the app (which
would make it a pain when upgrading). batchadmin[1], for example --
if you want to change the style of the batch action form, like to make
it pink, just provide a file in your app named media/css/
batchadmin.css, just like you would provide a template at templates/
batchadmin/actions.html if you wanted to change the markup.

[1]: http://code.google.com/p/django-batchadmin/

Erik Allik

unread,
Sep 20, 2008, 9:20:00 AM9/20/08
to django-d...@googlegroups.com
I see your point about overriding reusable app media.

I got an idea though when reading your post. Since people want to
update reusable apps to a more recent version/revision and since that
means the media files will/might change, why not add another command
that would show which installed media files differ from the default
ones provided by apps? That would show both which files have been
overridden and which files have been changed by recent updates. I
guess if you have means to compare file trees already, it would be a
simple thing to add. The command could be named "comparemedia" or
"mediastatus" or similar.

Erik

Travis Cline

unread,
Sep 22, 2008, 10:41:55 AM9/22/08
to Django developers
On Sep 20, 8:20 am, Erik Allik <race...@gmail.com> wrote:
> I see your point about overriding reusable app media.
>
> I got an idea though when reading your post. Since people want to
> update reusable apps to a more recent version/revision and since that
> means the media files will/might change, why not add another command
> that would show which installed media files differ from the default
> ones provided by apps? That would show both which files have been
> overridden and which files have been changed by recent updates. I
> guess if you have means to compare file trees already, it would be a
> simple thing to add. The command could be named "comparemedia" or
> "mediastatus" or similar.
>
> Erik

Defaulting to symlinks solves this on sane platforms. Regardless,
diff already exists, comparemedia/mediastatus are just added
complexity.

Travis
Reply all
Reply to author
Forward
0 new messages