GSoC 2007: Reusable Django applications

1 view
Skip to first unread message

Jannis Leidel

unread,
Aug 23, 2007, 9:22:10 PM8/23/07
to Django developers
Hi,

After the official end on monday and two weeks without a real status
update on my GSoC project I'd like to share some thoughts and ideas
that came up during the summer. As a spoiler (or if you don't feel
like reading the big chunk of text below): I have not completed all of
my original project goals but will continue to work.

My initial GSoC application [1] was divided in:

1) Create code infrastructure to simplify the process of building
reusable Django applications, e.g. with egg files.
2) Build a public repository website for reusable Django applications.

Even if these two topics are connected in a way, I stumbled over the
rather different approaches to package management on a programming
language, operating system and community level. I evaluated the
package formats I knew to learn more about common package management
idioms (apt, rpm, gems, distutils, setuptools) and realized that
reinventing the wheel should be avoided - especially for a specific
product like Django; reusable Django apps should happen with something
stable and widely known, preferable without having to tie users to a
"simple", wrapped version of a sophisticated system.

So, giving the user the option to have an easy start with package
management was my first actual goal. I looked at the standard
distutils and the defacto standard setuptools and found the features I
needed: package management, dependency tracking, uploading and an easy-
to-distribute format - eggs. Hacking the setuptools code and cloning
the cheeseshop (PyPI) wasn't an option of course, so instead I
introduced a "release" metaphor (thanks for the idea Neil Blakey-
Milner) which holds the basic metadata for a reusable Django
application. It's basically a simple Python file which lives in the
application directory, gets loaded by setup.py and can be hand-edited
or even abandoned if not needed.

To facilitate the preparation of apps I added hooks to
dango.core.management (the refactored code too) to ask for the
metadata with "startapp" and made it the default (--noskeleton for the
current behaviour). It also creates skeleton files, builds a MANIFEST
file and gives the option to edit the release information later
("editapp"). The prepared apps can be uploaded to the PyPI or use
virtually any other function of setuptools. [2] (django.utils.package)

In the second part of my project I needed to bridge the gap between
the reusable Django apps and a to-be repository. Since I already used
setuptools (and PyPI) to create application packages, I decided to add
a keyword to any app (e.g. "django.apps") which later can be used to
identify the packages during a PyPI scan. With a little inspiration by
Jacob's project cheeserater.com I got this scan working in the days
before final evaluation. [3]

So now I'm on building the rest of the repository website, using
several of the django-* Google Code projects. If you have any ideas
for it, feel free to contact me. What about djangoapps.org?

James, thanks for your support and constant patience, I really learned
a lot this summer about programming and this crazy open source thing.
Seriously, you got one contributer more.

All the others, if you find the time, please have a look at my code
[2] and tell me if this all makes sense. My weblog [4] has some more
information about my summer of code, too.

Best
Jannis


1: http://jannisleidel.com/2007/03/gsoc-application/
2: http://code.google.com/p/django-package/
3: http://websushi.org/trac/browser/django-apps/
4: http://jannisleidel.com

Etienne Robillard

unread,
Aug 27, 2007, 10:13:17 AM8/27/07
to django-d...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Jannis,

Thanks for your time writing this post. :)

I like this idea of having reusable apps for Django
all in the same place. Here's some suggestions.

> My initial GSoC application [1] was divided in:
>
> 1) Create code infrastructure to simplify the process of building
> reusable Django applications, e.g. with egg files.
> 2) Build a public repository website for reusable Django applications.

Egg files are great. We could easy_install them, assuming the user
or developer is already using setuptools.


> Even if these two topics are connected in a way, I stumbled over the
> rather different approaches to package management on a programming
> language, operating system and community level. I evaluated the
> package formats I knew to learn more about common package management
> idioms (apt, rpm, gems, distutils, setuptools) and realized that
> reinventing the wheel should be avoided - especially for a specific
> product like Django; reusable Django apps should happen with something
> stable and widely known, preferable without having to tie users to a
> "simple", wrapped version of a sophisticated system.

Agreed. In order to distribute reusable django apps, there should be
at least a common package namespace where all theses packages would get
installed. I've thought the namespace "apps" would be a good
starting point for this.

> So, giving the user the option to have an easy start with package
> management was my first actual goal. I looked at the standard
> distutils and the defacto standard setuptools and found the features I
> needed: package management, dependency tracking, uploading and an easy-
> to-distribute format - eggs. Hacking the setuptools code and cloning
> the cheeseshop (PyPI) wasn't an option of course, so instead I
> introduced a "release" metaphor (thanks for the idea Neil Blakey-
> Milner) which holds the basic metadata for a reusable Django
> application. It's basically a simple Python file which lives in the
> application directory, gets loaded by setup.py and can be hand-edited
> or even abandoned if not needed.

Again, I really like the idea to use setuptools and eggs for distribution
of django apps/packages. However, Im less sure about adding anything
special inside a app for specifying metadata. Why not just use a regular
setup.py script for distributing a standalone app ?

> So now I'm on building the rest of the repository website, using
> several of the django-* Google Code projects. If you have any ideas
> for it, feel free to contact me. What about djangoapps.org?

+1 for djangoapps.org and for using a common package namespace
for reusable django apps. [apps]

Also, the freebsd ports collection is a great example of a well-managed
collection of reusable apps. We should use this more for inspiration.. :)

Finally, please check out notmm. Its a Django toolkit containing various
packages, and a set of django apps (distributed as a standalone component, but included
in the standard distribution). [1]

1. svn co http://tools.assembla.com/svn/notmm/trunk/ notmm

Cheers,

- - Etienne
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.5 (FreeBSD)

iEYEARECAAYFAkbS2/wACgkQdXKAffkXj4OeTgCfSoiattnF4JoHChUvLqYoP4c7
QrgAoLDZcClKCDj83MEXz+Ngo0gL/PhS
=kk87
-----END PGP SIGNATURE-----

Jannis Leidel

unread,
Sep 14, 2007, 2:05:06 PM9/14/07
to django-d...@googlegroups.com
Hey Etienne,
Sorry for this late answer, I'm now back at the game..

>
> Egg files are great. We could easy_install them, assuming the user
> or developer is already using setuptools.

Indeed - eggs will be in some way included in the next Python
versions, if it is setuptools or as part of distutils. The fact
remains, having a distributable format is the way to go if we don't
want to collapse of the diversity of Django apps.

> Agreed. In order to distribute reusable django apps, there should be
> at least a common package namespace where all theses packages would
> get
> installed. I've thought the namespace "apps" would be a good
> starting point for this.

Definitely, "apps" is semantically and technically correct.

> Again, I really like the idea to use setuptools and eggs for
> distribution
> of django apps/packages. However, Im less sure about adding anything
> special inside a app for specifying metadata. Why not just use a
> regular
> setup.py script for distributing a standalone app ?

Well, good point. One answer is the easy manipulation of the metadata
by the user. Changing the setup.py could lead to unsuccessful
packaging. That's why I moved it to the new metaphorical construct of
a "release.py" within the application directory. The other reason is
the the "Release" API I implemented in django.utils.package which
gives the user the chance to manipulate a application release
directly. Think of a Trac plugin or a Paste wrapper for example,
using this information.

> +1 for djangoapps.org and for using a common package namespace
> for reusable django apps. [apps]
>
> Also, the freebsd ports collection is a great example of a well-
> managed
> collection of reusable apps. We should use this more for
> inspiration.. :)

Good idea, thanks for the tip!

Jannis

Reply all
Reply to author
Forward
0 new messages