At the end of the angsty autoescaping thread (said angst in no small part from my direction ^_^), I floated the idea that perhaps my concern over autoescaping was pointed in the wrong direction — that it might be time to split Django's template module out into a standalone library. The idea has been raised before, but discussion always hits the point of "good idea, but a PITA ..." and a combination of time constraints and the bystander effect kick in to shelve it.
Since I've been championing the use of Django's template library as a general purpose library, and spending plenty of time in the guts of the template code while writing the taghelpers module, I may as well step up and offer to do this. And no, I don't want another Jinja [1] [2]; I want a Django template library that's still distributed as part of Django, in the Django namespace, and available in its separate split-off form from the Django website, similar to how the various Twisted modules are distributed [3].
I know there are various obstacles here, the largest one being the use of Django settings in the template code — but nothing looks unsurmountable.
I'd ideally like to have this done in time for Django 1.0, and to ensure stable releases in line with future Django releases.
On Tue, 2007-08-07 at 17:39 -0500, Tom Tobin wrote: > At the end of the angsty autoescaping thread (said angst in no small > part from my direction ^_^), I floated the idea that perhaps my > concern over autoescaping was pointed in the wrong direction -- that it > might be time to split Django's template module out into a standalone > library. The idea has been raised before, but discussion always hits > the point of "good idea, but a PITA ..." and a combination of time > constraints and the bystander effect kick in to shelve it.
That isn't my recollection, and I was involved in some of those previous discussions. The main argument is that it isn't a really good idea, for reasons outlined below. You don't address why we would want to do this.
What's the really compelling use-case that isn't already handled by installing Django and just using the templating code with standalone configure (via settings.configure())? Package size isn't the argument -- Django is relatively small and you don't gain that much by isolating the templates, once you include all the supporting stuff, since the bulk of the disk usage of Django is in the PO files.
> Since I've been championing the use of Django's template library as a > general purpose library, and spending plenty of time in the guts of > the template code while writing the taghelpers module, I may as well > step up and offer to do this. And no, I don't want another Jinja [1] > [2]; I want a Django template library that's still distributed as part > of Django, in the Django namespace, and available in its separate > split-off form from the Django website, similar to how the various > Twisted modules are distributed [3].
> I know there are various obstacles here, the largest one being the use > of Django settings in the template code -- but nothing looks > unsurmountable.
The main reason we implemented standalone configuration (via settings.configure()) was so that you could use the template library in isolation. At the time (starting about 18 months ago), we had a bit of a debate about where to include the defaults (in django.conf or django.templates, etc) and decided the current approach wasn't really hindering anything and meant there was only one spot you had to look for things.
> I'd ideally like to have this done in time for Django 1.0, and to > ensure stable releases in line with future Django releases.
> So ... thoughts?
Please, no.
The main problems are
(1) It's not really worth it, since installing all of Django and then just not using part of it is easy -- it's not a huge package after all.
Experience has shown that the Twisted split hasn't been a no-brainer success. By and large, the easiest thing to do, even with Twisted, is install everything and just use the bits you need. Otherwise you spend ages wallowing around trying to work out which bits you do need. When it comes to packaging for distros, Twisted either has to be done as one mega-package or something like TurboGears or Pylons as a combination of a bunch of individual packages (usually the latter). That latter situation is a packaging strategy Django intentionally avoids and it will make downstream packaging after 1.0 much easier. People just have to install django-1.0.
(2) It leads to problems with debugging, installing, setting up and helping people when they have both the separate library and the whole Django package installed. Which template portion is being used, etc? Are they up-to-date? Are they interfering with each other? If everything's in one package, then we only have to worry about the people who have multiple Django versions installed and that's much easier to debug.
(3) If the goal is that django-core will use django-templates, we are going down the Pylons path. Batteries included is a good philosophy for something the size of Django, though. If the goal is that django-core will duplicate django-templates, you have problems (1) and (2), above.
> Since I've been championing the use of Django's template library as a > general purpose library, and spending plenty of time in the guts of > the template code while writing the taghelpers module, I may as well > step up and offer to do this.
This would be great, as long as you could figure out how to make it backwards compatible. Thanks for volunteering! :-)
Here are some broad goals/thoughts, IMHO:
* Users of Django the framework should not be affected in any way.
* Ideally we should not have to maintain two versions of the codebase.
* Figuring out the settings stuff could be tricky.
* Consider the case of people who download and use the standalone template system, then decide to install the whole framework. It would be nice (but not essential) if they didn't have to have conflicting versions of the template library installed on their systems. I don't know anything about the way Twisted modules are distributed (which is what you alluded to) -- do they have a nice way of solving this problem?
Adrian
-- Adrian Holovaty holovaty.com | djangoproject.com
On 8/7/07, Malcolm Tredinnick <malc...@pointy-stick.com> wrote:
> I want to hear the use-case first. I may change my mind, but nobody's > come up with a good reason so far.
I think Tom's primary goal is to provide an easy path to use the Django template system in non-HTML contexts without having to worry about autoescaping (since the standalone library could leave it off or configurable at a higher level while Django as a whole could force autoescaping on), and I can see that looming large; there are already a number of interesting uses for the template system that don't involve HTML at all :)
-- "Bureaucrat Conrad, you are technically correct -- the best kind of correct."
On Tue, 2007-08-07 at 19:03 -0500, James Bennett wrote: > On 8/7/07, Malcolm Tredinnick <malc...@pointy-stick.com> wrote: > > I want to hear the use-case first. I may change my mind, but nobody's > > come up with a good reason so far.
> I think Tom's primary goal is to provide an easy path to use the > Django template system in non-HTML contexts without having to worry > about autoescaping (since the standalone library could leave it off or > configurable at a higher level while Django as a whole could force > autoescaping on), and I can see that looming large; there are already > a number of interesting uses for the template system that don't > involve HTML at all :)
I want to hear a *good* reason. Having to use or not use a particular single tag, once, isn't a good reason for all that extra work and maintenance effort. If this is all coming out of the autoescape conversation then.. well... it's disappointing, to say the least.
Malcolm Tredinnick wrote: > I want to hear the use-case first. I may change my mind, but nobody's > come up with a good reason so far.
How about looking at things from the opposite direction? Yes, it would be nice if the Django template system could be more easily used standalone, but by doing this it would also most probably allow Django to plug in other template engines more easily.
This feature could be a deciding factor for folks who would use Django "if only I could use my favorite XML-based templating system" or those switching from another framework with existing templates that they wouldn't want to totally rewrite.
Also, food for thought, it seems that the "merger" of TurboGears and Pylons [1][2][3] sparked the creation of the pytemplates google group [4], where interested parties have been talking about creating a standard API with plugins for various template engines.
On Tue, 2007-08-07 at 23:17 -0500, Gary Wilson wrote: > Malcolm Tredinnick wrote: > > I want to hear the use-case first. I may change my mind, but nobody's > > come up with a good reason so far.
> How about looking at things from the opposite direction? Yes, it would > be nice if the Django template system could be more easily used > standalone, but by doing this it would also most probably allow Django > to plug in other template engines more easily.
Again, this needs an answer to the question of what does "more easily" mean. Currently, you import django.template, django.conf and make sure you call django.conf.configure(...) prior to using the template stuff. There's limited scope for streamlining there.
Maybe adding something like django.conf.default_do_nothing_template_setup() reduces the complexity and this was something we discussed as a possible enhancement when django.conf.configure() was implemented. However, it won't be one size fits all (if you ever want to use template loaders -- via extends or include tags -- some configuration is required). It could live in the django.template namespace so that only django.template needed to be imported. Aside from that, what else comes wrapped up in this new "easier" system.. I'm not trying to be difficult here; I genuinely don't understand these perceived benefits and it's not for want of using the system or thinking about the problem space.
> This feature could be a deciding factor for folks who would use Django > "if only I could use my favorite XML-based templating system" or those > switching from another framework with existing templates that they > wouldn't want to totally rewrite.
Unconvinced.
The sort of questions I would ask (and answer) are:
Can Django's templating system be used standalone at the moment without difficulty? Yes.
Can other templating systems already be used with Django without needing to write lots of extra code? Yes.
We added the necessary features (configuration without requiring DJANGO_SETTINGS_MODULE)for isolated template usage 18 months ago. What needs to be made easier? Why can't whatever those changes are be done without splitting?
How to make plugging in separate template engines easier is also interesting, but, again, not particularly contingent on a split. It's already possible and not particularly difficult: writing an appropriate render_to_response makes things easier for some people, but by and large, it's pretty much the same code you write now. When I did this for a client late last year (using Cheetah throughout), the only real mod I needed to make was a way to get a snapshot of the current Context's top-layer of dictionary keys and values so that I could pass a real dictionary around. Replacing the Context class with my own variant at runtime fixed that. Including the method in core would help, though.
So the focus is on whether we need to make this easier and how. What needs to be done there?
I really worry that we shouldn't be trying to reinvent Turbogears here. If somebody wants plug-and-change-their-mind components, it's already spelt "Turbogears" or "Pylons". Adding the necessary stuff to integrate Django templates with Turbogears can't be that difficult (since it can already be used standalone).
Please let's make sure we're solving worthwhile problems and trying to solve the same problems. None of the stuff you've mentioned, Garry, requires splitting templates off from Django. They're worthwhile things to do on some level, but we can pretty much already do most of what is needed, in practice.
It comes back to my original question for me: what can't be done now or with a couple of small tweaks that can be done after the much more disruptive change of splitting out the template system?
On 8/7/07, Malcolm Tredinnick <malc...@pointy-stick.com> wrote:
> On Tue, 2007-08-07 at 17:39 -0500, Tom Tobin wrote: > > At the end of the angsty autoescaping thread (said angst in no small > > part from my direction ^_^), I floated the idea that perhaps my > > concern over autoescaping was pointed in the wrong direction -- that it > > might be time to split Django's template module out into a standalone > > library. The idea has been raised before, but discussion always hits > > the point of "good idea, but a PITA ..." and a combination of time > > constraints and the bystander effect kick in to shelve it.
> That isn't my recollection, and I was involved in some of those previous > discussions. The main argument is that it isn't a really good idea, for > reasons outlined below. You don't address why we would want to do this.
Err, to make it more straightforward for others to use *just* the templating system?
> What's the really compelling use-case that isn't already handled by > installing Django and just using the templating code with standalone > configure (via settings.configure())? Package size isn't the argument -- > Django is relatively small and you don't gain that much by isolating the > templates, once you include all the supporting stuff, since the bulk of > the disk usage of Django is in the PO files.
You're right, package size *isn't* the argument; when it comes down to it, many people don't like installing stuff they have zero intention of using.
> > Since I've been championing the use of Django's template library as a > > general purpose library, and spending plenty of time in the guts of > > the template code while writing the taghelpers module, I may as well > > step up and offer to do this. And no, I don't want another Jinja [1] > > [2]; I want a Django template library that's still distributed as part > > of Django, in the Django namespace, and available in its separate > > split-off form from the Django website, similar to how the various > > Twisted modules are distributed [3].
> > I know there are various obstacles here, the largest one being the use > > of Django settings in the template code -- but nothing looks > > unsurmountable.
> The main reason we implemented standalone configuration (via > settings.configure()) was so that you could use the template library in > isolation. At the time (starting about 18 months ago), we had a bit of a > debate about where to include the defaults (in django.conf or > django.templates, etc) and decided the current approach wasn't really > hindering anything and meant there was only one spot you had to look for > things.
That doesn't mean we can't re-approach the idea, assuming it was done well.
> > I'd ideally like to have this done in time for Django 1.0, and to > > ensure stable releases in line with future Django releases.
> > So ... thoughts?
> Please, no.
> The main problems are
> (1) It's not really worth it, since installing all of Django and then > just not using part of it is easy -- it's not a huge package after all.
> Experience has shown that the Twisted split hasn't been a no-brainer > success. By and large, the easiest thing to do, even with Twisted, is > install everything and just use the bits you need. Otherwise you spend > ages wallowing around trying to work out which bits you do need. When it > comes to packaging for distros, Twisted either has to be done as one > mega-package or something like TurboGears or Pylons as a combination of > a bunch of individual packages (usually the latter). That latter > situation is a packaging strategy Django intentionally avoids and it > will make downstream packaging after 1.0 much easier. People just have > to install django-1.0.
People will *still* be able to just install django-1.0. Perhaps I didn't explain the packaging situation quite as clearly as I could have; the Django templating system would *still* be distributed as part of Django, in the django.template namespace. No separate package required. Batteries included. The Twisted analogy was meant to draw a parallel more with the namespacing than enforced separation (and even for Twisted, there's always the "Sumo" package).
> (2) It leads to problems with debugging, installing, setting up and > helping people when they have both the separate library and the whole > Django package installed. Which template portion is being used, etc? Are > they up-to-date? Are they interfering with each other? If everything's > in one package, then we only have to worry about the people who have > multiple Django versions installed and that's much easier to debug.
See point above.
> (3) If the goal is that django-core will use django-templates, we are > going down the Pylons path. Batteries included is a good philosophy for > something the size of Django, though. If the goal is that django-core > will duplicate django-templates, you have problems (1) and (2), above.
Partially see point above; I intend that the Django template system continue to be distributed alongside Django, and for the Django packaging system to pull in the Django template library. No duplication is involved; there would be only one place for django-template code development.
