Pinax template loader

[Rock]

I just created a new pinax template loader that allows for the
creation of reusable apps that work "out of the box" in either
standalone mode or within pinax. Here is how it works.

The loader is named pinax_app_directories.py. (A copy of this file
should be included in basic_project.py and complete_project.py so that
the pinax user copies it into their projects.) The template loader is
adjusted to invoke this loader after "filesystem" but before
"app_directories".

The new pinax loader works just like the "app_directories" loader but
looks for templates in each installed app within "templates/pinax"
instead of "templates". (Of course if there is no such directory in
the app, then there is no cost to the running system since that
directory is not added to the list of places to look during the
loading process.)

So here is how this works in practice. In my upgraded version of
django-faq there are templates in the "templates/faq" directory that
extend "faq/base.html" in order to support normal non-pinax
development. However I also have "templates/pinax/faq/base.html" which
is a version of base.html that plugs into pinax. When the pinax loader
is used, this file is found first and everything works. Note that my
personal pinax project file no longer has a "templates/faq" directory
at all, but the app works fine in pinax due to the pinax template
loader. Of course other app developers might chose to alter a bunch of
their templates for pinax and not just a single one as I did. That
works too.

Note that this change is safe and backwards compatible. When this new
app loader is put into place and the settings file is adjusted to use
it, all of the apps that don't take advantage of the new loader
continue to work in the present manner.

I am sending these bits along to James now, but if anyone else wants
to try it out, just drop me a note at rockmhoward via gmail.

[Dougn]

On Nov 19, 8:30 am, Rock <r...@rockhoward.com> wrote:
> I just created a new pinax template loader that allows for the
> creation of reusable apps that work "out of the box" in either
> standalone mode or within pinax. Here is how it works.
>
> The loader is named pinax_app_directories.py. (A copy of this file
> should be included in basic_project.py and complete_project.py so that
> the pinax user copies it into their projects.) The template loader is
> adjusted to invoke this loader after "filesystem" but before
> "app_directories".
>
> The new pinax loader works just like the "app_directories" loader but
> looks for templates in each installed app within "templates/pinax"
> instead of "templates".  (Of course if there is no such directory in
> the app, then there is no cost to the running system since that
> directory is not added to the list of places to look during the
> loading process.)

Question: what benefit does this get us over just changing all the
pinax templates (starting at base, and including all the reusable
apps) to load templates like "app/pinax/foo.html" ?
I would find it extremely confusing to have "app/foo.html" in fact be
loading "app/pinax/foo.html"

the app/pinax/xxx.html templates would all reference other app/pinax/
xxx.html templates (with stub defaults to just include or extend app/
xxx.html which could be set at the project instead of app level in
pinax for those externals apps which dont have it).

The only thing I see we loose is some minor auto-magic for falling
back from app/pinax/xxx.html to app/xxx.html if the first (not even
asked for) template does not exist. auto-magic like this is bad, auto-
magic which changes django auto-magic is even worse.

Python Zen #1:
explicit is better than implicit.

(just my $0.02)

-Doug

[Rock]

To implement your suggestion would require discarding the django
template loaders and instead only having a custom pinax template
loader that would, in turn, require all templates in reusable apps to
be replicated unnecessarily. That goes against the well known and
explicitly defined django conventions for template loading. My
suggestion does not break the current model. It simple extends it in
such a way that a 3rd party app provider can deliver his app in a way
that it works "out of the box" in standalone mode as well as within
pinax without requiring any manipulations by the end user or even the
core team that is assembling the various example pinax projects.

Of course with various aspects of the ultimate pinax 1.0 templating
model still in flux, it is too early to decide on the ultimate loading
model that makes the most sense. I just wanted to point out that
adding a special pinax loader was a safe and interesting alternative
that might improve significantly improve the experience of the typical
pinax implementor (and especially those who track trunk which
currently is a bit of a land mine field.)

Finally I must point out that your question does not accurately
reflect the template loading that I am proposing. For an app like
django-faq which is installed as 'faq', the loading sites would be (in
order):

basic_project/templates/faq/

faq/templates/pinax/faq/

faq/templates/faq/

The first and the last site are the same as we have today. The middle
one would be provided by the suggested pinax_app_directory loader. In
tha case of django-faq, the only file in the pinax area would be faq/
base.html which would override the faq/base.html in the faq templates
area in order to switch the app so it works as a fully functional
pinax page instead of a standalone page.

[Dougn]

On Nov 20, 12:04 pm, Rock <r...@rockhoward.com> wrote:
> To implement your suggestion would require discarding the django
> template loaders and instead only having a custom pinax template
> loader that would, in turn, require all templates in reusable apps to
> be replicated unnecessarily. That goes against the well known and
> explicitly defined django conventions for template loading. My
> suggestion does not break the current model. It simple extends it in
> such a way that a 3rd party app provider can deliver his app in a way
> that it works "out of the box" in standalone mode as well as within
> pinax without requiring any manipulations by the end user or even the
> core team that is assembling the various example pinax projects.

No, you misunderstand.
I just want to use the standard django template loaders.
I don't want any custom loaders.

I want to have all the pinax templates do things like:

{% extends "app/pinax/foo.html" %}

Instead of:

{% extends "app/foo.html" %}


The default app/pinax/foo.html (which might exist on the project level
pinax/templates/ sub tree) might just be {% include "app/foo.html" %}
but it allows for the configuration.

In other words use the standard template loader system and be explicit
about the pinax specialization in reusable apps instead of using some
'magic' loader.

-Doug

[Rock]

That is not DRY at all.

Now I have to have two copies of every template in my 3rd party app.
One in the templates/faq/ directory and one in the templates/pinax/
faq/ directory. Except for base.html (where the actual differences are
located), the other templates are identical with their twins except
one extends "faq/pinax/base.html" and the other copy extends "faq/
base.html". Well I guess I can write a script to handle that, but if I
may -- yuk!

With django-faq, which is pretty simple, I now have 6 extra template
files to maintain. In many of the more sophisticated 3rd party apps
that number would be much much higher.

Furthermore with your scheme there is a very good chance that a pinax
project developer will end up using the wrong templates. That would
never happen with my approach. You just plug the app into
"INSTALLED_APPS", add the include('faq.urls') in the urls.py and you
are ready to roll.

[Dougn]

Ok I think I am very confused then.

I thought the point of your template loader was that it would first
look in the app/pinax/ directory and if its not found there go look
for it in the /app/ template directory (assuming the app prefix as the
standard which we live by). Thus if you had a pinax specialization
template under /app/pinax/ it would use that one instead of the one
in /app/ (which is the default template to use when the app is not
used in pinax).
As such you still have two templates.

If that is not what you are talking about then I am sorry for
misunderstanding.

If you are going to have two templates in most circumstances (one for
non pinax and an optional pinax override) and have the loader make the
decision, (which is what I though was going on), then I really do not
see any point in having that done with a custom template loader. At
least IMHO.

-Doug

[Jannis Leidel]

> If you are going to have two templates in most circumstances (one for
> non pinax and an optional pinax override) and have the loader make the
> decision, (which is what I though was going on), then I really do not
> see any point in having that done with a custom template loader. At
> least IMHO.

Just for the record, I talked to Rock on IRC about his template loader
and had the same doubts about its usefulness. Introducing that kind of
magic isn't the right way to enable 3rd party app developers to
prepare their apps for Pinax.

In fact I'm against any special treatment of templates of 3rd party
apps in regards of integration with Pinax because that'd introduce a
norm that'd also be easy to understand for users by documentation of
the optional template hooks (if and only if we want to support
template hooks at all, of course).

Jannis

[Rock]

This discussion has been useful. I can see why Doug is concerned about
the additional loaderas it will make some his future plans for plugin
points trickier to realize. (Personally I am not convinced yet that
plug-in points is the right way to go for granting that kind of
personal layout flexibility, but I admit it is interesting stuff.)

Jannis makes a good point and that has refined my vision a bit.
Instead of having the custom loader look inside the app for a pinax
template area, maybe this area should be inside the pinax project.
This would work pretty much identically.

As to benefits, here is one I hadn't pondered at first. When
installing the Pinax project the project's main template area would
only contain base, site_base, 400, 500 and homepage. All of the app
specific Pinax templates included in the project would be elsewhere.
Only overrides by the developer using Pinax would be in the main
template area. That could make life much better for the Pinax
developer.

I should also state in this thread that over half of the problems that
Pinax has with 3rd party templates is due to the unconventional use of
blocks in the Pinax base templates. See Holscher's latest blog post at
http://ericholscher.com/blog/2008/nov/20/gentlemans-agreement-django-templates/
for a first attempt at defining some conventions that would reduce
"template friction" for everyone.

[Dougn]

On Nov 21, 9:31 am, Rock <r...@rockhoward.com> wrote:
> This discussion has been useful. I can see why Doug is concerned about
> the additional loaderas it will make some his future plans for plugin
> points trickier to realize. (Personally I am not convinced yet that
> plug-in points is the right way to go for granting that kind of
> personal layout flexibility, but I admit it is interesting stuff.)

I don't see how it will cause any problems for the plugins.
The plugins just use a basic template lookup. If this is something
which causes problems for template lookups to the point where every
third party developer must be aware of or use the custom template
loader (which I did not think is the case) then there is a major
problem in general.

I think you are missing the point here, in that there should not be
any reason to have a custom template loader for doing basic block
operations. If you do, then there is something wrong with your
templates.

> Jannis makes a good point and that has refined my vision a bit.
> Instead of having the custom loader look inside the app for a pinax
> template area, maybe this area should be inside the pinax project.
> This would work pretty much identically.
>
> As to benefits, here is one I hadn't pondered at first. When
> installing the Pinax project the project's main template area would
> only contain base, site_base, 400, 500 and homepage. All of the app
> specific Pinax templates included in the project would be elsewhere.
> Only overrides  by the developer using Pinax would be in the main
> template area. That could make life much better for the Pinax
> developer.

I dont see why you need a custom template loader to achieve this. This
is how PyCon-Tech has worked sense day 1.
I have a local Pinax project and it has 6 templates at the project
level, with the rest pushed to the apps.

> I should also state in this thread that over half of the problems that
> Pinax has with 3rd party templates is due to the unconventional use of

> blocks in the Pinax base templates. See Holscher's latest blog post athttp://ericholscher.com/blog/2008/nov/20/gentlemans-agreement-django-...

> for a first attempt at defining some conventions that would reduce
> "template friction" for everyone.

I would claim that _ALL_ the problems are with not understanding how
to properly use blocks, templates, includes, and extensions.

There is no reason to have a custom template loader like this, if
there was a need for it it would be part of core django.

-Doug

[Rock]

>
> I would claim that _ALL_ the problems are with not understanding how
> to properly use blocks, templates, includes, and extensions.
>

Close to all, but if you want to hook into the tabs, then that is
Pinax specific for the time being. Maybe it is time for a django-tabs
app and at that point I would be inclined to agree with you. However
another that must happen is for Pinax to adopt block names that fit
better with the majority of third party apps as per Holscher's
article. (Heck it was Pinax that prompted Eric to write that blog
post!)

> There is no reason to have a custom template loader like this, if
> there was a need for it it would be part of core django.
>

Core django supports defining your own template loader. Many projects
do just that. Personally I don't see it as being a much bigger deal
than writing your own middleware or context processor. In my opinion
if a framework needs to define a custom template loader to support
application integration efforts, then so be it.

The caveat is that with good design and respected conventions for
blocks and such, the integration effort may be reduced to practically
zero which would obviate the need for the custom loader. Personally I
would have no problem with that solution at all!

[Dougn]

On Nov 21, 1:50 pm, Rock <r...@rockhoward.com> wrote:
> > I would claim that _ALL_ the problems are with not understanding how
> > to properly use blocks, templates, includes, and extensions.
>
> Close to all, but if you want to hook into the tabs, then that is
> Pinax specific for the time being. Maybe it is time for a django-tabs
> app and at that point I would be inclined to agree with you. However
> another that must happen is for Pinax to adopt block names that fit
> better with the majority of third party apps as per Holscher's
> article. (Heck it was Pinax that prompted Eric to write that blog
> post!)

Something like django-navbar which PyCon-Tech uses maybe? ;-)
http://us.pycon.org/2009/about/

I agree, the CSS hack is cool and I have struggled with it in the
past. Even had a working solution which used a custom tag which talked
up and down the context stack (yikes!).

>
> > There is no reason to have a custom template loader like this, if
> > there was a need for it it would be part of core django.
>
> Core django supports defining your own template loader. Many projects
> do just that. Personally I don't see it as being a much bigger deal
> than writing your own middleware or context processor. In my opinion
> if a framework needs to define a custom template loader to support
> application integration efforts, then so be it.

I agree with that, I have written many, including ones to integrate
django with PHP, Zope:TAL, and others. Heck the app-plugins are little
more than a template loader in a tag. I just don't like using them to
mask the real problem of bad design. (NOTE: there are many examples of
bad design, this is the most common. bad design due to evolution. At
some point a project grows and design decisions which were the proper
ones early on become bad ones once the project grows)

>
> The caveat is that with good design and respected conventions for
> blocks and such, the integration effort may be reduced to practically
> zero which would obviate the need for the custom loader. Personally I
> would have no problem with that solution at all!

Glad we are in agreement :-)

-Doug