Proposal: Revised form rendering

75 views
Skip to first unread message

Russell Keith-Magee

unread,
Jul 11, 2010, 11:36:30 AM7/11/10
to Django Developers
Hi all,

I'd like to propose a few extensions to Django's form library for 1.3.
I'm still working on some fine details, but before I get too far, I'd
like to field opinions so that I can:

* Discover any edge cases I've missed in my analysis
* Field any criticisms from people with more design/frontend
experience than myself
* Determine any related problems that we have the opportunity to
solve at the same time
* Find out if there is anyone in the community who is interested in
helping out.

Apologies in advance for the length, but there's a lot of detail to cover.

With this proposal, I'd like to address three problems:

1. The layout problem. Django's forms can be rendered "as_ul",
"as_table" or "as_p", and that's it. These layout schemes can be
overridden and customized if you know what you're doing, but it's not
easy to do so. Furthermore, visual layout concerns aren't separated
from data processing concerns. You need to write (and install) a form
subclass to implement your own form layout. Although it's good
app-writing practice to ensure that forms can be easily substituted,
it's not an enforced or universal practice.

2. The widget problem. This is a variant on the previous point. A
designer that wants to use a specialized calendar widget for a date
field needs to modify form code. This is a complexity that shouldn't
exist; a designer should be able to specify the widget library that
needs to be used (with all it's required rendering requirements,
javascript triggers etc) without modifying views and form processing
code.

3. The DOCTYPE problem. Most importantly, there is the closing slash
problem, but the introduction of HTML5 also means that there are
richer input types like <input type="email"> that aren't available in
HTML4 or XHTML1. Django currently outputs XHTML1 unconditionally, and
has no support for the new HTML5 input types.

To solve these three problems, I'd like to propose that we add (and
promote) the use of a new approach to form rendering, based around the
use of a new {% form %} template tag. This proposal has some
similarities to a proposal made by in the 1.2 feature phase [1] -- but
that proposal was only aiming to solve the doctype issue.

[1] http://groups.google.com/group/django-developers/browse_thread/thread/bbf75f0eeaf9fa64

So: What I'm proposing is that we introduce a new template tag: {% form %}.

How does this solve the three problems?

Layout
------

The simplest approach for rendering a form would become:

{% form myform %}

This would effectively implement the as_table rendering strategy, just
as {{ myform }} does right now.

If we want a different rendering, we exploit the fact that {% load
%}ing a template library will override any template tags that are
redefined. {% form %} would be defined as part of the default template
tag library, implementing the 'as_table' strategy. However, if we load
a library that also defines the {% form %} tag, that definition will
override the base definition. If we want to use a custom rendering
style, we can get that by simply loading a different renderer that
implements that style:

{% load custom_renderer %}
{% form myform %}

Django would ship with {% form %} implementations of the 'as_p' and
'as_ul' strategies, so getting 'as_p' rendering would mean:

{% load xhtml_p_forms %}
{% form myform %}

{% form %} is just a template tag, but the default implementation
would be designed in such a way that it could be easily subclassed to
alter the rendering strategy for the form. I'm still tinkering with
details here, but broadly, the intention is to expose a similar
interface to that used by Form.as_*() -- that is, returning format
strings that specify like '<td>%(errors)s%(field)s%(help_text)s</td>'
to define the rendering strategy; this would be implemented as a
function so that forms could decide on a per field basis what output
format is appropriate. However, unlike the existing as_*() approach,
you don't need to have access to the form in order to use the
different renderer, which means you can define and apply your
rendering strategy independent of the view and form code.

Since the form renderer exposes the logic for rendering individual
form rows, we can also expose the ability to render individual form
fields, plus the non-field errors and hidden fields:

{% form myform errors %} -- All the non-field form errors, plus
hidden field errors
{% form myform field birthdate %} - output a full row for the
birthdate field (wrappers, label, errors, help etc)
{% form myform hidden %} -- output all the hidden fields

This just exposes the internal mechanics that makes the full-form
rendering of {% form myform %} possible.

Widgets
-------

The second problem is support for widgets and other rendering
customization. This can be addressed using extra arguments to the {%
form %} tag when rendering individual fields:

{% form myform field birthdate using calendar %}

This instructs the rendering of the birthday DateField to use the
'calendar' chrome.

What is chrome? Chrome is an attempt to overcome the practical
limitations of Django's Widgets.

When we introduced newforms, the intention was that widgets would be
the point at which form rendering would be customized. If a developer
wanted to use a rich Javascript rendering for the calendar, they would
define a custom Date widget, override the render() method to introduce
the appropriate Javascript and CSS hooks, and then define a form with
fields that specify the use of that widget. The way admin uses widgets
is probably the best example of how this was intended to work.

However, in practice, widgets aren't used like this. Widgets aren't
trivial to define, and they aren't easy to deploy, either. The
convoluted mechanics that ModelAdmin goes through in order to install
a custom calendar widget is probably the best demonstration of why
this idea hasn't taken off -- it's just too complex.

It's also not simple to "just use a different widget", either. The
interplay between form and widget is sufficiently complex that the
only time at which you can define the widget that is to be used is
when you define the field itself. This means that widget choice is
closely bound to form design, which makes it nigh impossible define a
scheme for deploying widgets as part of the template rendering layer.

So -- the idea behind chrome is to separate the raw HTML input
mechanism from the bits that make the input look good on the rendered
page.

Widgets remain as they are, but we de-emphasize their use as a
rendering customization tool. They become little more than the
decision over which HTML <input> (or <select>) will be used. It's
still important that widgets are distinct from fields. After all,
there are times when there are multiple input options for particular
field data -- consider the case of a location field, that will require
a custom lat/long split widget.

It's the job of the chrome to make the <input> provided by the Widget
look pretty. This might mean adding extra classes to the rendering of
the <input>; it might mean putting placeholder <div>s or other
elements around the input; or it might mean registering that a
javascript block is required to support that input.

Chrome can also be layered. Different pieces of chrome could perform
different roles, and could be applied one after he other. For example
in the following:

{% form myform field birthdate using calendar important %}

The "calendar" chrome might add the javascript and classes to enable
the rich widget, where the "important" chrome might just add CSS
classes that enables a particular field to be rendered in a particular
style.

Chrome can also be parameterized; for example:

{% form myform field name using autocomplete:"name_autocomplete" %}

might define a chrome that implements an Ajax autocomplete widget
using the named URL "name_autocomplete" as a data source. This has to
potential to start giving an answer to the "Why doesn't Django do
AJAX" monkey; Django won't provide an AJAX solution out of the box,
but packaging a chrome that implements AJAX should be a lot easier.

Chrome could also be programatically assigned as part of the form.
Using a formfield_callback() style setup, it should be possible to
automatically assign certain chromes to certain field types (e.g.,
always apply the calendar chrome to DateFields).

I also hope to be able to provide a solution for the problem of
putting all the javascript for a form at the bottom of the page. At
present, widget rendering allows you to insert a <script> tag
immediately before or after a form element, but best practice says it
should be contained in a single block at the end of your page.

By rendering a widget using a template tag, we get the ability to set
variables in the context. When a form field is rendered using a
particular piece of chrome, that chrome can register that it requires
a specific javascript trigger; a {% form triggers %} call can then be
used to retrieve all the triggers that have been registered, and embed
them in a <script> call at the end of the page.

Again, I'm still working on details here, but the intention is to make
chrome easy to define -- hopefully just a class with some fairly
simple entry points.

Doctypes
--------

The third problem is doctype support. To solve this, I propose two changes.

1) We introduce a 'doctype' argument to the render() method on
widgets. Widgets would then be expected to generate HTML output that
is compatible with the supplied doctype. For backwards compatibility,
internal calls to widget.render() would need to introspect for the
existence of the 'doctype' kwarg on the specific widget, and raise
warnings if the kwargs isn't available.

2) We introduce a number of new default widgets to support the new
HTML5 input types. At present, EmailField uses a TextInput by default.
Under the new scheme, we would introduce an EmailInput, and EmailField
would use EmailInput as the default widget. When rendered using the
HTML4/XHTML1 doctype, this would output <input type="text" ...; under
HTML5, this would output <input type="email".... Again, the doctype
argument to render() would be used to determine which input type is
used.

Once these two changes are in place, we use the form template tag
specify the doctype that is passed to the widget render call. A
'html5_p_forms' library will pass 'html5' as the doctype when
rendering fields, and get HTML5-compliant form fields; the
'xhml1_p_forms' library will pass 'xhtml1', and get XHMTL1-compliant
form fields.

Summary
-------

In summary, I'm proposing:
- A new {% form %} template tag with a fairly simple interface for overriding
- A couple of default implementations of the {% form %} template tag
- Adding a chrome layer
- Propagating the 'doctype' kwarg through the existing
form/field/widget infrastructure

It's important to note that all of this is backwards compatible. The
old {{ form }} and {{ form.as_p }} approaches to rendering will
continue to work, as will the longhand {{ field.errors}} {{
field.label }} {{ field }} approach. What I'm proposing is a layer on
top of the primitives Django already provides. Hopefully, this new
layer will be sufficient to make the common use cases for form
customization slightly more automated, while providing hooks that
allow designers to customize the way forms are rendered without
needing to delve into view/form internals.

I appreciate that some of this is hard to judge without specifics --
in particular, the specifics of how end users will customize the Form
template tag rendering and specify chrome. I really wanted to propose
this with a sample implementation, but:

1) I accidentally let the cat out of the bag on Twitter,
2) I wasn't sure I was going to have a implementation ready for
comment before we draw the curtain on the 1.3 feature list,
3) I wanted to get some feedback on the broad design goals before I
went too far down a rabbit hole
4) I wanted to work out if there was any interest in the community in
helping out to achieve these goals

Comments? Criticisms? Queries?

Yours,
Russ Magee %-)

David Larlet

unread,
Jul 11, 2010, 1:16:22 PM7/11/10
to django-d...@googlegroups.com
Hi,

Le 11 juil. 2010 à 17:36, Russell Keith-Magee a écrit :
> I'd like to propose a few extensions to Django's form library for 1.3.

First of all, thanks or your proposal, the current form rendering is the worst part of Django to me and I'd like to help to improve that in 1.3.

> Layout
> ------


> style, we can get that by simply loading a different renderer that
> implements that style:
>
> {% load custom_renderer %}
> {% form myform %}
>
> Django would ship with {% form %} implementations of the 'as_p' and
> 'as_ul' strategies, so getting 'as_p' rendering would mean:
>
> {% load xhtml_p_forms %}
> {% form myform %}

Just a personal feedback, to me the rendering strategy is related to a whole project and should be defined in settings, it's too easy to forget a loading in a template. I know that you can use the django.template.add_to_builtins function but it in this case it should be documented.

>
> Widgets
> -------


>
> Chrome can also be parameterized; for example:
>
> {% form myform field name using autocomplete:"name_autocomplete" %}
>
> might define a chrome that implements an Ajax autocomplete widget
> using the named URL "name_autocomplete" as a data source. This has to
> potential to start giving an answer to the "Why doesn't Django do
> AJAX" monkey; Django won't provide an AJAX solution out of the box,
> but packaging a chrome that implements AJAX should be a lot easier.

If it requires an extra {% form %} arg it will not be that easier if you need to override all third-party apps' templates. Note that I haven't got any solution, that's more to bring the discussion on that topic :-).

>
> Doctypes
> --------


>
> Once these two changes are in place, we use the form template tag
> specify the doctype that is passed to the widget render call. A
> 'html5_p_forms' library will pass 'html5' as the doctype when
> rendering fields, and get HTML5-compliant form fields; the
> 'xhml1_p_forms' library will pass 'xhtml1', and get XHMTL1-compliant
> form fields.

Again, why not directly in settings in order to be project's specific? Is there anybody mixing doctypes on the same website? (backward compatibility maybe?)

Regards,
David

Alex Gaynor

unread,
Jul 11, 2010, 1:32:41 PM7/11/10
to django-d...@googlegroups.com

Sure. The admin is XHTML and plenty of the frontends I work with are HTML[45].

Alex

> Regards,
> David
>
> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
>
>

--
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me

Javier Guerra Giraldez

unread,
Jul 11, 2010, 1:35:38 PM7/11/10
to django-d...@googlegroups.com
On Sun, Jul 11, 2010 at 12:16 PM, David Larlet <lar...@gmail.com> wrote:
> Le 11 juil. 2010 à 17:36, Russell Keith-Magee a écrit :
>> {% load custom_renderer %}
>> {% form myform %}
>>
>> Django would ship with {% form %} implementations of the 'as_p' and
>> 'as_ul' strategies, so getting 'as_p' rendering would mean:
>>
>> {% load xhtml_p_forms %}
>> {% form myform %}
>
> Just a personal feedback, to me the rendering strategy is related to a whole project and should be defined in settings, it's too easy to forget a loading in a template. I know that you can use the django.template.add_to_builtins function but it in this case it should be documented.


i'd say the other way around; it's not hard to imagine a single page
with two or three forms rendered with different styles: a 'main' one
rendered as_table(), and a footer field rendered as_p(). in that
case, {%load%}'ing the style is too coarse granularity. i'd prefer an
optional parameter to {%form%} to choose the renderer.

something like:

class myformrenderer (forms.as_table):
name = 'left_table'
......

and in the template you could say {% form myform:"left_table" %}

>> Doctypes
>> --------
>>
>> Once these two changes are in place, we use the form template tag
>> specify the doctype that is passed to the widget render call. A
>> 'html5_p_forms' library will pass 'html5' as the doctype when
>> rendering fields, and get HTML5-compliant form fields; the
>> 'xhml1_p_forms' library will pass 'xhtml1', and get XHMTL1-compliant
>> form fields.
> Again, why not directly in settings in order to be project's specific? Is there anybody mixing doctypes on the same website? (backward compatibility maybe?)


here i fully agree, doctype should be project-specific.

--
Javier

flo...@gmail.com

unread,
Jul 11, 2010, 2:14:16 PM7/11/10
to Django developers
I'm glad to see that some serious thought is going into this issue!
This proposal sound very good (to me, at least) on the whole.

One thing that occurs to me, though, is that the people who are going
to want to customize form output are very frequently going to be the
people working mostly in the templates: the designers. It looks like
you're not fully settled on the interface, but I think that exposing
it on the form class like as_* means that it's going to be difficult
for designers to customize that output.

It seems to me that one way to give designers the ability to customize
that output is to move some of that output logic out of Python and
into templates. Widgets could load up a template from a well-known
location--something like e.g. django/widgets/textarea.html--and then
template authors could customize that by providing their own template
in that same path. There would be some issues to sort out there, like
the fact that Django would have to install some implicit Django-
provided template directory at the end of TEMPLATE_DIRS, but I don't
think that's too onerous.

I'm not 100% sure that that's the answer, but whatever the answer is,
I think it's important to note that the target audience for
customizing form output isn't always going to be the Python
programmer.

Thanks,
Eric Florenzano

Iván Raskovsky

unread,
Jul 11, 2010, 5:35:58 PM7/11/10
to django-d...@googlegroups.com
Hi, thanks for the proposal!

One thing that might be worth looking at is templatetag namespaces.
There are a couple of issues that I see emerging with this changes.

It's been mentioned before that one would like to render with different {% form %} tags different forms in one template, namespaces would allow this.

Imagine now, that with this proposal addition some chrome libraries start to appear. What about using chromes from more than one library? Or combining chromes? I imagine that each library would start namespacing the chromes in their names, so that their names don't clash. This isn't a nice solution.

There's been some work in this before: http://code.djangoproject.com/ticket/2539

What do you think?
    Iván

Idan Gazit

unread,
Jul 11, 2010, 5:38:44 PM7/11/10
to Django developers
What a fantastic proposal. I have some concerns, but I'm not sure if
any of them have to do with my misunderstanding.

1. The {% load %} mechanism can get ugly, fast. What if I am rendering
multiple different forms on the same page? {% load %} {% form %} {%
load %} {% form %} feels mildly unclean to me. The only alternative
that comes to (my) mind is specifying an alternate renderer in the {%
form %} tag, but that will add yet another argument to a tag that
already has an unwieldy list of (possible) arguments.

2. The {% load %} mechanism is overloaded with two orthogonal
functions: doctype selection and custom rendering. If I write a
reusable widget that need some custom rendering fanciness, then the
logic for rendering the widget goes in one place (the widget class,
based on the doctype kwarg) but I have to provide several custom
renderers, one for each doctype. Seems inconsistent to me. I like the
idea of widgets being responsible for the widget and chrome being
responsible for everything else, but feel like the two concerns might
need to be represented individually. Doctype doesn't change throughout
the document, but renderers might.

3. Related to #2, what is the behavior of a Widget if I ask it for a
doctype it doesn't support?

I need to think about the renderer/chrome bits some more, will weigh
in again in the morning with a clear head.

-I

On Jul 11, 6:36 pm, Russell Keith-Magee <freakboy3...@gmail.com>
wrote:
> [1]http://groups.google.com/group/django-developers/browse_thread/thread...
> would use EmailInput as the default widget. When rendered using the ...
>
> read more »

Tim Chase

unread,
Jul 11, 2010, 5:43:32 PM7/11/10
to django-d...@googlegroups.com
[please excuse the slight chop-up-and-reordering of your
original into my quoting]

On 07/11/2010 10:36 AM, Russell Keith-Magee wrote:
> {% form myform %}

For what my vote may be worth, I'm -0 on this...

> {% form myform field birthdate using calendar %}

and -1 on this

> {% form myform field birthdate using calendar important %}

> {% form myform field name using autocomplete:"name_autocomplete" %}

and REALLY -1.0e1 on this syntax-soup. Does this result in
something that looks like the following monstrosity?

{% form myform field name using

autocomplete:"name_autocomplete" field birthdate using calendar
field other_field using some_other_chrome_or_widget%}


Putting in a subtle plug for one of my wish-list items, I've
always been curious why form-rendering wasn't implemented as
a filter:

{{ myform|as_ul }}

> These layout schemes can be overridden and customized if you
> know what you're doing, but it's not easy to do so.

I've occasionally wished for an as_dl (which would render
labels as <dt> and controls as <dd> elements), and having a
base/ancestor as_* filter object to base it off of would
make this a lot easier. This would take care of your
issue#1 (more easily extensible renderings).

Under such a scheme, I'd imagine your above examples might
look something like

{{ myform|as_ul }}
{{ myform|use_widget:"birthdate=calendar"|as_ul }}

where the use_widget (deep?)copies the input form object,
adjusting its field-widgets per the requested use_widget
modifiers. It could even be neatly stacked something like
(line-broken for clarity)

{{ myform
|use_widget:"birthdate=calendar"
|use_widget:"name=name_autocomplete"
|use_widget:"other_field=some_other_chrome_or_widget"
|as_ul
}}

For backwards compatibility, the implementations of
Form.as_*() should be able to just look something like

def as_table(self, ...):
return filters.as_table(self, ...)

I'd suspect this could take care of your issue#2.

However that leaves me with no good answer to

1) your "parameters to chrome" wish (though that might be
possibly incorporated in the use_widget parameters). I'm
not entirely clear on your chrome ideas, but I like some of
what I read in your novella. :)

2) your "put all JS at the bottom of the page" wish

3) your issue#3 about DOCTYPES, but unless filters were given
access to a rendering context in which some global
DOCTYPE-preference was specified, I haven't liked any of
the solutions I've seen bandied about on this matter
(including a settings.py directive, since I can easily
imagine a single site producing a mix of html4, xhtml
and/or html5).

However I think all 3 of my outliers are somewhat
orthogonal to the rejiggering of form rendering.


> - Propagating the 'doctype' kwarg through the existing
> form/field/widget infrastructure

I'm +1 on this (or something like it such as request-context
or some other bag-of-attributes that could be useful in
rendering a widget) so that widgets can tweak their
display behavior.

Just my late-afternoon rambling thoughts...

-tkc


Daniel Greenfeld

unread,
Jul 11, 2010, 3:16:54 PM7/11/10
to Django developers
I agree with Eric and my experiences back it up. Most of the people
who want to custom form widgets are the ones who are unprepared to dig
into Django/Python code. The easier we can make creating/extending
form widgets the better.

This looks like what I'll be sprinting on at DjangoCon. :)

Danny

Russell Keith-Magee

unread,
Jul 11, 2010, 9:55:52 PM7/11/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 1:16 AM, David Larlet <lar...@gmail.com> wrote:
> Hi,
>
> Le 11 juil. 2010 à 17:36, Russell Keith-Magee a écrit :
>> {% load xhtml_p_forms %}
>> {% form myform %}
>
> Just a personal feedback, to me the rendering strategy is related to a whole project and should be defined in settings, it's too easy to forget a loading in a template. I know that you can use the django.template.add_to_builtins function but it in this case it should be documented.

This may be the case on many websites. However, in my experience, it
isn't the case for all websites. I wouldn't consider it unusual to
have different rendering strategies per-page, or even multiple
strategies on a single page.

>> Widgets
>> -------
>>
>> Chrome can also be parameterized; for example:
>>
>> {% form myform field name using autocomplete:"name_autocomplete" %}
>>
>> might define a chrome that implements an Ajax autocomplete widget
>> using the named URL "name_autocomplete" as a data source. This has to
>> potential to start giving an answer to the "Why doesn't Django do
>> AJAX" monkey; Django won't provide an AJAX solution out of the box,
>> but packaging a chrome that implements AJAX should be a lot easier.
>
> If it requires an extra {% form %} arg it will not be that easier if you need to override all third-party apps' templates. Note that I haven't got any solution, that's more to bring the discussion on that topic :-).

I acknowledge that this is a weakness; however, I don't see it as a
major concern. I'm of a similar mind to James Bennett on this: I don't
generally consider templates to be reusable app elements, except when
you're dealing with entire applications like admin.

James doesn't ship templates with his reusable apps specifically
because template aren't ever really reusable. If you're building a
site, you can't just drop large chunks of someone else's markup into
your site. The only time I've found templates truly reusable is when
you are given the template for an entire page -- at which point, the
issue of customizing the rendering scheme for an indiviual form is a
non-issue, since you just use whatever the full page gives you.

> Again, why not directly in settings in order to be project's specific? Is there anybody mixing doctypes on the same website? (backward compatibility maybe?)

Although a single doctype across a site will be common, there's
certainly no guarantee that this will be the case.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 11, 2010, 10:00:31 PM7/11/10
to django-d...@googlegroups.com

I'm in completely agreement on this. One of the reasons I posted this
in an incomplete form is to get feedback. Feedback from the designer
community on the utility of the interface is a big part of this, both
in terms of how the tag would be used, and how new tag implementations
would be defined.

I'm not 100% convinced that templates are the answer, but they're
certainly worth considering. I have two immediate concerns:

- the complexity of getting the templates picked up by the form
rendering template tag (getting all the template paths lined up so
that they are discovered and used correctly)
- the performance overhead of using templates to handle very small
template snippets. This is less of a concern now that we have cached
loaders, but

One possibility might be to include a template-based implementation of
form rendering as one of the default implementations. This way, the
template-based tag would be a gateway drug that designers could use to
prototype a new style; if a particular rendering needs to be optimized
or becomes common across multiple projects, it should be easy for a
developer to take a bunch of prototype templates and turn them into a
native implementation.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 11, 2010, 10:23:49 PM7/11/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 1:35 AM, Javier Guerra Giraldez
<jav...@guerrag.com> wrote:
> On Sun, Jul 11, 2010 at 12:16 PM, David Larlet <lar...@gmail.com> wrote:
>> Le 11 juil. 2010 à 17:36, Russell Keith-Magee a écrit :
>>> {% load xhtml_p_forms %}
>>> {% form myform %}
>>
>> Just a personal feedback, to me the rendering strategy is related to a whole project and should be defined in settings, it's too easy to forget a loading in a template. I know that you can use the django.template.add_to_builtins function but it in this case it should be documented.
>
> i'd say the other way around; it's not hard to imagine a single page
> with two or three forms rendered with different styles: a 'main' one
> rendered as_table(), and a footer field rendered as_p().

I agree - form rendering scheme isn't something that is site-wide, and
it isn't necessarily page-wide either. I acknowledge that page-wide
rendering is a limitation of my proposal. If we can find an elegant
way to represent it, I agree that it is worth considering.

> in that
> case, {%load%}'ing the style is too coarse granularity.  i'd prefer an
> optional parameter to {%form%} to choose the renderer.
>
> something like:
>
> class myformrenderer (forms.as_table):
>  name = 'left_table'
>  ......
>
> and in the template you could say {% form myform:"left_table" %}

I can see where you're going with this; I have two concerns:

* Duplication. The 'left_table' flag needs to be applied to every use
of the {% form %} tag on a page. If you're
manually rolling out every field on a form, this is a lot of code duplication.

* Composibility. If I understand your intention, a form library would
need to provide all the layout schemes that you want to have available
on a page. That means if you wanted "P" and "UL" forms on the same
page, you would need to define a combined "P & UL" form library. This
sort of composition doesn't strike me as a desirable goal.

One proposal that has been made elsewhere in this thread (by Iván
Raskovsky) is to introduce namespacing for tempatetags. This has been
mooted several times in the past, and there have been several proposed
syntaxes; here's yet another:

{% load xhtml_div_forms %} -- loads into the root namespace
{% load namespace foo xhtml_p_forms %} -- loads forms into the 'foo' namespace
{% load namespace bar xhtml_ul_forms otherlib %} -- loads forms and
otherlib into the bar namespace

{% bar:form firstform %} -- renders as ul
{% foo:form secondform %} -- renders as p
{% form thirdform %} -- renders as div

This way, you can have two different implementations of {% form %} on
the same page if you need them. Whatever the final agreed syntax, this
isn't a form-specific feature; it could be useful anywhere that you
need to disambiguate template tags/filters. This means the discussion
is orthogonal to this forms discussion; if we agree that namespacing
is a way to tackle this general problem, we can keep the discussion
about how to implement namespaces as a separate issue.

>> Again, why not directly in settings in order to be project's specific? Is there anybody mixing doctypes on the same website? (backward compatibility maybe?)
>
> here i fully agree, doctype should be project-specific.

As noted elsewhere, there's no guarantee that doctype will be
consistent across an entire project.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 11, 2010, 10:24:47 PM7/11/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 5:35 AM, Iván Raskovsky <rask...@gmail.com> wrote:
> Hi, thanks for the proposal!
> One thing that might be worth looking at is templatetag namespaces.
> There are a couple of issues that I see emerging with this changes.
> It's been mentioned before that one would like to render with different {%
> form %} tags different forms in one template, namespaces would allow this.
> Imagine now, that with this proposal addition some chrome libraries start to
> appear. What about using chromes from more than one library? Or combining
> chromes? I imagine that each library would start namespacing the chromes in
> their names, so that their names don't clash. This isn't a nice solution.
> There's been some work in this
> before: http://code.djangoproject.com/ticket/2539
> What do you think?

Agreed -- namespacing could be very useful in this context. See my
response to Javier for some comments.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 11, 2010, 10:25:24 PM7/11/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 3:16 AM, Daniel Greenfeld <pyd...@gmail.com> wrote:
> I agree with Eric and my experiences back it up. Most of the people
> who want to custom form widgets are the ones who are unprepared to dig
> into Django/Python code. The easier we can make creating/extending
> form widgets the better.

Agreed.

> This looks like what I'll be sprinting on at DjangoCon. :)

That's the response I was looking for :-)

Russ %-)

André Eriksson

unread,
Jul 11, 2010, 10:31:39 PM7/11/10
to Django developers
Good proposal overall. One thought I have in order to try and combat
the massive parameter list of {% form %} is to optionally add an
ending tag, as well as sub-tags:

{% form myform %}
{% using birthday=calendar %}
{% renderer "as_ul" %}
{% autocomplete "name_autocomplete" %}
{% doctype xhtml1 %}
{% endform %}

I see this as having several advantages:

1) It introduces a clearer way of laying out settings.
2) It leverages the template engine for specifying defaults in a
simple but ingenious way:
{% form myform %}
{% include "form_settings.django" %}
{% endform %}
-- form_settings.django:
{% doctype xhtml1 %}
{% replace_widget datetime=calendar %}

And because of the flexibility of the template engine we can easily
specify either section-, site- or project-specific form settings, with
the use of block tags and overriding them. This can be accomplished in
a few ways:

1) Keeping the current {{ myform }} method for displaying forms
without settings attached.
2) Converting to always using {% form myform %}, where the {% form %}
tag parses until it either finds a matching {% endform %} tag or a new
{% form %} tag; a new {% form %} tag means that there are no settings
attached.
3) Adding a parameter to {% form %} for settings: {% form myform
detailed %} {# settings #} {% endform %}
4) Adding two template tags, one for simple form rendering, another
for detailed form rendering. Naming them is left as an exercise to the
reader.

Javier Guerra Giraldez

unread,
Jul 11, 2010, 11:27:11 PM7/11/10
to django-d...@googlegroups.com
On Sun, Jul 11, 2010 at 9:31 PM, André Eriksson <ean...@gmail.com> wrote:
> {% form myform %}
>    {% using birthday=calendar %}
>    {% renderer "as_ul" %}
>    {% autocomplete "name_autocomplete" %}
>    {% doctype xhtml1 %}
> {% endform %}

i like this syntax; it's a lot more readable than a chain of
parameters of filters on a single tag.

> I see this as having several advantages:
>
> 1) It introduces a clearer way of laying out settings.
> 2) It leverages the template engine for specifying defaults in a
> simple but ingenious way:
> {% form myform %}
>    {% include "form_settings.django" %}
> {% endform %}
> -- form_settings.django:
>    {% doctype xhtml1 %}
>    {% replace_widget datetime=calendar %}


personally, i don't think this is a nice solution. {%include%}'ing a
set of settings is not a default, it's a packaged setting.

i like better to define a python form of the same; IOW document a
mapping, so that your first example would be equivalent to:

myform.render ({
'birthday_widget':'calendar',
'renderer': "as_ul",
'autocomplete': "name_autocomplete",
'doctype': "xhtml1"
})

or:

myform.render (birthday_widget='calendar', renderer="as_ul",
autocomplete="name_autocomplete", doctype="xhtml1")


and allow the use of either of these syntaxes in settings.py, form
definition or context processors to set defaults.


--
Javier

Javier Guerra Giraldez

unread,
Jul 11, 2010, 11:27:48 PM7/11/10
to django-d...@googlegroups.com
On Sun, Jul 11, 2010 at 9:23 PM, Russell Keith-Magee
<rus...@keith-magee.com> wrote:
>  * Duplication. The 'left_table' flag needs to be applied to every use
> of the {% form %} tag on a page. If you're
> manually rolling out every field on a form, this is a lot of code duplication.

absolutely. see my answer to André for an idea on this

>  * Composibility. If I understand your intention, a form library would
> need to provide all the layout schemes that you want to have available
> on a page. That means if you wanted "P" and "UL" forms on the same
> page, you would need to define a combined "P & UL" form library. This
> sort of composition doesn't strike me as a desirable goal.

no, the P, UL (and my hypothetical left_table) would each one be a
class; you could import each one separately (or maybe several included
by default). in my example, left_table would inherit from as_table,
simplifying the implementation. the {%form%} syntax wouldn't be a
parameter to a single renderer, it's a selector to choose which
renderer to use.

--
Javier

André Eriksson

unread,
Jul 11, 2010, 11:35:45 PM7/11/10
to Django developers
On Jul 12, 5:27 am, Javier Guerra Giraldez <jav...@guerrag.com> wrote:
> > 1) It introduces a clearer way of laying out settings.
> > 2) It leverages the template engine for specifying defaults in a
> > simple but ingenious way:
> > {% form myform %}
> >    {% include "form_settings.django" %}
> > {% endform %}
> > -- form_settings.django:
> >    {% doctype xhtml1 %}
> >    {% replace_widget datetime=calendar %}
>
> personally, i don't think this is a nice solution.  {%include%}'ing a
> set of settings is not a default, it's a packaged setting.

My idea was merely to provide a very flexible way for template authors
to define a single point of form settings if they so choose, without
introducing yet another way of defining settings and figuring out a
way to make it flexible enough to handle the various use cases. As
Russ pointed out, sometimes doctypes aren't sitewide, whereas other
times they are. By allowing (but not requiring!) the inherent
flexibility of django's template system in the form settings, the
template authors and by extension the designers are put in control of
how forms are rendered and to what extent default settings should be
provided and to what magnitude they affect the site.

André

Tai Lee

unread,
Jul 12, 2010, 1:05:50 AM7/12/10
to Django developers
Regarding the DOCTYPE problem, I don't think it's appropriate to set
the DOCTYPE as a setting for an entire project. As many have pointed
out, each bundled app could define a different DOCTYPE in their base
templates.

It doesn't make sense to apply a DOCTYPE setting to the context of a
form either, as the DOCTYPE is not set at the form level but the page
level, and all template tags that render HTML (not only forms) should
render markup that is appropriate for the DOCTYPE in use.

What about adding a {% doctype %} template tag for use in base
templates which will not only render the appropriate DOCTYPE, but
store it in the context? Then templates that extend from the base
template will have access to this context variable and know whether or
not to include a trailing forward slash for self closing elements, or
whether or not `email` fields should be rendered as `text` or `email`
type?

Regarding the application of settings and substitution of widgets
inside templates, I'd like to see an easy way for designers to create
bundles of presentational logic (templates, stylesheets, images) which
they can pass in or refer to as arguments to {% form %} or {%
formfield %} template tags. This could take the form of {% form myform
myformpresentationbundle %} or {% formblock %}{% formbundle
myformpresentationbundle %}{% fieldbundle mydatefield mycalendarbundle
%}{{ myform }}{% endformblock %} or {% formfield myform.mydatefield
mycalendarbundle %}.

I'm not sure how these bundles would be defined or made available to
and loaded from the templates. I don't think having to create template
tags and load them (especially if they override built ins) is going to
be user friendly for designers. Perhaps the {% form %} and {%
formfield %} template tags could just be inclusion template tags, but
instead of having the template that they include defined with the
template tag, they load it from the templates folder (or another
folder) according to the arguments?

Cheers.
Tai.

mattim...@gmail.com

unread,
Jul 12, 2010, 3:11:04 AM7/12/10
to Django developers


On Jul 12, 12:31 pm, André Eriksson <ean...@gmail.com> wrote:
> Good proposal overall. One thought I have in order to try and combat
> the massive parameter list of {% form %} is to optionally add an
> ending tag, as well as sub-tags:
>
> {% form myform %}
>     {% using birthday=calendar %}
>     {% renderer "as_ul" %}
>     {% autocomplete "name_autocomplete" %}
>     {% doctype xhtml1 %}
> {% endform %}
>

I'm +1 on Russell's proposal in general but I'm also -1 on the big
parameter list to the proposed {% form %} tag. Having written some
complex form field rendering tags for extjs/sencha I can say that
without some kind of built-in *args/**kwargs-like parameter handling
for template tags, then having more than a handful of parameters gets
long and messy quickly. Also, it doesn't make your tag that user
friendly either when they have to keep referring to the docs to figure
out how to use it.

Andre's idea is interesting and is certainly more readable.

regards

Matthew

--
http://wadofstuff.blogspot.com/

Russell Keith-Magee

unread,
Jul 12, 2010, 8:35:46 AM7/12/10
to django-d...@googlegroups.com
k

On Mon, Jul 12, 2010 at 5:38 AM, Idan Gazit <id...@pixane.com> wrote:
> What a fantastic proposal. I have some concerns, but I'm not sure if
> any of them have to do with my misunderstanding.
>
> 1. The {% load %} mechanism can get ugly, fast. What if I am rendering
> multiple different forms on the same page? {% load %} {% form %} {%
> load %} {% form %} feels mildly unclean to me. The only alternative
> that comes to (my) mind is specifying an alternate renderer in the {%
> form %} tag, but that will add yet another argument to a tag that
> already has an unwieldy list of (possible) arguments.

I think templatetag namespacing may address this issue. See my
response to Javier for details.

> 2. The {% load %} mechanism is overloaded with two orthogonal
> functions: doctype selection and custom rendering. If I write a
> reusable widget that need some custom rendering fanciness, then the
> logic for rendering the widget goes in one place (the widget class,
> based on the doctype kwarg) but I have to provide several custom
> renderers, one for each doctype. Seems inconsistent to me. I like the
> idea of widgets being responsible for the widget and chrome being
> responsible for everything else, but feel like the two concerns might
> need to be represented individually. Doctype doesn't change throughout
> the document, but renderers might.

I will admit I don't really have a good answer to this.

One counterargument would be that from the perspective of form layout,
there isn't much that is doctype specific -- from my initial
experimentation, the UL/P/DIV/TABLE based layouts are pretty much
doctype-agnostic. This isn't true for chrome, however.

Another counterargument would be that you only have to provide
multiple form renderers (or chrome implementation) if you actually
need to support multiple doctypes; a form renderer/chrome that only
works under HTML4 would always be a simple option.

A related issue here is that mixing chromes from external sources
isn't trivial. For example, if Alice wants to ship my own custom
calendar chrome, and Bob want to ship a custom autocomplete chrome,
the only way for Chuck to use both chromes on the same form is to
define a form renderer that absorbs (possibly by mixin) both chrome
definitions. This isn't a solution that most designers are going to be
comfortable with.

Ideally, chrome would be completely independent of the form
definition, but the problem I hit here is working out how to
"register" chrome without either duplicating the template tag
registration services, or making a special case in the existing
registration services for chrome. I'm not a big fan of special cases,
but in this case I may have to make an exception. Of course, if
someone can make the case for a general facility for registering
capabilities as part of the template library, we can add this feature
and chrome need not be a special case.

> 3. Related to #2, what is the behavior of a Widget if I ask it for a
> doctype it doesn't support?

I'm not sure this is actually a problem. Django currently ships a
widget for every legal <input> type (plus <select>). In order to
support HTML5, we'll need to add a couple more (like EmailField), but
these new types all have documented regressions to type="text". In the
new vision of Widgets as "just the input mechanics", there isn't much
need for end users to define custom widgets. Django will ship with all
the possible basic widgets; if you need customization, you should be
defining chrome instead.

Of course, there is the issue of dealing with people in the wild that
have written custom widgets. Those will continue to work exactly as
they do now, but they won't be adaptive to doctypes (they won't even
necessarily accept the doctype argument, so there will need to be some
internal jiggery-pokery to support that backwards compatibility case).
I suppose you *could* continue to use widgets as the keeper of rich
rendering, but I'm not convinced this needs to be a focus of our
design discussions.

> I need to think about the renderer/chrome bits some more, will weigh
> in again in the morning with a clear head.

Much appreciated. As a designer (or someone with a design focus)
you're part of the target audience, so I want to make sure we
implement something that will actually be useful.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 12, 2010, 9:12:14 AM7/12/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 10:31 AM, André Eriksson <ean...@gmail.com> wrote:
> Good proposal overall. One thought I have in order to try and combat
> the massive parameter list of {% form %} is to optionally add an
> ending tag, as well as sub-tags:
>
> {% form myform %}
>    {% using birthday=calendar %}
>    {% renderer "as_ul" %}
>    {% autocomplete "name_autocomplete" %}
>    {% doctype xhtml1 %}
> {% endform %}

As I commented to Tim; the {% form X field ... %} tag isn't trying to
render the entire form -- it's just rendering a single field. This
removes the need for "using" and "autocomplete" as a subtag. The only
way a full field is rendered is in the simple case of {% form X %},
which is the equivalent of the current {{ form }} -- i.e., just render
the entire form using defaults (where "default" == doctype, layout and
chrome).

Secondly, the "renderer" subtag misses the significance of the trick
with the template tag loading. The aim of the loading trick is that {%
form %} *is* an as_ul renderer; it doesn't need to be specified
separately.

Lastly, the doctype subtag isn't something that should be specified on
a per-form basis. It's a per-document attribute. My proposal gets
around this by tying the doctype to the form tag itself, which is a
constant through the rendered page.

> I see this as having several advantages:
>
> 1) It introduces a clearer way of laying out settings.

For some definitions of clever. It also has some potential problems:
* How do you put a <hr/> between the name and birthday fields?
* How do you render fields in a different order to the form specified
on the form?
* How do we handle content that is included inside the {% form %} block?

> 2) It leverages the template engine for specifying defaults in a
> simple but ingenious way:
> {% form myform %}
>    {% include "form_settings.django" %}
> {% endform %}
> -- form_settings.django:
>    {% doctype xhtml1 %}
>    {% replace_widget datetime=calendar %}

Again, most of these default don't actually need to be specified if
you treat the {% form %} tag as encompassing the doctype and renderer
requirement. Arguing that template inclusions are a convenient way to
get around the fact that defining a form requires lots of subtags
doesn't seem like a net win to me.

Call me -0 on this; Given that there's only one tag that actually
needs to be a subtag (using), I don't see what making {% form %} a
full block tag gains us.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 12, 2010, 9:16:53 AM7/12/10
to django-d...@googlegroups.com

I'm not sure I understand.

In my proposal, {% load custom_form %} loads a single form template
tag. That template tag implements a single specific rendering scheme.
You import library X, you get library X's layout. If you want a
different layout, you import a different library (namespaces being one
way to get around the 'multiple layouts in a single )

How are you getting your "as_p" renderer into your template? How is it
made available to the {% form %} tag as an argument? How does the
"selector" determine what is available for selection? What defines the
default behavior of {% form %} in the first place?

Yours,
Russ Magee %-)

Tim Chase

unread,
Jul 12, 2010, 9:38:16 AM7/12/10
to django-d...@googlegroups.com, Russell Keith-Magee
On 07/12/2010 08:12 AM, Russell Keith-Magee wrote:

> On Mon, Jul 12, 2010 at 10:31 AM, Andr� Eriksson<ean...@gmail.com> wrote:
>> Good proposal overall. One thought I have in order to try and combat
>> the massive parameter list of {% form %} is to optionally add an
>> ending tag, as well as sub-tags:
>>
>> {% form myform %}
>> {% using birthday=calendar %}
>> {% renderer "as_ul" %}
>> {% autocomplete "name_autocomplete" %}
>> {% doctype xhtml1 %}
>> {% endform %}
>
> As I commented to Tim; the {% form X field ... %} tag isn't trying to
> render the entire form -- it's just rendering a single field

Looking back over the thread, I'm the only Tim, but I don't seem
to see your response (neither in my email nor via gmane). If you
could disinter it from your sent-mail folder and resend, I'd
appreciate reading your thoughts in reply.

-tkc

Russell Keith-Magee

unread,
Jul 12, 2010, 9:35:38 AM7/12/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 1:05 PM, Tai Lee <real....@mrmachine.net> wrote:
> Regarding the DOCTYPE problem, I don't think it's appropriate to set
> the DOCTYPE as a setting for an entire project. As many have pointed
> out, each bundled app could define a different DOCTYPE in their base
> templates.
>
> It doesn't make sense to apply a DOCTYPE setting to the context of a
> form either, as the DOCTYPE is not set at the form level but the page
> level, and all template tags that render HTML (not only forms) should
> render markup that is appropriate for the DOCTYPE in use.

It's not appropriate to set doctype on a per-form basis, but that's
not really what my proposal is doing -- it's being defined on a
template tag that has scope over the template. If you import form
rendering library X, that library provides a {% form %} implementation
that specifies a doctype. Everywhere you use that instance of {% form
%}, you get the same doctype without the need to specify doctype.

> What about adding a {% doctype %} template tag for use in base
> templates which will not only render the appropriate DOCTYPE, but
> store it in the context? Then templates that extend from the base
> template will have access to this context variable and know whether or
> not to include a trailing forward slash for self closing elements, or
> whether or not `email` fields should be rendered as `text` or `email`
> type?

This has been proposed in the past [1]. I can certainly see how this
is compatible with what I've proposed.

[1] http://github.com/simonw/django-html

However, I prefer keeping things local to the templatetag rather than
relying on a template context variable. If you use the context
variable approach, writing an 'as_ul' layout requires that you provide
an implementation that can support every possible doctype, or provide
error handling when the wrong doctype is present. It also requires
handling for the 'doctype not provided' case.

Using the 'doctype defined on the tag' approach, you only have to
support the doctype that your form defines. If you want to support a
different doctype, write a different tag; if there are commonalities,
you can use a common base class. To me, this ultimately results in a
cleaner interface for end-users to implement.

> Regarding the application of settings and substitution of widgets
> inside templates, I'd like to see an easy way for designers to create
> bundles of presentational logic (templates, stylesheets, images) which
> they can pass in or refer to as arguments to {% form %} or {%
> formfield %} template tags. This could take the form of {% form myform
> myformpresentationbundle %} or {% formblock %}{% formbundle
> myformpresentationbundle %}{% fieldbundle mydatefield mycalendarbundle
> %}{{ myform }}{% endformblock %} or {% formfield myform.mydatefield
> mycalendarbundle %}.

Agreed - templates are a natural way to do this, even if only as a
prototyping tool. And bundling/packaging of these templates is
probably the biggest examples for why this might not be the best
approach.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 12, 2010, 9:43:28 AM7/12/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 3:11 PM, mattim...@gmail.com
<mattim...@gmail.com> wrote:
>
>
> On Jul 12, 12:31 pm, André Eriksson <ean...@gmail.com> wrote:
>> Good proposal overall. One thought I have in order to try and combat
>> the massive parameter list of {% form %} is to optionally add an
>> ending tag, as well as sub-tags:
>>
>> {% form myform %}
>>     {% using birthday=calendar %}
>>     {% renderer "as_ul" %}
>>     {% autocomplete "name_autocomplete" %}
>>     {% doctype xhtml1 %}
>> {% endform %}
>>
>
> I'm +1 on Russell's proposal in general but I'm also -1 on the big
> parameter list to the proposed {% form %} tag.

Ok - we must have different measures for what constitutes a "long
parameter list". As I noted in my response to Tim, there's exactly 1
redundant word in the template tag I've proposed.

The only other way I can think of to shorten the argument list is to
introduce different tags for rendering a single field, rendering
top-of-form field errors, and rendering javascript triggers; so:

{% form myform %} -- would render an entire form

but rolling out the full form manually would require:

{% form_errors myform %}
{% form_field myform.name using autocomplete:"name_ajax" %}
{% form_field myform.email %}
{% form_field myform.birthdate using calendar %}
{% form_triggers %}

To me, this just means a bunch more underscores in templates, and 3
extra tags to register (along with finding a way to share the form row
rendering logic between the {% form %} tag and the other tags.

> Having written some
> complex form field rendering tags for extjs/sencha I can say that
> without some kind of built-in *args/**kwargs-like parameter handling
> for template tags, then having more than a handful of parameters gets
> long and messy quickly.

I'll concede args/kwargs handling in template arguments isn't pretty.
My counterargument would be that I don't see the use case for lots of
argument and kwargs to chrome. "Use this calendar", or "show this
decoration", not "here are 5 different arguments to configure how this
chrome will operate".

The one use case I can think of for chrome arguments is to provide the
named URL for AJAX callbacks. However, this may just be a case of me
not being imaginative enough. If you can provide a use case, I'll
happily reconsider my position.

> Also, it doesn't make your tag that user
> friendly either when they have to keep referring to the docs to figure
> out how to use it.
>
> Andre's idea is interesting and is certainly more readable.

I'm having difficulty reconciling these two positions. My template tag
is too complex because it requires you to remember the idiom FORM X
FIELD Y USING Z; but a nested tag structure with 4 different subtags
is more readable and won't require reference to documentation to
understand how and when to use each subtag?

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 12, 2010, 9:47:19 AM7/12/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 9:38 PM, Tim Chase
<django...@tim.thechases.com> wrote:
> On 07/12/2010 08:12 AM, Russell Keith-Magee wrote:
>>
>> On Mon, Jul 12, 2010 at 10:31 AM, André Eriksson<ean...@gmail.com>  wrote:
>>>
>>> Good proposal overall. One thought I have in order to try and combat
>>> the massive parameter list of {% form %} is to optionally add an
>>> ending tag, as well as sub-tags:
>>>
>>> {% form myform %}
>>>    {% using birthday=calendar %}
>>>    {% renderer "as_ul" %}
>>>    {% autocomplete "name_autocomplete" %}
>>>    {% doctype xhtml1 %}
>>> {% endform %}
>>
>> As I commented to Tim; the {% form X field ... %} tag isn't trying to
>> render the entire form -- it's just rendering a single field
>
> Looking back over the thread, I'm the only Tim, but I don't seem to see your
> response (neither in my email nor via gmane).  If you could disinter it from
> your sent-mail folder and resend, I'd appreciate reading your thoughts in
> reply.

Hrm. Gmail says my response was sent, but it's not turning up in the
public archive -- I'll resend. Apologies if this results in a repost
for anyone.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 12, 2010, 9:48:16 AM7/12/10
to Django Developers
On Mon, Jul 12, 2010 at 5:43 AM, Tim Chase
<django...@tim.thechases.com> wrote:
> [please excuse the slight chop-up-and-reordering of your
> original into my quoting]

Only if you grant me the same liberty :-)

> On 07/11/2010 10:36 AM, Russell Keith-Magee wrote:

>> {% form myform field birthdate using calendar important %}
>> {% form myform field name using autocomplete:"name_autocomplete" %}
>
> and REALLY -1.0e1 on this syntax-soup.  Does this result in
> something that looks like the following monstrosity?
>
>  {% form myform field name using autocomplete:"name_autocomplete" field
> birthdate using calendar field other_field using
> some_other_chrome_or_widget%}

I think some wires have gotten crossed here - this example wouldn't be
legal syntax under what I'm proposing.

Under the scheme I'm proposing:
 * {% form myform %} would render an entire form, and
 * {% form myform field name ... %} would render a *single* field
using the strategy described by the form.

If you want to render multiple fields, then you make multiple calls to
{% form myform field ... %}. Your example would require the following
template:

{% form myform field name using autocomplete:"name_autocomplete" %}

{% form myform field birthdate using calendar %}

{% form myform field other_field using some_other_chrome_or_widget %}

The intention is to encompass the rules for a full line of a form in a
call to {% form X field Y %}, not to try and put then entire form and
all it's individual field customizations into a single tag. This also
ensures that the user has final control over form ordering, layout of
forms on the page, and any interstitial elements they want to put in
between form elements on the page.

> Putting in a subtle plug for one of my wish-list items, I've
> always been curious why form-rendering wasn't implemented as
> a filter:
>
>  {{ myform|as_ul }}

If you dig into the archives, this was proposed (I'm fairly certain
*I* suggested it or supported it at one point); I can't say I can give
a concrete answer as to why the suggestion wasn't adopted. However, in
the context of this proposal, I can give some counterreasons -- more
in a bit.

>> These layout schemes can be overridden and customized if you
>> know what you're doing, but it's not easy to do so.
>
> I've occasionally wished for an as_dl (which would render
> labels as <dt> and controls as <dd> elements), and having a
> base/ancestor as_* filter object to base it off of would
> make this a lot easier.  This would take care of your
> issue#1 (more easily extensible renderings).
>
> Under such a scheme, I'd imagine your above examples might
> look something like
>
>  {{ myform|as_ul }}
>  {{ myform|use_widget:"birthdate=calendar"|as_ul }}

I think this last example should probably be:

{{ myform.birthdate|chrome:"calendar"|as_ul }}

or even:

{{ myform.birthdate|calendar_chrome|as_ul }}

since this enables you to define chrome as a filter. This bit appeals to me.

However, beyond that advantage, I have three reasons for preferring a
template tag over a filter.

 * The first is stylistic. {{ myform|as_ul }} is certainly elegant;
however, I don't see that {{ myform.birthdate|calendar_chrome|as_ul }}
is equally elegant. Filter syntax requires that you take out all your
whitespace, which just seems messy to me. In a template tag, arguments
are all distinct, ordering is obvious, and when it isn't, you can
introduce syntactic sugar (like "using") to clarify.

 * Secondly, while I can certainly explain from a programmers
perspective as_ul formatting and calendar_chrome are applied as
filters in the order that they are being applied, I can't think of a
good conceptual explanation, suitable for non-programmers, for why
form rendering is a 'filtering' activity.

 * Lastly, a template tag gives you access to the context, which means
tricks like collating javascript triggers becomes possible. For me,
this is one of the major goals of this refactoring, and it's something
you can't do with a template tag.

> where the use_widget (deep?)copies the input form object,
> adjusting its field-widgets per the requested use_widget
> modifiers.  It could even be neatly stacked something like
> (line-broken for clarity)

... which, it's worth noting, isn't legal Django template syntax.
Django's template tags and filters are all limited to a single line
specifically to avoid the contents becoming syntax soup.

>  {{ myform
>  |use_widget:"birthdate=calendar"
>  |use_widget:"name=name_autocomplete"
>  |use_widget:"other_field=some_other_chrome_or_widget"
>  |as_ul
>  }}

Again, this shouldn't be a single template substitution -- it should
be three (one for each field).

> For backwards compatibility, the implementations of
> Form.as_*() should be able to just look something like
>
>  def as_table(self, ...):
>    return filters.as_table(self, ...)
>
> I'd suspect this could take care of your issue#2.

I suspect you're correct, although the implementation could be interesting.

> However that leaves me with no good answer to
>
> 1) your "parameters to chrome" wish (though that might be
>        possibly incorporated in the use_widget parameters).  I'm
>        not entirely clear on your chrome ideas, but I like some of
>        what I read in your novella. :)

If we make chrome fully fledged template tags, arguments to chrome
could be covered by normal templatetag argument handling:

{{ myform.name|autocomplete:"name_view"...

> 2) your "put all JS at the bottom of the page" wish
>
> 3) your issue#3 about DOCTYPES, but unless filters were given
>        access to a rendering context in which some global
>        DOCTYPE-preference was specified, I haven't liked any of
>        the solutions I've seen bandied about on this matter
>        (including a settings.py directive, since I can easily
>        imagine a single site producing a mix of html4, xhtml
>        and/or html5).
>
> However I think all 3 of my outliers are somewhat
> orthogonal to the rejiggering of form rendering.

I'm not sure I agree. Points 2 and 3 are major components of the the
raison d'être for this refactor.

>> {% form myform %}
>
> For what my vote may be worth, I'm -0 on this...
>
>> {% form myform field birthdate using calendar %}
>
> and -1 on this

You've said you're -0/-1, but you haven't really explained why. Is it
just the preference for using filter-style syntax (which, as I've
shown, has it's own problems), or is there some deeper objection? You
call it syntax soup, but the final syntax is no more than:

{% form A field B using C D ... %}

Of this syntax:
  * "form" identifies the tag

  * "field" specifies a particular rendering action; it's necessary
so that you can also render form errors and form javascript triggers
without introducing a name ambiguity (i.e., is it the form field named
"errors", or a request to display errors"

  * "using" is syntactic sugar; strictly, it could be dropped, but
personally I think it adds to readability.

Everything else is specifically naming an operand, except that I let
you use whitespace, whereas a filter doesn't :-)

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 12, 2010, 9:49:42 AM7/12/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 9:47 PM, Russell Keith-Magee
<rus...@keith-magee.com> wrote:
> On Mon, Jul 12, 2010 at 9:38 PM, Tim Chase
>> Looking back over the thread, I'm the only Tim, but I don't seem to see your
>> response (neither in my email nor via gmane).  If you could disinter it from
>> your sent-mail folder and resend, I'd appreciate reading your thoughts in
>> reply.
>
> Hrm. Gmail says my response was sent, but it's not turning up in the
> public archive -- I'll resend. Apologies if this results in a repost
> for anyone.

Ah - on closer inspection, I sent the original response to
django...@tim.thechases.com. I'm not sure why that address came up
as the default reply-to.

I've reposted to the list for posterity. Apologies for the confusion.

Yours,
Russ Magee %-)

Daniel Greenfeld

unread,
Jul 12, 2010, 9:53:47 AM7/12/10
to Django developers
On Jul 12, 8:12 am, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
> On Mon, Jul 12, 2010 at 10:31 AM, André Eriksson <ean...@gmail.com> wrote:
> > Good proposal overall. One thought I have in order to try and combat
> > the massive parameter list of {% form %} is to optionally add an
> > ending tag, as well as sub-tags:
>
> > {% form myform %}
> >    {% using birthday=calendar %}
> >    {% renderer "as_ul" %}
> >    {% autocomplete "name_autocomplete" %}
> >    {% doctype xhtml1 %}
> > {% endform %}

In django-uni-form (http://github.com/pydanny/django-uni-form) we pass
into the form tag a 'form helper' object that contains attributes as a
parameter. This means that assembling the special qualities of the
form can happen in the view.py or forms.py.

helper = FormHelper()
helper.use_csrf_protection = True

{% uni_form myform helper %}

This approach has been popular in the community that uses django-uni-
form although mostly it is used to add buttons and layout options that
come in django-uni-form. People do really complex forms using the
layout extension of the project (especially using BartTC's Layout
module).

Its possible to extend this approach to include other options and
capabilities and work that into the proposed forms changes:

helper = FormHelper()
helper.doctype = 'xhtml1'
helper.use_csrf_protection = True

{% form myform helper %}
{% form myform errors %}
{% form myform hidden %}
{% form myform field name using autocomplete:"name_autocomplete" %}
{% form myform field birthdate using calendar important %}
{% endform %}

I'm not sure this is the right way to do things here. The FormHelper
object as implemented needs to be controlled outside the templates.
Which possibly defeats some of Russ' goals in this effort. Also, from
experience, when people put a lot of functionality in the FormHelper
object (django-uni-form's Layout module comes to mind), it means that
the layout controls are in view.py, not in the template. Which
arguably means that the controls are out of the hands of the designer
and in the hands of the developer.

Until I get around to refactoring django-uni-form,

Daniel Greenfeld
pydanny.com

Javier Guerra Giraldez

unread,
Jul 12, 2010, 10:28:17 AM7/12/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 8:16 AM, Russell Keith-Magee
<rus...@keith-magee.com> wrote:
> On Mon, Jul 12, 2010 at 11:27 AM, Javier Guerra Giraldez
> <jav...@guerrag.com> wrote:
>> no, the P, UL (and my hypothetical left_table) would each one be a
>> class; you could import each one separately (or maybe several included
>> by default).  in my example, left_table would inherit from as_table,
>> simplifying the implementation.  the {%form%} syntax wouldn't be a
>> parameter to a single renderer, it's a selector to choose which
>> renderer to use.
>
> I'm not sure I understand.
>
> In my proposal, {% load custom_form %} loads a single form template
> tag. That template tag implements a single specific rendering scheme.
> You import library X, you get library X's layout. If you want a
> different layout, you import a different library (namespaces being one
> way to get around the 'multiple layouts in a single )

i didn't think too much about namespaces; maybe that's a cleaner
solution than my proposal.


> How are you getting your "as_p" renderer into your template? How is it
> made available to the {% form %} tag as an argument? How does the
> "selector" determine what is available for selection? What defines the
> default behavior of {% form %} in the first place?

what i think is: currently, the form system has a few renderers as
class methods. the new system would factorize them out as classes.
for this, it would expose a small API, so that the user could easily
write new renderers, ideally even supporting inheritance to make it
easier to just modify existing renderers.

a new renderer class would have to be registered with the form system,
just like a new template tag has to be registered with the template
system. this might be at app loading, or more likely, with {%load%}
tags on the template. you could {%load%} as many renderers you want,
since they don't overwrite any behavior, they don't define a new
{%form%} tag.

the {% form %} tag is provided by the form system, not by the
renderers. there are a few 'included' renderers, which probably you
don't have to load, (just like you don't have to load the standard
template tags). one of these would be the default (just like the
current form system has a default of as_table() )

a parameter to the {%form%} tag (or a subtag, on André's proposal)
would be the 'selector' to choose between the registered renderers.
again, the renderer doesn't override the {%form%} tag; it's the tag
the one that chooses the selected renderer.

--
Javier

Tim Chase

unread,
Jul 12, 2010, 11:01:00 AM7/12/10
to Russell Keith-Magee, django-d...@googlegroups.com
On 07/12/2010 07:03 AM, Russell Keith-Magee wrote:
> On Mon, Jul 12, 2010 at 5:43 AM, Tim Chase
> <django...@tim.thechases.com> wrote:
>> [please excuse the slight chop-up-and-reordering of your
>> original into my quoting]
>
> Only if you grant me the same liberty :-)

Fair's only fair :)

>> and REALLY -1.0e1 on this syntax-soup. Does this result in
>> something that looks like the following monstrosity?
>>
>> {% form myform field name using autocomplete:"name_autocomplete" field
>> birthdate using calendar field other_field using
>> some_other_chrome_or_widget%}
>

> I think some wires have gotten crossed here - this example wouldn't be
> legal syntax under what I'm proposing.
>
> Under the scheme I'm proposing:
> * {% form myform %} would render an entire form, and
> * {% form myform field name ... %} would render a *single* field
> using the strategy described by the form.

Ah, that does clarify (and minimize my strenuous -1.0 objections
back to just a -0 "meh")

>> {{ myform|as_ul }}
>
> If you dig into the archives, this was proposed (I'm fairly certain
> *I* suggested it or supported it at one point); I can't say I can give
> a concrete answer as to why the suggestion wasn't adopted. However, in
> the context of this proposal, I can give some counterreasons -- more
> in a bit.

Okay -- A little improvement on my previously-decaffeinated
google-fu turned up at least one thread[1]...though as you say,
your participation in the thread was "Put me down as a +1"
advocating filters ;-)

>> {{ myform|as_ul }}
>> {{ myform|use_widget:"birthdate=calendar"|as_ul }}
>

> I think this last example should probably be:
>
> {{ myform.birthdate|chrome:"calendar"|as_ul }}
>
> or even:
>
> {{ myform.birthdate|calendar_chrome|as_ul }}
>
> since this enables you to define chrome as a filter. This bit appeals to me.

But I want a BLUE bikeshed ;-) Yeah, I'll give you the point
that your solution looks more elegant for individual field
rendering, while one of my envisioned use cases was "I want this
form exactly as it would normally render, only I want to override
the widget-choice for a particular field". It's the difference
between something like

{{ myform|use_widget:"birthdate=calendar"|as_ul }}

and

<li>{{ myform.name }}</li>
<li>{{ myform.favorite_cheese }}</li>
<li>{{ myform...8 more fields here }}</li>
<li>{{ myform.birthdate|calendar_chrome }}</li>
<li>{{ myform...7 more fields here }}</li>

or perhaps a (forgive the pseudo-code template) slightly less
verbose version:

{% for field in form.fields %}
<li>
{% if field.name == "birthdate" %}
{{ field|calendar_chrome }}
{% else %}
{{ field }}
{% endif %}
</li>
{% endfor %}

> * The first is stylistic. {{ myform|as_ul }} is certainly elegant;
> however, I don't see that {{ myform.birthdate|calendar_chrome|as_ul }}
> is equally elegant. Filter syntax requires that you take out all your
> whitespace, which just seems messy to me. In a template tag, arguments
> are all distinct, ordering is obvious, and when it isn't, you can
> introduce syntactic sugar (like "using") to clarify.

The whitespace issue is a grumble for another day (I can't count
the number of times I've reached to use whitespace for improved
readability in a filter-chain or template, only to be burned by
exploding template shrapnel).

However since you're only dealing with one field and controlling
the rendering yourself (wrapping in <li> tags), it would be
reduced to

<li>{{ myform.birthdate|calendar_chrome }}</li>
<li>{{ myform.name|auto_suggest:"some_view_name" }}</li>

which isn't quite so bad, IMHO. Unless the volume of parameters
to the filter balloons, in which case I can cede you the point.
For most of what I do, the 0-or-1 parameter case is by far the
most prevalent. Individual tweaks such as CSS styles can be done
by identifying a containing tag:

<style>
li#fabulous tr td { background-color: #f0f; }
</style>
...
<li class="spiffy" id="fabulous"
>{{ myform.birthdate|calendar_chrome }}</li>


so I've not really found myself reaching for attribute tweaks
inside rendered controls.

> * Secondly, while I can certainly explain from a programmers
> perspective as_ul formatting and calendar_chrome are applied as
> filters in the order that they are being applied, I can't think of a
> good conceptual explanation, suitable for non-programmers, for why
> form rendering is a 'filtering' activity.

Filters apply transformations of input to output -- the "lower"
filter takes the input (as text) and makes it lower-case; the
$FILTER takes the input and makes it ${FILTER}ed. The "as_ul"
filter takes the input form and makes it <ul>ified; the
calendar_chrome filter takes the input field and makes it
calendar'ified. I don't see this as a particularly big
conceptual jump for non-programmers.

> * Lastly, a template tag gives you access to the context, which means
> tricks like collating javascript triggers becomes possible. For me,
> this is one of the major goals of this refactoring, and it's something
> you can't do with a template tag.

I think this is your biggest win here -- while I'd still advocate
(from your original post)

>>> - Propagating the 'doctype' kwarg through the existing
>>> form/field/widget infrastructure

which would allow smarter rendering.

>> It could even be neatly stacked something like (line-broken
>> for clarity)
>

> ... which, it's worth noting, isn't legal Django template
> syntax. Django's template tags and filters are all limited to
> a single line specifically to avoid the contents becoming
> syntax soup.

Yeah, I know it's invalid syntax and have elided that rant for
another day. Suffice to say that decisions to purposefully make
things less readable (even if they should hopefully be fairly
easy tag/filter-parser tweaks for the most boring cases) in an
effort to prevent long filter-chains makes for a step backwards,
IMHO.

> You've said you're -0/-1, but you haven't really explained
> why. Is it just the preference for using filter-style syntax
> (which, as I've shown, has it's own problems), or is there
> some deeper objection?
>

>>> {% form myform %}
>>
>> For what my vote may be worth, I'm -0 on this...

In the base case above (whole form), I'm not yet seeing value
beyond the traditional

{{ myform }}

thus my -0.

>>> {% form myform field birthdate using calendar %}
>>
>> and -1 on this
>

> You call it syntax soup,

I reserved the "syntax soup" slur for what I misunderstood as the
modification of a *form* in that hideous fashion above (also
referenced as a "monstrosity"), not a *field*. It becomes much
less soupy (it congeals? get strained? like this metaphor?) if
you're only doing a single field. No soup for you!? ;-)

> but the final syntax is no more than:
>
> {% form A field B using C D ... %}

With your clarification that this is a *field* rendering, not a
*form* rendering, I'm more okay with this now. Waffling between
a -0 and a +0, but easily swayed either direction depending on
readability vs. power (yay, whitespace and args/kwargs). I still
begrudge the "form" keyword as misleading (and possibly
redundant) as you're talking about fields. I'd prefer something like

{% field myform.myfield using ... %}
{% field some_formless_field using ... ? ... %}
{% modify_field myform.myfield using ... %}

or some such variant that emphasizes the field-nature instead of
the form-nature. (I don't know if formless fields would ever
reasonably show up in a template, so take that
psychopathical/hypothetical example with a grain of salt)

Thanks for your detailed response and thoughtful arguments.

-tkc

[1]
http://groups.google.com/group/django-developers/browse_thread/thread/5f3694b8a19fb9a1/00a5d83e8c80df6c


André Eriksson

unread,
Jul 12, 2010, 11:00:32 AM7/12/10
to Django developers
On Jul 12, 3:43 pm, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
> > Andre's idea is interesting and is certainly more readable.
>
> I'm having difficulty reconciling these two positions. My template tag
> is too complex because it requires you to remember the idiom FORM X
> FIELD Y USING Z; but a nested tag structure with 4 different subtags
> is more readable and won't require reference to documentation to
> understand how and when to use each subtag?

I think the other approach lends itself to a lot of non-DRY. For
example, if you want to define the chrome of a single field you end up
having to define *every single field* along with it. Example:

{% form myform field A using X Z %}
{% form myform field B %}
{% form myform field C %}
{% form myform field D %}
{% form myform field E %}

My idea with the form block tag was to allow a way to replace the
chrome of a single field without having to define all the other
fields. In other words, the above would just be:

{% form myform %}
{% using A = X Z %} {# Or whatever syntax looks good %}
{% endform %}

And the form would be rendered as {{ myform }} would, except with the
chrome of the field A being replaced with X and Z. Perhaps this wasn't
clear from my proposition.
Message has been deleted

Preston Timmons

unread,
Jul 12, 2010, 12:28:11 PM7/12/10
to Django developers
Hey Russ,

I think this is a great proposal so far!

Is there a way with the proposed solution for the template designer to
add custom attributes to a form field? If so, do you envision that
happening in the chrome layer?

Here's a use case:

The designer wants to render an email input field. They also want to
set the ``autocapitalize`` attribute to "off" so Mobile Safari doesn't
capitalize the first letter of the email while it's being typed. In
addition, they want to set the placeholder attribute.


Internally, we solve this problem through a custom template tag doing
something like this:

Form Template::

{% load form_tags %}

{% with "forms/text_field.html" as text_field_template %}
{% text_field form.email placeholder="Email"
autocapitalize="off" autocorrect="off" %}
{% endwith %}

Field Template::

<span{% if field.errors %} class="field-error"{% endif %}>
{{ field }}
</span>


We use the template tag because calling
``form_field.field.widget.attrs.update(attrs)`` is the only way I know
to get the {{ field }} object to render custom attributes on the input
field from the template. Other use cases involve overriding the label
defined in the form definition and adding custom classes to the input
field.

Thanks,

Preston

Gabriel Hurley

unread,
Jul 12, 2010, 3:03:00 PM7/12/10
to Django developers
Hi all,

I'm certainly excited to see improvements in the form rendering arena,
so thanks Russ for putting in the work here!

I work with Django plenty as a programmer, but in truth I work more as
a designer. And as a designer, I've spent more than my share of time
wrangling Django's existing templating mechanisms into forms which
accomplish the requisite goals. As such, I'd like to chime in on a few
cases that certainly matter to me:

1. Ability to add attributes to individual fields.

Preston raised this concern and I strongly second it. It will only
become more important as HTML5 inputs expand the number of valid
attributes (I believe there are 10 or so, at least). If the new form
rendering can't accommodate this it ends up being a -0 for me since
I'll still be writing tags by hand. And I don't think writing custom
chrome is the place to add these types of attributes. They're
functional, not presentational.

2. Ability to alter individual fields without writing out the entire
form.

Tim has been the main proponent of this one so far, and I'm with him.
In the scope of "perfectionists with deadlines" I don't have time to
write out the entire form just to replace the chrome on a single
field! ;-) Not that this is the right syntax or method, but the idea
of being able to do something like {{ myform|
chrome:"birthdate=calendar" }} is VERY appealing. That said, I do like
the "using" syntax you've proposed.

3. Syntax and Readability.

There's been a lot of discussion on template tag(s) vs. filters, and
while wearing my designer hat I'd say that calling a "form" tag to
render individual fields doesn't make sense. Adding a {% field %} tag
or something of the sort sits better with me.

As for filters, I support them in elegance but not at the expense of
functionality. That said, I don't see any conceptual problem with "|
as_ul" since in my mind filters are simply transformations. Anybody
who's used Photoshop has seen the term "filter" applied in a similar
fashion. This leads me to my last concern, though...

4. Layout renderers being loaded implicitly by form libraries.

I'm a HUGE -1 on this. The choice of "as_ul", "as_p", etc. needs to be
as explicit as possible, and the idea that you could load multiple tag
libraries and accidentally override your form rendering sounds like a
nightmare. So, whether these become filters, or arguments to the {%
form %} tag, I really can't support them being implicit in the form
tag that gets loaded. No magic here, please!


Those concerns aside, I really like the rest of the proposal. This
seems like a great step forward, and the ideas about chrome and
doctype sound excellent to me.

All the best,

- Gabriel Hurley

On Jul 11, 8:36 am, Russell Keith-Magee <freakboy3...@gmail.com>
wrote:
> Hi all,
>
> I'd like to propose a few extensions to Django's form library for 1.3.
> I'm still working on some fine details, but before I get too far, I'd
> like to field opinions so that I can:
>
>  * Discover any edge cases I've missed in my analysis
>  * Field any criticisms from people with more design/frontend
> experience than myself
>  * Determine any related problems that we have the opportunity to
> solve at the same time
>  * Find out if there is anyone in the community who is interested in
> helping out.
>
> Apologies in advance for the length, but there's a lot of detail to cover.
>
> With this proposal, I'd like to address three problems:
>
>  1. The layout problem. Django's forms can be rendered "as_ul",
> "as_table" or "as_p", and that's it. These layout schemes can be
> overridden and customized if you know what you're doing, but it's not
> easy to do so. Furthermore, visual layout concerns aren't separated
> from data processing concerns. You need to write (and install) a form
> subclass to implement your own form layout. Although it's good
> app-writing practice to ensure that forms can be easily substituted,
> it's not an enforced or universal practice.
>
>  2. The widget problem. This is a variant on the previous point. A
> designer that wants to use a specialized calendar widget for a date
> field needs to modify form code. This is a complexity that shouldn't
> exist; a designer should be able to specify the widget library that
> needs to be used (with all it's required rendering requirements,
> javascript triggers etc) without modifying views and form processing
> code.
>
>  3. The DOCTYPE problem. Most importantly, there is the closing slash
> problem, but the introduction of HTML5 also means that there are
> richer input types like <input type="email"> that aren't available in
> HTML4 or XHTML1. Django currently outputs XHTML1 unconditionally, and
> has no support for the new HTML5 input types.
>
> To solve these three problems, I'd like to propose that we add (and
> promote) the use of a new approach to form rendering, based around the
> use of a new {% form %} template tag. This proposal has some
> similarities to a proposal made by in the 1.2 feature phase [1] -- but
> that proposal was only aiming to solve the doctype issue.
>
> [1]http://groups.google.com/group/django-developers/browse_thread/thread...
>
> So: What I'm proposing is that we introduce a new template tag: {% form %}.
>
> How does this solve the three problems?
>
> Layout
> ------
>
> The simplest approach for rendering a form would become:
>
> {% form myform %}
>
> This would effectively implement the as_table rendering strategy, just
> as {{ myform }} does right now.
>
> If we want a different rendering, we exploit the fact that {% load
> %}ing a template library will override any template tags that are
> redefined. {% form %} would be defined as part of the default template
> tag library, implementing the 'as_table' strategy. However, if we load
> a library that also defines the {% form %} tag, that definition will
> override the base definition. If we want to use a custom rendering
> style, we can get that by simply loading a different renderer that
> implements that style:
>
> {% load custom_renderer %}
> {% form myform %}
>
> Django would ship with {% form %} implementations of the 'as_p' and
> 'as_ul' strategies, so getting 'as_p' rendering would mean:
>
> {% load xhtml_p_forms %}
> {% form myform %}
>
> {% form %} is just a template tag, but the default implementation
> would be designed in such a way that it could be easily subclassed to
> alter the rendering strategy for the form. I'm still tinkering with
> details here, but broadly, the intention is to expose a similar
> interface to that used by Form.as_*()  -- that is, returning format
> strings that specify like '<td>%(errors)s%(field)s%(help_text)s</td>'
> to define the rendering strategy; this would be implemented as a
> function so that forms could decide on a per field basis what output
> format is appropriate. However, unlike the existing as_*() approach,
> you don't need to have access to the form in order to use the
> different renderer, which means you can define and apply your
> rendering strategy independent of the view and form code.
>
> Since the form renderer exposes the logic for rendering individual
> form rows, we can also expose the ability to render individual form
> fields, plus the non-field errors and hidden fields:
>
> {% form myform errors %}  -- All the non-field form errors, plus
> hidden field errors
> {% form myform field birthdate %} - output a full row for the
> birthdate field (wrappers, label, errors, help etc)
> {% form myform hidden %} -- output all the hidden fields
>
> This just exposes the internal mechanics that makes the full-form
> rendering of {% form myform %} possible.
>
> Widgets
> -------
>
> The second problem is support for widgets and other rendering
> customization. This can be addressed using extra arguments to the {%
> form %} tag when rendering individual fields:
>
> {% form myform field birthdate using calendar %}
>
> This instructs the rendering of the birthday DateField to use the
> 'calendar' chrome.
>
> What is chrome? Chrome is an attempt to overcome the practical
> limitations of Django's Widgets.
>
> When we introduced newforms, the intention was that widgets would be
> the point at which form rendering would be customized. If a developer
> wanted to use a rich Javascript rendering for the calendar, they would
> define a custom Date widget, override the render() method to introduce
> the appropriate Javascript and CSS hooks, and then define a form with
> fields that specify the use of that widget. The way admin uses widgets
> is probably the best example of how this was intended to work.
>
> However, in practice, widgets aren't used like this. Widgets aren't
> trivial to define, and they aren't easy to deploy, either. The
> convoluted mechanics that ModelAdmin goes through in order to install
> a custom calendar widget is probably the best demonstration of why
> this idea hasn't taken off -- it's just too complex.
>
> It's also not simple to "just use a different widget", either. The
> interplay between form and widget is sufficiently complex that the
> only time at which you can define the widget that is to be used is
> when you define the field itself. This means that widget choice is
> closely bound to form design, which makes it nigh impossible define a
> scheme for deploying widgets as part of the template rendering layer.
>
> So -- the idea behind chrome is to separate the raw HTML input
> mechanism from the bits that make the input look good on the rendered
> page.
>
> Widgets remain as they are, but we de-emphasize their use as a
> rendering customization tool. They become little more than the
> decision over which HTML <input> (or <select>) will be used. It's
> still important that widgets are distinct from fields. After all,
> there are times when there are multiple input options for particular
> field data -- consider the case of a location field, that will require
> a custom lat/long split widget.
>
> It's the job of the chrome to make the <input> provided by the Widget
> look pretty. This might mean adding extra classes to the rendering of
> the <input>; it might mean putting placeholder <div>s or other
> elements around the input; or it might mean registering that a
> javascript block is required to support that input.
>
> Chrome can also be layered. Different pieces of chrome could perform
> different roles, and could be applied one after he other. For example
> in the following:
>
> {% form myform field birthdate using calendar important %}
>
> The "calendar" chrome might add the javascript and classes to enable
> the rich widget, where the "important" chrome might just add CSS
> classes that enables a particular field to be rendered in a particular
> style.
>
> Chrome can also be parameterized; for example:
>
> {% form myform field name using autocomplete:"name_autocomplete" %}
>
> might define a chrome that implements an Ajax autocomplete widget
> using the named URL "name_autocomplete" as a data source. This has to
> potential to start giving an answer to the "Why doesn't Django do
> AJAX" monkey; Django won't provide an AJAX solution out of the box,
> but packaging a chrome that implements AJAX should be a lot easier.
>
> Chrome could also be programatically assigned as part of the form.
> Using a formfield_callback() style setup, it should be possible to
> automatically assign certain chromes to certain field types (e.g.,
> always apply the calendar chrome to DateFields).
>
> I also hope to be able to provide a solution for the problem of
> putting all the javascript for a form at the bottom of the page. At
> present, widget rendering allows you to insert a <script> tag
> immediately before or after a form element, but best practice says it
> should be contained in a single block at the end of your page.
>
> By rendering a widget using a template tag, we get the ability to set
> variables in the context. When a form field is rendered using a
> particular piece of chrome, that chrome can register that it requires
> a specific javascript trigger; a {% form triggers %} call can then be
> used to retrieve all the triggers that have been registered, and embed
> them in a <script> call at the end of the page.
>
> Again, I'm still working on details here, but the intention is to make
> chrome easy to define -- hopefully just a class with some fairly
> simple entry points.
>
> Doctypes
> --------
>
> The third problem is doctype support. To solve this, I propose two changes.
>
>  1) We introduce a 'doctype' argument to the render() method on
> widgets. Widgets would then be expected to generate HTML output that
> is compatible with the supplied doctype. For backwards compatibility,
> internal calls to widget.render() would need to introspect for the
> existence of the 'doctype' kwarg on the specific widget, and raise
> warnings if the kwargs isn't available.
>
>  2) We introduce a number of new default widgets to support the new
> HTML5 input types. At present, EmailField uses a TextInput by default.
> Under the new scheme, we would introduce an EmailInput, and EmailField
> would use EmailInput as the default widget. When rendered using the ...
>
> read more »

Carl Meyer

unread,
Jul 12, 2010, 3:24:17 PM7/12/10
to Django developers
Hi Russ,

First of all, thanks very much for this proposal! Form rendering has
been a major pain point for us (thus the existence of
django-form-utils), and improving it is tops on my 1.3 wishlist. I
will also be at DjangoCon and eager to sprint in this area.

Django has a really good template system. With template inheritance,
includes, and overrides, it allows the designer I work with a
remarkable amount of flexibility in producing exactly the HTML he
wants, without repeating himself. He's not a programmer (and thus
doesn't follow django-developers), but he works full-time with the
Django template system and loves it. His (frequent) complaints to me
about Django form rendering are always the same: "Why is this markup
generated deep inside Python code, and why can't I override a template
to fix it? I thought we'd moved on from PHP!"

I think it's possible to solve the problems you're aiming at here
without so much new Python machinery, just by delegating more to
templates (as Eric and Danny have already mentioned briefly). HTML is
the native language of web designers, and I'm firmly convinced that
the less we hide their HTML behind Django-specific abstractions, the
more usable the system will be.

I'll first outline in brief what I think a fully template-oriented
approach might look like, address some possible concerns with it, and
follow that with some specific comments on your proposal.

A fully template-based approach to form rendering
=================================================

You have a form object in your template. You render it using a simple
"render_form" filter:

{{ form|render_form }}

"render_form" just renders a default form rendering template (at some
sensible template path, the bikeshed could be painted
"django/forms/default.html" or some such), with the form object itself
passed into the context.

This template just does the usual iterating over form fields, and the
default one provided by Django could mimic form.as_table(). That
default template could of course be overridden for any given project,
and the "render_form" filter also accepts as argument an alternative
template to use for a particular form:

{{ form|render_form:"path/to/my/form_template.html" }}

So far, this approach is workable in current Django. This is what
django-form-utils does, and what we do in all our projects.

The next step is to have the default widgets render themselves using
templates found at a standard path
(e.g. "django/forms/widgets/textinput.html"). This template would be
passed the boundfield and widget, and so would have access to all the
information it needs to render itself correctly.

There would also be a "render_field" template filter, that would
optionally accept an arbitrary widget template path:

{{ form.fieldname|render_field:"path/to/my/widget_template.html" }}

Normally this would only be used inside a form rendering template, but
if you wanted to skip the "render_form" filter and render your form
directly using this filter repeatedly (or inside a loop), that's
possible too.

All of the problems you identified as targets are now trivially
solvable, just by giving the designer direct access to the template
system, without all kinds of magical abstractions in between. Form
layouts can be fully controlled in templates. Widget markup can be
fully controlled in templates (and of course can validate to whatever
doctype you want it to, without any special machinery for that).

The last target you identified, allowing widget-related JS to be
rendered in a block at the bottom of the template, can be solved
simply with another filter and set of templates:

{{ form|render_form_js }}

Which would look for templates such as
"django/forms/widgets/js/textinput.html" for each widget in the form,
rendering nothing for a widget if that template doesn't exist. This
filter could also optionally accept a path to a template containing
form-wide JS. These templates would probably just contain script tags,
either linking to an external JS file or containing inline JS. (IMO,
collating these into a single script tag is out of scope and better
handled by a dedicated solution like django-compressor; though the
possibility of referencing a single form-wide JS template opens the
door to other possible solutions).

Advantages
----------

- As mentioned, HTML is the native language for a web designer: this
exposes it directly and simply.

- Similarity to existing approach: this is a less radical
change. Rather than introducing a new template tag that does all
kinds of magic behind the scenes, and several new abstractions
(renderers, chrome) to boot, this just leverages existing, proven,
well-understood tools, and the resulting templates look more similar
to existing form-rendering templates.

- No need for special-casing things like doctypes, adding attributes
directly to a widget-generated HTML input tag. Everything is in
templates, everything is equally customizable via that familiar
route.

Possible objections
-------------------

- Russ mentions the "complexity" of "getting all the template paths
lined up." I'm not sure what the issue is here; Django template
overrides use a simple path-based scheme, and it's a proven system
that works. Designers use it successfully all the time. I think
introducing new complexity in the form of new Python abstractions is
more likely to be a real problem.

- Concerns about filter syntax. The first two concerns Russ raised are
only issues if many filters have to be strung together, which this
proposal does not require. The third objection is predicated on the
necessity of doing tricks with the context to load widget JS; again,
a template-based solution doesn't need this.

- "Too much work to override all these widget templates just to switch
doctypes!" Again, I think the complexity of dedicated Python
machinery is likely to be worse. Django's philosophy (and James
Bennett's philosophy of reusable app templates, referenced above)
assume that templates should be per-project: why should forms or
widgets be different? Packaging up a reusable app with a set of
generic "HTML5" or "XHTML1" widget templates is almost certainly
simpler (and less bug prone) than writing the equivalent Python code
for the customized {% form %} tag renderer.

- Possible performance issues with rendering too many templates. I
think for many people this will not in fact be a problem, and for
those for whom it is, we now have the cached template loader, which
should make it disappear.

Specific comments on Russ' proposal
===================================

- Using {% load %} to import new renderers is insufficiently flexible
(as has already been noted). Templatetag namespacing might help with
this, and is a good idea in its own right; but IMO this is still
more complex and less flexible than the template-based approach.

- I'm skeptical that the "chrome" abstraction buys very much in
exchange for the overhead of the new concept and the syntactical and
registration problems. As mentioned above, in practice I don't think
layering different chromes on top of each other will work very
often; they'll frequently have to be manually integrated into a new
unified chrome anyway.

I'm also concerned that it's still insufficiently flexible; the core
input tag markup will still be hardcoded in the widget, thus
requiring special-cases for anything in it that you might want to
tweak (doctype, extra classes or attributes). This is the big one
for me, as it means our core problem with the current system (markup
hardcoded in Python code) would not be fully fixed.

In contrast, being able to directly edit widget templates for your
specific cases is simpler, more intuitive for designers, and fully
flexible.

- Big +1 to adding new widgets for new HTML5 input types! This is a
necessity for good HTML5 support regardless of which approach is
used for rendering.

Again, thanks for kickstarting the discussion with a
well-thought-through proposal! In the end if the renderer/chrome
proposal is implemented, I think we'd probably still be able to
implement much of the template-based system as a custom renderer (as
you mentioned), except for the problem with core widget rendering. I'm
just not sure what practical benefits would be gained over just
implementing the template-based system directly.

cheers,

Carl

Idan Gazit

unread,
Jul 12, 2010, 6:25:24 PM7/12/10
to Django developers
Russ/Carl:

Finally got a chance to catch up on the thread, and found Carl penned
a (lovely, much more detailed) version of what I had in mind.

In the end, forms is a repository of unusually common fail because
designers must figure out Python and a lot of how django works in
order to customize forms. One of django's strongest assets is the
separation-of-roles philosophy, and this is a common use-case where
that philosophy is not implemented.

As a general design goal, improving forms should be about moving
Django towards a state where designers do not need to write python in
order to customize how forms render.

A big +1 on the template approach.

-I

Tai Lee

unread,
Jul 13, 2010, 2:03:48 AM7/13/10
to Django developers
I really like the idea of using Django templates instead of generating
markup in Python, which should enable designers (who don't know and
don't want to know Python) to customise markup in almost any way they
like. We just need to add a few hooks that will make it easier to
override specific templates (per form, per page, per app, rather than
per project).

To this end, I'd like to propose a new stackable block tag (let's call
it `templatedirs` and `endtemplatedirs` for now) that designers can
use to signal that any templates loaded within the block tag should be
tried from the specified folder first.

For example, if {{ myform }} renders "forms/form.html" by default, and
{{ myform.mydatetimefield }} renders "forms/datetime.html" by default,
then:

{% templatedirs "html401strict" %}
{{ form }}
{% endtemplatedirs %}

would try to load "html401strict/forms/form.html" before trying "forms/
form.html", and:

{% templatedirs "html401strict" %}
{% templatedirs "fancyforms" %}
{{ myform.mydatetimefield }
{% endtemplatedirs %}
{% endtemplatedirs %}

would try to load "html401strict/fancyforms/forms/datetime.html",
"fancyforms/forms/datetime.html" and "html401strict/forms/
datetime.html" before trying "forms.datetime.html".

This would allow flexible overriding of templates from inside
templates themselves, with whatever granularity is required by the
designer, without the designer needing to learn Python or register
template tags.

Templates for a specific doctype could be applied to an entire site by
adding {% templatedirs "html401strict" %} to the base template, and
adding the appropriate templates to the "html401strict" folder in your
app or project templates folder. Custom templates could be applied to
a specific page or a specific form or a specific form element the same
way.

I'm sure this could be useful outside of the forms framework, too. For
example, {% templatedirs request.user.profile.client.slug %}...{%
endtemplatedirs %} to render customised templates for each client (or
category of user, e.g. staff, admin, client, etc.)

Cheers.
Tai.

mattim...@gmail.com

unread,
Jul 13, 2010, 8:40:13 AM7/13/10
to Django developers


On Jul 12, 11:43 pm, Russell Keith-Magee <russ...@keith-magee.com>
Your proposal really needs to cater to two different audiences:

1. People who will use the new {% form myform %} where they just want
all the fields rendered without any fuss, much like {{ form }} now.
2. The tweakers that need to control every aspect of the each field
being rendered.

I probably fall into the latter category. I also have separate logic
for handling form and non-form errors.

Appropriate naming of the new tags is another shed to paint.

> > Having written some
> > complex form field rendering tags for extjs/sencha I can say that
> > without some kind of built-in *args/**kwargs-like parameter handling
> > for template tags, then having more than a handful of parameters gets
> > long and messy quickly.
>
> I'll concede args/kwargs handling in template arguments isn't pretty.
> My counterargument would be that I don't see the use case for lots of
> argument and kwargs to chrome. "Use this calendar", or "show this
> decoration", not "here are 5 different arguments to configure how this
> chrome will operate".
>
> The one use case I can think of for chrome arguments is to provide the
> named URL for AJAX callbacks. However, this may just be a case of me
> not being imaginative enough. If you can provide a use case, I'll
> happily reconsider my position.

I currently pass arguments to my custom tags that allow me to override
the field label, ajax callback, default field value, url for adding
another object (like FK's in admin), and more. Not having args/kwargs
is biting me and i've been too lazy to address it in my own code and I
have to handle cases where args are positional and args 3 and 4
optional but I have to pass an "empty" arg 3 to be able to pass a
valid arg 4.

>
> > Also, it doesn't make your tag that user
> > friendly either when they have to keep referring to the docs to figure
> > out how to use it.
>
> > Andre's idea is interesting and is certainly more readable.
>
> I'm having difficulty reconciling these two positions. My template tag
> is too complex because it requires you to remember the idiom FORM X
> FIELD Y USING Z; but a nested tag structure with 4 different subtags
> is more readable and won't require reference to documentation to
> understand how and when to use each subtag?

The point I am trying to make is that overloading a single tag with
such diverse functionality (that is likely to grow and probably father
more arguments) would be better served by splitting that functionality
into separate tags.

regards

Matthew

Tim Chase

unread,
Jul 13, 2010, 10:53:26 AM7/13/10
to django-d...@googlegroups.com
> Your proposal really needs to cater to two different audiences:
>
> 1. People who will use the new {% form myform %} where they just want
> all the fields rendered without any fuss, much like {{ form }} now.
> 2. The tweakers that need to control every aspect of the each field
> being rendered.

I'd say there's a 3rd subset of folks (Gabriel Hurley and myself
have voiced advocacy for this camp) who would be interested in a
middle-road "mostly the default form, only override the behavior
for fields X & Y". Andre's proposed {% form myform %}...{%
endform %} syntax does this nicely, making it cleaner by removing
the redundancy of referencing the form in each field-modifier and
without the need to manually render each field.

-tkc

Russell Keith-Magee

unread,
Jul 13, 2010, 10:53:34 AM7/13/10
to django-d...@googlegroups.com

Ok - now I see where you're driving at. I wasn't trying to hit this use case.

One of the historical traps that form rendering discussions have
fallen into is the idea that somehow we can come up with the One True
Rendering Implementation; if we just add enough options and flags, we
will be able to programatically define all the fieldsets, field
alignments, and individual per-field layout options you could ever
want.

The core team has generally rejected these proposals, for the same
reason we have rejected ORM extensions that expose edge cases of SQL.
We have a language for describing form layouts -- it's called HTML.
Trying to come up with programatic abstractions of HTML just leads to
leaky abstractions.

I know that's not what you're proposing here, but I'm wary of getting
into territory where it looks like that might be what we're willing to
support. By tackling the form layout problem as a set of rules for
rendering individual fields, it leaves no doubt that you're *supposed*
to manually layout forms if you have rendering requirements that step
outside the simplest possible case.

I'd also like to think that it would be easy enough to write a chrome
package in a way that doesn't require "if field.name == birthdate"; I
would have thought that chrome could generally be applied on a
per-field-type, rather than per-field-name basis (ie. apply calendar
to all dates, rather than having to specifically name the date
fields). However, absent of an actual implementation, it's difficult
for me to actually assert this.

>> but the final syntax is no more than:
>>
>> {% form A field B using C D ... %}
>
> With your clarification that this is a *field* rendering, not a *form*
> rendering, I'm more okay with this now.  Waffling between a -0 and a +0, but
> easily swayed either direction depending on readability vs. power (yay,
> whitespace and args/kwargs).  I still begrudge the "form" keyword as
> misleading (and possibly redundant) as you're talking about fields.  I'd
> prefer something like
>
>  {% field myform.myfield using ... %}
>  {% field some_formless_field using ... ? ... %}
>  {% modify_field myform.myfield using ... %}
>
> or some such variant that emphasizes the field-nature instead of the
> form-nature.  (I don't know if formless fields would ever reasonably show up
> in a template, so take that psychopathical/hypothetical example with a grain
> of salt)

I toyed with using multiple tags; I ended up proposing a single tag to
keep the internal implementation simple. Given that {% field %} needs
to use the per-row logic defined as part of {% form %}, using a single
tag with a qualifying argument was the easiest way I could see to
avoid duplicating logic. However, there are some other ways that this
could be achieved.

> Thanks for your detailed response and thoughtful arguments.

Likewise.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 13, 2010, 10:56:23 AM7/13/10
to django-d...@googlegroups.com
On Mon, Jul 12, 2010 at 11:29 PM, André Eriksson <ean...@gmail.com> wrote:
> It appears my reply got eaten so I'm trying again.
>
> On Jul 12, 3:43 pm, Russell Keith-Magee <russ...@keith-magee.com>

> wrote:
>> I'm having difficulty reconciling these two positions. My template tag
>> is too complex because it requires you to remember the idiom FORM X
>> FIELD Y USING Z; but a nested tag structure with 4 different subtags
>> is more readable and won't require reference to documentation to
>> understand how and when to use each subtag?
>
> My biggest issue with using {% form myform field A %} to render a
> single field is the non-DRY-ness of it all, especially if you have a
> semi-large form. The issue being that if one field has its chrome
> changed, all the other fields must be hand-defined as well. Example:
>
> {% form myform field A using X Y %}

> {% form myform field B %}
> {% form myform field C %}
> {% form myform field D %}
> {% form myform field E %}
> ...
>
> My proposition was to let the {% form myform %}....{% endform %} block
> tag render the whole form if desired, akin to how {{ myform }} works
> today. In this scenario, {% using %} would override the chrome of the
> field without affecting its order in the form. As such, the above
> becomes:
>
> {% form myform %}
>    {% using A = X Y %} {# Or some other syntax #}
> {% endform %}
>
> Perhaps this wasn't clear enough from my original proposition.

It wasn't in the direction I was originally thinking, so there was
some confusion on my part. I have a better understanding of what you
(and Tim) proposing now, and I've responded with some of my reasoning
in my response to Tim.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 13, 2010, 11:02:19 AM7/13/10
to django-d...@googlegroups.com
On Tue, Jul 13, 2010 at 12:28 AM, Preston Timmons
<preston...@gmail.com> wrote:
> Hey Russ,
>
> I think this is a great proposal so far!
>
> Is there a way with the proposed solution for the template designer to
> add custom attributes to a form field? If so, do you envision that
> happening in the chrome layer?

Under my proposal as I presented it -- yes, it's a chrome issue.
Chrome is essentially a set of operations that can be performed on the
raw widget output; adding attributes, classes, wrapping the widget
HTML in other HTML, or registering the need for trigger Javascript.

As a helper mechanism, we could ship a generic "attribute" chrome that
allows you to set or override a specific attribute:

{% form myform field name using attr:"autocorrect=off" %}

Adding classes could also be handled as chrome:

{% form myform field name using class="colorful large" %}

Since chrome can be stacked, you could have multiple attr, class, or
any other chrome.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 13, 2010, 11:15:08 AM7/13/10
to django-d...@googlegroups.com
On Tue, Jul 13, 2010 at 3:03 AM, Gabriel Hurley <gab...@gmail.com> wrote:
> Hi all,
>
> I'm certainly excited to see improvements in the form rendering arena,
> so thanks Russ for putting in the work here!
>
> I work with Django plenty as a programmer, but in truth I work more as
> a designer. And as a designer, I've spent more than my share of time
> wrangling Django's existing templating mechanisms into forms which
> accomplish the requisite goals. As such, I'd like to chime in on a few
> cases that certainly matter to me:
>
>  1. Ability to add attributes to individual fields.

> And I don't think writing custom


> chrome is the place to add these types of attributes. They're
> functional, not presentational.

As noted in my reply to Preston, this should easily possibly using
chrome; I'm not sure I see why you disagree. The new options may be
funcitonal, but from my reading of the HTML5 spec, they're functional
in terms of UX, not in terms of data modification. The widget and
field are used to process and parse the content of any value that is
POSTed by a form; anything that the user sees about a form is a
presentational issue, AFAICT.

>  2. Ability to alter individual fields without writing out the entire
> form.

I missed the focus of Tim et al's proposals on this front; I can now
see what he's driving at. To that end, I can see the appeal of a {%
form %} syntax that allows for overriding individual field settings.

> 3. Syntax and Readability.
>
> There's been a lot of discussion on template tag(s) vs. filters, and
> while wearing my designer hat I'd say that calling a "form" tag to
> render individual fields doesn't make sense. Adding a {% field %} tag
> or something of the sort sits better with me.

Sure; I've admitted elsewhere in this thread that my use of a single
tag was mostly driven by a desire to keep the implementation simple.
This was especially important keeping in mind that my original intent
was that extension would be largely in code; this becomes less of an
issue if templates become a larger part of the common usage of this
new form rendering approach.

> 4. Layout renderers being loaded implicitly by form libraries.
>
> I'm a HUGE -1 on this. The choice of "as_ul", "as_p", etc. needs to be
> as explicit as possible, and the idea that you could load multiple tag
> libraries and accidentally override your form rendering sounds like a
> nightmare. So, whether these become filters, or arguments to the {%
> form %} tag, I really can't support them being implicit in the form
> tag that gets loaded. No magic here, please!

I think I understand the nature of your concern, but I'm not sure if I
agree with calling this magic. You import a library that exercises the
defaults you want to use. Library loads only survive for the duration
of a single template, so there isn't any risk of a load in one
template file accidentally changing behavior elsewhere.

That said, the block-based {% form %} rendering proposal potentially
gets around this by opening the possibility of making the layout
selection a subtag that is explicitly specified in the context of a
specific {% form %} instance.

Thanks for the feedback, Gabriel.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 13, 2010, 12:11:48 PM7/13/10
to django-d...@googlegroups.com
On Tue, Jul 13, 2010 at 3:24 AM, Carl Meyer <carl.j...@gmail.com> wrote:
> Hi Russ,
>
> First of all, thanks very much for this proposal! Form rendering has
> been a major pain point for us (thus the existence of
> django-form-utils), and improving it is tops on my 1.3 wishlist. I
> will also be at DjangoCon and eager to sprint in this area.

Excellent. Sounds like we might have a posse of form wranglers forming.

> I'll first outline in brief what I think a fully template-oriented
> approach might look like, address some possible concerns with it, and
> follow that with some specific comments on your proposal.
>
> A fully template-based approach to form rendering
> =================================================

An interesting counterproposal Carl; thanks for the input.

> Possible objections
> -------------------
>
> - Russ mentions the "complexity" of "getting all the template paths
>  lined up." I'm not sure what the issue is here; Django template
>  overrides use a simple path-based scheme, and it's a proven system
>  that works. Designers use it successfully all the time. I think
>  introducing new complexity in the form of new Python abstractions is
>  more likely to be a real problem.

To explain my concerns, I perhaps need to explain my motivation better.

When I introduced the Media framework 3 years ago, I had something
very specific in mind: I wanted to start a marketplace for reusable
widgets.

I have exactly 0 mad skillz as a front end guy, but I know that rich,
pretty ajax widgets et al are necessary for a good user experience. My
goal, as a backend guy with limited frontend skills, was to find a way
to get the people who *did* have frontend skills to be able to package
their widgets in such a way that I could reuse them.

Before the design community starts to think that I'm trying to
minimize the importance of what they do -- I know that there's a lot
more to the design process than this. I'm just looking at this from a
prototyping perspective -- I should be able to get a decent calendar
or AJAX autocomplete onto a page without much more effort than it
takes to put a simple text field on the page. The legacy of "How does
Django do AJAX" questions on django-users is the more public face of
this desire.

Now, the Media framework has been largely a flop in this regard
(although we may be able to use bits of it in a support role in this
new form work). However, my desire for a widget library is unquenched
:-)

My concern about doing this entirely in templates is that it makes the
process of sharing a widget library a little more difficult. If it's
just code, then it just needs to be in the PYTHONPATH. If there are
templates too, then the templates need to be somewhere that they can
be accessed.

There's also more to a widget in this context than just the template
-- for example, there's nominating the javascript triggers that the
widget requires.

I'm not saying that these problems can't be overcome -- just that it's
a complexity that I want to make sure is covered.

> - Concerns about filter syntax. The first two concerns Russ raised are
>  only issues if many filters have to be strung together, which this
>  proposal does not require. The third objection is predicated on the
>  necessity of doing tricks with the context to load widget JS; again,
>  a template-based solution doesn't need this.

The filter syntax doesn't hit the 'render the form completely standard
except for this one change on this one field' use case. However,
there's nothing about the 'template' aspects of your proposal that
need to be bound to filters; they could be equally accomodated in a
template tag.

> - "Too much work to override all these widget templates just to switch
>  doctypes!" Again, I think the complexity of dedicated Python
>  machinery is likely to be worse. Django's philosophy (and James
>  Bennett's philosophy of reusable app templates, referenced above)
>  assume that templates should be per-project: why should forms or
>  widgets be different? Packaging up a reusable app with a set of
>  generic "HTML5" or "XHTML1" widget templates is almost certainly
>  simpler (and less bug prone) than writing the equivalent Python code
>  for the customized {% form %} tag renderer.

My manifestation of this problem is slightly different -- it's the
issue of how to ship a widget library that can be used anywhere. I
suppose one argument here is that you just need to advertise your
chrome with doctype support notes.

Alternatively, since this is a template language, you could always
introduce an {% if %} block and render conditionally on doctype...

> - I'm skeptical that the "chrome" abstraction buys very much in
>  exchange for the overhead of the new concept and the syntactical and
>  registration problems. As mentioned above, in practice I don't think
>  layering different chromes on top of each other will work very
>  often; they'll frequently have to be manually integrated into a new
>  unified chrome anyway.

This depends a little on the level at which chrome is implemented.
Very low level "modify attribute" and "add class" chrome could easily
be layered, and would address the common request of "how do I add
class X to my widget". More complex chrome (like the skeleton for a
calendar widget) would require more markup, and probably wouldn't be
mixed with other complex chromes, but it might still be mixed with a
simple chrome to add a class or an attribute (or a collection of
attributes and classes packaged into a single chrome).

>  I'm also concerned that it's still insufficiently flexible; the core
>  input tag markup will still be hardcoded in the widget, thus
>  requiring special-cases for anything in it that you might want to
>  tweak (doctype, extra classes or attributes). This is the big one
>  for me, as it means our core problem with the current system (markup
>  hardcoded in Python code) would not be fully fixed.
>
>  In contrast, being able to directly edit widget templates for your
>  specific cases is simpler, more intuitive for designers, and fully
>  flexible.

Classes and attributes on the base widget are something that could be
modified with chrome. If you look at the API for widget.render() right
now, it's pretty flexible -- the issue is that there's no programatic
way to get at that API from a template. Chrome is the way into that
existing API, as well as providing a way to wrap decorating HTML and
Javascript around the base widget.

What exactly is your use case for something that designers want to
customize in the raw widget that widget.render() doesn't expose?

Yours,
Russ Magee %-)

Carl Meyer

unread,
Jul 13, 2010, 3:55:16 PM7/13/10
to Django developers
Hi Russ,

On Jul 13, 12:11 pm, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
> I have exactly 0 mad skillz as a front end guy, but I know that rich,
> pretty ajax widgets et al are necessary for a good user experience. My
> goal, as a backend guy with limited frontend skills, was to find a way
> to get the people who *did* have frontend skills to be able to package
> their widgets in such a way that I could reuse them.

Indeed, I share that goal.

> Now, the Media framework has been largely a flop in this regard
> (although we may be able to use bits of it in a support role in this
> new form work). However, my desire for a widget library is unquenched
> :-)

From my perspective as a programmer, I think the Media framework is
brilliant. It just has one gigantic Achilles heel: it requires
touching Python code.

Honestly, I think if we're serious about wanting a vibrant ecosystem
of Django-enabled frontend widgets, design goal #1 should be that
creating a widget requires touching zero Python code. And I think
that's possible to achieve.

> My concern about doing this entirely in templates is that it makes the
> process of sharing a widget library a little more difficult. If it's
> just code, then it just needs to be in the PYTHONPATH. If there are
> templates too, then the templates need to be somewhere that they can
> be accessed.

So I think this is very much a programmer's perspective, almost
humorously so :-) "Just code" may seem easy to you, but from the
perspective of at least one experienced Django-template designer I've
talked to, being able to do it entirely in templates makes the idea of
sharing a widget library conceivable, as opposed to almost impossibly
intimidating if its "just code."

The app_directories template loader is installed by default. Getting
templates "somewhere they can be accessed" means creating a directory
on the PYTHONPATH with an empty models.py, an empty __init__.py, and a
"templates/" subdirectory, then putting that in INSTALLED_APPS. That's
it. How is that harder than following exactly the same steps for an
app that includes Python code? From the perspective of a non-
programmer, it's much easier.

> There's also more to a widget in this context than just the template
> -- for example, there's nominating the javascript triggers that the
> widget requires.

Yep, I had a lengthy conversation with Idan in IRC yesterday about how
to approach this. I touched on it in my proposal above briefly (it can
still just be handled with templates), but didn't discuss details like
the fact that a given widget's "JS/media" template should only be
included once on a page, regardless of how many times that widget was
used on the page, even in multiple forms. Which means that the
"render_form_js" filter probably actually needs to be a tag, so it can
accept an arbitrary number of form objects and render the widget JS
once for every widget in every form.

> I'm not saying that these problems can't be overcome -- just that it's
> a complexity that I want to make sure is covered.

Indeed. I think they can be overcome, but I fully agree that the
template-based proposal needs more fleshing-out to make sure it covers
all these use cases adequately.

> The filter syntax doesn't hit the 'render the form completely standard
> except for this one change on this one field' use case. However,
> there's nothing about the 'template' aspects of your proposal that
> need to be bound to filters; they could be equally accomodated in a
> template tag.

I agree that tag vs filter is an orthogonal question, and I'm not
opposed to converting them to tags. I agree that it may be necessary
in order to add an extra optional argument or two to handle some use
cases.

The default-except-for-this-one-field use case is actually quite
doable under the proposal as written; you'd just create a derivative
form-rendering template that inherits your default one, and just
override a small block of it to add a bit of conditional logic to
render a certain field differently, otherwise deferring to
block.super. That's DRY and reasonably easy, but I if render_form were
converted to a tag it might be possible to add some optional arguments
to it to make it even easier.

The other improvement I would probably want to make to the API as I
first proposed it would be a way to point a given form-render or field-
render at a given directory of widget-templates, with the individual
template still picked out by widget-name.

> My manifestation of this problem is slightly different -- it's the
> issue of how to ship a widget library that can be used anywhere. I
> suppose one argument here is that you just need to advertise your
> chrome with doctype support notes.
>
> Alternatively, since this is a template language, you could always
> introduce an {% if %} block and render conditionally on doctype...

Exactly. I can imagine a number of different ways a reusable widget
library might approach this problem: simply advertise support for a
certain doctype (don't discount the value of supporting this low-
barrier-to-entry option, even if it isn't ideally reusable), or use
template conditionals, or provide separate widget templates for
separate doctypes... I find it preferable to make the full power of
the template language available, and then let template authors use the
tools they know to iterate towards best practices in this area, rather
than hardcoding into Django particular ideas of which doctypes are
relevant and what changes have to be made in templates to support
them. It's not just a matter of closing slashes, also which attributes
are valid (of which there may be more added later that we don't even
know about now)... I think hardcoding current assumptions about
doctypes is just asking for maintenance headaches. I'd much rather
leave that concern where it belongs, fully in the hands of template
authors.

> > - I'm skeptical that the "chrome" abstraction buys very much in
> >  exchange for the overhead of the new concept and the syntactical and
> >  registration problems. As mentioned above, in practice I don't think
> >  layering different chromes on top of each other will work very
> >  often; they'll frequently have to be manually integrated into a new
> >  unified chrome anyway.
>
> This depends a little on the level at which chrome is implemented.
> Very low level "modify attribute" and "add class" chrome could easily
> be layered, and would address the common request of "how do I add
> class X to my widget". More complex chrome (like the skeleton for a
> calendar widget) would require more markup, and probably wouldn't be
> mixed with other complex chromes, but it might still be mixed with a
> simple chrome to add a class or an attribute (or a collection of
> attributes and classes packaged into a single chrome).

Of course, if template authors can override and edit the widget HTML
directly, they will never need to ask how to add class X to their
widget. Whereas if it's exposed via its own Django-specific
abstraction, we can be sure we'll still be seeing that question for
years to come. Finally having an answer ("well, you write some Python
code and some HTML and package it up as this thing called a 'chrome',
then you apply it using this new template tag API we just invented")
is a distant second-best to making the question unnecessary.

> >  I'm also concerned that it's still insufficiently flexible; the core
> >  input tag markup will still be hardcoded in the widget, thus
> >  requiring special-cases for anything in it that you might want to
> >  tweak (doctype, extra classes or attributes). This is the big one
> >  for me, as it means our core problem with the current system (markup
> >  hardcoded in Python code) would not be fully fixed.
>
> >  In contrast, being able to directly edit widget templates for your
> >  specific cases is simpler, more intuitive for designers, and fully
> >  flexible.
>
> Classes and attributes on the base widget are something that could be
> modified with chrome. If you look at the API for widget.render() right
> now, it's pretty flexible -- the issue is that there's no programatic
> way to get at that API from a template. Chrome is the way into that
> existing API, as well as providing a way to wrap decorating HTML and
> Javascript around the base widget.
>
> What exactly is your use case for something that designers want to
> customize in the raw widget that widget.render() doesn't expose?

Only one off the top of my head: modifying the actual rendered input
type with no Python-level changes. If we'd had full template-level
widget customization three or four months ago, my designer could have
swapped in an "email" input type for a "text" input type where
appropriate, fully at the template level, as soon as browser support
for it appeared, with no need for a new Python form widget or any
other Python-level changes. If you ask him, he will tell you in no
uncertain terms that this is something he ought to be able to do as a
front-end engineer. In fact, I just did ask him. His response: "Form
inputs are HTML. HTML is my job. In most areas, Django makes it really
easy for me to do my job. What's the motivation for making it hard
here?"

The key issue is familiarity and ease-of-use. For template authors,
not programmers. Template authors know perfectly well how to add
classes and attributes to HTML elements. Why should they have to use a
special API to do the same thing for form input elements? Again: if
the goal is for Django template authors to create an ecosystem of
reusable widgets, the system needs to be geared around the tools they
are comfortable with, not the ones programmers are comfortable with.
That means templates. And not Python code.

cheers,

Carl

Gabriel Hurley

unread,
Jul 13, 2010, 4:00:57 PM7/13/10
to Django developers
Thanks for the thoughtful reply, Russ! Just a couple quick points:

> As noted in my reply to Preston, this should easily possibly using
> chrome; I'm not sure I see why you disagree. The new options may be
> funcitonal, but from my reading of the HTML5 spec, they're functional
> in terms of UX, not in terms of data modification. The widget and
> field are used to process and parse the content of any value that is
> POSTed by a form; anything that the user sees about a form is a
> presentational issue, AFAICT.

Having now read that reply, I see what you mean about the chrome. I
hadn't understood chrome as being something that could be "layered".
It sounded more like your idea was "here is a complete calendar UI
solution" and that to add new attributes you'd essentially have to
subclass/rewrite the whole piece of chrome. If they're "stackable",
I'm all for it.

> > So, whether these become filters, or arguments to the {%
> > form %} tag, I really can't support them being implicit in the form
> > tag that gets loaded. No magic here, please!
>
> I think I understand the nature of your concern, but I'm not sure if I
> agree with calling this magic. You import a library that exercises the
> defaults you want to use. Library loads only survive for the duration
> of a single template, so there isn't any risk of a load in one
> template file accidentally changing behavior elsewhere.

The case where it becomes "magic" is if tag library authors start
including these renderers with bundles of other tags... so you go to
load up your favorite handy tag library, and you forget that it also
comes with a custom renderer, and suddenly you've got a mess on your
hands because it replaced your explicitly-loaded renderer. Admittedly
it's not the most common case, but it still concerns me.

> That said, the block-based {% form %} rendering proposal potentially
> gets around this by opening the possibility of making the layout
> selection a subtag that is explicitly specified in the context of a
> specific {% form %} instance.

That certainly seems reasonable to me.

> Thanks for the feedback, Gabriel.

Thank you!

- Gabriel

Nick Phillips

unread,
Jul 13, 2010, 5:58:06 PM7/13/10
to django-d...@googlegroups.com
On Wed, 2010-07-14 at 00:11 +0800, Russell Keith-Magee wrote:

> What exactly is your use case for something that designers want to
> customize in the raw widget that widget.render() doesn't expose?

That reminds me of one thing I'd very much like to see in any
refactoring of form/widget handling.

I'd like to be able to get at the "rendered" data (I say rendered, what
I suppose I really mean is "transformed" either due to rendering needs
or conversion from a graphical representation in the widget to something
that could be resubmitted), without any of the surrounding chrome. There
are various cases in which this can be useful, the most obvious of which
is to tell whether the data in the form has been edited before
submission. It's currently not possible to work this out from the
"before" and "after" data, as you don't know how any particular field's
data might have been munged by the widget (e.g. how is empty data
rendered, any forcing to unicode, Python boolean to textual
representation, date format transformation etc.).

This would make solving problems like those in #11303 simple. As another
example, changeset 10757 is heading down this path to fix #10288 in
adding the _format_value method. I'd suggest that this be generalised
and applied to all widgets (and the _format_value or whatever you want
to call it to lose the "_" and be documented as part of the widget API).

In one project here, I've had to write "duplicate" versions of all
widgets to add a "disable" method and enable them to disable themselves
in the UI while still submitting data (actually I'd like the standard
widgets to support this too, but... ;-) ); while doing so I took the
opportunity to refactor slightly more and ensure that all munging of
data took place in one method which could then be used for the purpose
described above - which worked just fine.


Cheers,


Nick

--
Nick Phillips / +64 3 479 4195 / nick.p...@otago.ac.nz
# these statements are my own, not those of the University of Otago

Tai Lee

unread,
Jul 13, 2010, 8:42:39 PM7/13/10
to Django developers
Hi Russ and Carl,
I wouldn't like to see widget libraries being packaged up with
templates including conditional templates based on doctype. This seems
messy, not friendly to override in your own templates, and effectively
combining what should be multiple separate templates into one.

Wouldn't a mechanism that allows the path from which templates are
loaded to be changed benefit both widget libarary authors and the
designers who are implementing those widget libraries? A widget
library author would include and advertise templates for each doctype
they support, e.g. "html4/mywidgetlibrary/forms/datetime.html" and
"xhtml1/mywidgetlibrary/forms/datetime.html", then the designer would
use:

{% templatedirs "html4/mywidgetlibrary" %}
{{ myform }}
{% endtemplatedirs %}

or:

<form ...>
{{ form.myregulartextfield %}
...
{% templatedirs "html4/mywidgetlibrary" %}
{{ myform.mydatetimefield }}
{% endtemplatedirs %}
</form>

to load the appropriate template from the widget library templates.
This gives a lot of flexibility and allows widget library authors to
support multiple doctypes or flavours (not just specific to doctypes),
and designers to pick and choose (or even override at the project
level) the widget templates according to their doctype or preferred
flavour, without Django needing any specific machinery for doctypes or
designers needing to edit any Python code.

Cheers.
Tai.

Carl Meyer

unread,
Jul 13, 2010, 9:44:48 PM7/13/10
to Django developers
Hi Tai,

On Jul 13, 8:42 pm, Tai Lee <real.hu...@mrmachine.net> wrote:
> I wouldn't like to see widget libraries being packaged up with
> templates including conditional templates based on doctype. This seems
> messy, not friendly to override in your own templates, and effectively
> combining what should be multiple separate templates into one.

I agree with you that separate template trees is much better than
conditionals. Also, if using conditionals, where does the value come
from that the conditional switches on? Part of my idea is that there
should be no need for some magic "use this doctype" value to be passed
around.

> Wouldn't a mechanism that allows the path from which templates are
> loaded to be changed benefit both widget libarary authors and the
> designers who are implementing those widget libraries? A widget
> library author would include and advertise templates for each doctype
> they support, e.g. "html4/mywidgetlibrary/forms/datetime.html" and
> "xhtml1/mywidgetlibrary/forms/datetime.html", then the designer would
> use:

Your {% templatedirs %} tag is an intriguing idea in its own right,
and certainly could intersect with template-based form rendering, but
I'd prefer to keep the two proposals orthogonal.

> This gives a lot of flexibility and allows widget library authors to
> support multiple doctypes or flavours (not just specific to doctypes),
> and designers to pick and choose (or even override at the project
> level) the widget templates according to their doctype or preferred
> flavour, without Django needing any specific machinery for doctypes or
> designers needing to edit any Python code.

I totally agree with this. I think very similar results can be
achieved in the form-rendering system just with an optional parameter
to a "form/render_form" tag that specifies a "base path" for finding
template widgets for this form render, as I mentioned above. I think
it's better to keep the form-rendering proposal specific to forms, and
evaluate {% templatedirs %} separately as a feature proposal on its
own merits.

Carl

flo...@gmail.com

unread,
Jul 13, 2010, 9:52:10 PM7/13/10
to Django developers
Carl's proposal is exactly the kind of thing I was suggesting in my
first response to this thread (only better thought out and stated more
eloquently than I could have). Count me as a big +1 to it!

Thanks,
Eric Florenzano

Tai Lee

unread,
Jul 14, 2010, 12:10:02 AM7/14/10
to Django developers
Hi Carl,

On Jul 14, 11:44 am, Carl Meyer <carl.j.me...@gmail.com> wrote:
>
> I totally agree with this. I think very similar results can be
> achieved in the form-rendering system just with an optional parameter
> to a "form/render_form" tag that specifies a "base path" for finding
> template widgets for this form render, as I mentioned above. I think
> it's better to keep the form-rendering proposal specific to forms, and
> evaluate {% templatedirs %} separately as a feature proposal on its
> own merits.

To me this sounds like basically the same thing, but limited to the {%
form %} and possibly {% field %} tags as a positional argument,
instead of being a generic tag that can be used for multiple forms and
form fields, as well as other (non-form) purposes.

I don't want to hijack this thread and take the focus away from
enhanced form rendering, but if such a feature is going to be tied to
forms specifically I'd like to see that it can be applied to multiple
{% form %} and {% field %} tags, without having to define a new "base
path" for each individual form or field that needs to be customised,
which is what led me to the use of a block tag instead of positional
arguments, and it just so happens that it could be useful outside of
form rendering so I tried to come up with a generic name :)

Cheers.
Tai.

Russell Keith-Magee

unread,
Jul 14, 2010, 9:14:53 AM7/14/10
to django-d...@googlegroups.com
On Wed, Jul 14, 2010 at 3:55 AM, Carl Meyer <carl.j...@gmail.com> wrote:
> Hi Russ,
>
> On Jul 13, 12:11 pm, Russell Keith-Magee <russ...@keith-magee.com>
> wrote:
>> My concern about doing this entirely in templates is that it makes the
>> process of sharing a widget library a little more difficult. If it's
>> just code, then it just needs to be in the PYTHONPATH. If there are
>> templates too, then the templates need to be somewhere that they can
>> be accessed.
>
> So I think this is very much a programmer's perspective, almost
> humorously so :-) "Just code" may seem easy to you, but from the
> perspective of at least one experienced Django-template designer I've
> talked to, being able to do it entirely in templates makes the idea of
> sharing a widget library conceivable, as opposed to almost impossibly
> intimidating if its "just code."

By "just code", I don't mean "code is easy", I meant "code, and
nothing else". Anything you can do in template you can also do in
code. The same is not true the other way around. I'll concede that the
intended audience may not see code as an easy option, but that doesn't
change the fact that code offers solutions that aren't otherwise
available.

> The app_directories template loader is installed by default. Getting
> templates "somewhere they can be accessed" means creating a directory
> on the PYTHONPATH with an empty models.py, an empty __init__.py, and a
> "templates/" subdirectory, then putting that in INSTALLED_APPS. That's
> it. How is that harder than following exactly the same steps for an
> app that includes Python code? From the perspective of a non-
> programmer, it's much easier.

Fair point.

> The key issue is familiarity and ease-of-use. For template authors,
> not programmers. Template authors know perfectly well how to add
> classes and attributes to HTML elements. Why should they have to use a
> special API to do the same thing for form input elements? Again: if
> the goal is for Django template authors to create an ecosystem of
> reusable widgets, the system needs to be geared around the tools they
> are comfortable with, not the ones programmers are comfortable with.
> That means templates. And not Python code.

Ok. I can certainly see that a template-based solution is the popular
solution, especially amongst designers. I still have some
reservations, but they're almost entirely about technical feasibility
and complexity of representation, not about desirability.

Part of the problem is that we've spent a lot of time talking about
the template tag/filter, but almost none on what the templates would
look like, and how the multiple layers of templates will knit
together.

Here's my first cut at a list of the use cases we need to support:

We need to be able to define templates for:

a) The layout for a single widget (e.g., a DateWidget)
b) The layout for a single row of a form
c) The layout for an entire form (top errors, fields, hidden fields)
in as_* style.

In each of these templates, (b) will need to render instances of (a),
and (c) will need to render multiple instances of (b) -- and not
necessarily the same instances, either.

Each of templates could be applied:

1) As a background system default - what you get if you don't override anything
2) As a specified default for the entire rendered page
3) As a specified default for a single form on a page
4) As a specified override for a field on a form (a/b) or a form as a whole (c)

There's also the issue of doctypes; If I have a document that is html4
document, I need to ensure I only get html4 snippets; ideally without
having to specify html4 every time I use a snippet.

If anybody has any other use cases or points worth considering, please speak up.

I'd like to spend a couple of days digesting what's been said in this
thread; I also want to look at some existing template-based projects
(like django-form-utils), and play around with a couple of ideas that
are lurking in the back of my brain. Hopefully, I'll be back in a week
or so with a revised proposal.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 14, 2010, 9:15:01 AM7/14/10
to django-d...@googlegroups.com
On Wed, Jul 14, 2010 at 5:58 AM, Nick Phillips
<nick.p...@otago.ac.nz> wrote:
> On Wed, 2010-07-14 at 00:11 +0800, Russell Keith-Magee wrote:
>
>> What exactly is your use case for something that designers want to
>> customize in the raw widget that widget.render() doesn't expose?
>
> That reminds me of one thing I'd very much like to see in any
> refactoring of form/widget handling.
>
> I'd like to be able to get at the "rendered" data (I say rendered, what
> I suppose I really mean is "transformed" either due to rendering needs
> or conversion from a graphical representation in the widget to something
> that could be resubmitted), without any of the surrounding chrome. There
> are various cases in which this can be useful, the most obvious of which
> is to tell whether the data in the form has been edited before
> submission. It's currently not possible to work this out from the
> "before" and "after" data, as you don't know how any particular field's
> data might have been munged by the widget (e.g. how is empty data
> rendered, any forcing to unicode, Python boolean to textual
> representation, date format transformation etc.).

Sounds like a reasonable request; and one that may almost be a
necessity if we're planning to make widgets rendered by template,
since we're going to need to expose most of the internals of a widget
to make them available for any possible rendering scheme.

Yours,
Russ Magee %-)

Russell Keith-Magee

unread,
Jul 14, 2010, 9:15:44 AM7/14/10
to django-d...@googlegroups.com

I'm not wildly enthusiastic about this. You've posted this snippet (or
something like it) a couple of times, and every time my immediate
reaction has been "You're in a twisty maze of endtemplatedir tags, all
alike" :-)

I can see what you're trying to do in terms of setting default
directories; but the syntax you are proposing:

* makes the representation of setting a default over a wide range quite elegant
* makes the representation for overriding a default on a single
field/form quite verbose.

To me, this seems like priorities the wrong way around. Overriding a
single default should be the simple case, not the verbose case.
Consider the template where you want to override the default widget on
every field on a form - your entire form will potentially be a nest of
{% templatedir %}{% endtemplatedir %} tags.

Yours,
Russ Magee %-)

SmileyChris

unread,
Jul 14, 2010, 4:15:58 PM7/14/10
to Django developers
On Jul 15, 1:14 am, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
> We need to be able to define templates for:
>
>  a) The layout for a single widget (e.g., a DateWidget)
>  b) The layout for a single row of a form
>  c) The layout for an entire form (top errors, fields, hidden fields)
> in as_* style.

We also need d) Media hooks for a single widget - whether this can be
done in only the template layer is a tricky problem...

I'm still struggling with how you'd be able to render all the media at
the top of your template for the form. I guess you'd have to define
any field widget overrides in that block, before rendering the form
media.

Carl Meyer

unread,
Jul 14, 2010, 7:22:06 PM7/14/10
to Django developers
On Jul 14, 4:15 pm, SmileyChris <smileych...@gmail.com> wrote:
> We also need d) Media hooks for a single widget - whether this can be
> done in only the template layer is a tricky problem...
>
> I'm still struggling with how you'd be able to render all the media at
> the top of your template for the form. I guess you'd have to define
> any field widget overrides in that block, before rendering the form
> media.

Yeah, this is tricky. For JS generally it shouldn't be rendered until
the bottom of the template, for page-speed reasons, but the reverse is
true for CSS. I don't think there's yet been a proposal in this thread
for how to enable rendering widget media before the form is rendered,
both proposals so far would have you specifying/overriding chrome (or
widget templates) only at form-render time.

Obviously it'd also be good to be able to split CSS from JS, like the
Media framework allows.

One half-baked idea: what about annotating the form object itself with
information about customizations you've made: paths to form-rendering
templates and widget templates? So you could use the same
customization syntax when rendering the form or rendering the form
media, whichever you do first, and it would remember those
customizations for later?

This introduces a little more cleverness than I'd prefer, but if the
only other option is repeating every customization twice... I'm also
not sure how this would interact with the idea that the "form media
render" tag should accept multiple form objects, so even if you use
the same widget/chrome in multiple forms on a page, its media still
only gets rendered once.

Carl

Tai Lee

unread,
Jul 15, 2010, 1:10:14 AM7/15/10
to Django developers


On Jul 14, 11:15 pm, Russell Keith-Magee <russ...@keith-magee.com>
wrote:
>
> I'm not wildly enthusiastic about this. You've posted this snippet (or
> something like it) a couple of times, and every time my immediate
> reaction has been "You're in a twisty maze of endtemplatedir tags, all
> alike" :-)
>
> I can see what you're trying to do in terms of setting default
> directories; but the syntax you are proposing:
>
>  * makes the representation of setting a default over a wide range quite elegant
>  * makes the representation for overriding a default on a single
> field/form quite verbose.
>
> To me, this seems like priorities the wrong way around. Overriding a
> single default should be the simple case, not the verbose case.
> Consider the template where you want to override the default widget on
> every field on a form - your entire form will potentially be a nest of
> {% templatedir %}{% endtemplatedir %} tags.

Thanks for the feedback, Russ. I'm not set on any particular syntax, I
just wanted to demonstrate the idea of augmenting the path that
template loaders use as a way of customising presentation for a group
of pages, a single page, a group of elements on a page, or a single
element on a page.

I admit that several instances of {% templatedirs "mypath" %}
{{ form.field %}{% endblock %} or nested calls to {% templatedirs %}{%
endtemplatedirs %} in a template could get messy. I'm not sure how
else it could be done though, without tying it to a positional
argument to {% form %} and {% field %} tags, in which case it would
need to be added to every template tag you might want to override
templates for in this way. It might also be more difficult to override
when you have pluggable apps that include templates which already use
the {% form %} and {% field %} templates.

One thing to keep in mind is that you could actually wrap your entire
form (or page, or master template) in a {% templatedirs %} block tag
and then simply add/remove templates from your file system, without
having to make any other changes to existing or pluggable templates in
order to explicitly replace the templates for specific elements.

The {% form %} and {% field %} template tags could even be updated to
look for multiple templates, based on their name as well as their
type. I'm thinking of something like the admin here, where you can
define templates at different locations in order to apply them with
more or less granularity.

For example:

In your projects master template you wrap the whole thing in {%
templatedirs "myforms" %}{% endblock %}. Then, in any of the templates
derived from your master template (but not templates that belong to
pluggable apps that extend their own master template), the template
engine will look there first when loading templates. Effectively
adding a folder to the TEMPLATE_DIRS setting, but only for templates
that extend your master template, without affecting other pluggable
apps that you might be using.

Then, if you tried to render a form that had been assigned a name or
id of "contact", it would try to load:

myforms/django/forms/contact.html
myforms/django/forms/default.html
django/forms/contact.html
django/forms/default.html

And if you tried to render a field on the contact form that has a name
of "email", it would try to load:

myforms/django/formfields/contact/email.html
myforms/django/formfields/contact/default.html
myforms/django/formfields/email.html
myforms/django/formfields/default.html
django/formfields/contact/email.html
django/formfields/contact/default.html
django/formfields/email.html
django/formfields/default.html

Perhaps the block tag could either accept multiple paths to search, to
avoid ugly nesting. Or alternatively it could require two arguments,
the first being the path to search, and the second a filter so that
search paths are not searched unecessarily. E.g. if you knew you only
wanted to load temlates from "myforms" instead of "django/forms", you
could do {% templatedirs "myforms" "django/forms" %}...{%
endtemplatedirs %}? Or, perhaps it doesn't need to be a block level
tag. Perhaps it just just applies to the whole template, like {% load
%}?

Just food for thought.

Cheers.
Tai.

petr.marhoun

unread,
Jul 18, 2010, 8:16:59 AM7/18/10
to Django developers
/* I posted this message few days ago - but it is not here so I try it
again. I am sorry for possible double post. */

> Here's my first cut at a list of the use cases we need to support:
>
> We need to be able to define templates for:
>
>  a) The layout for a single widget (e.g., a DateWidget)
>  b) The layout for a single row of a form
>  c) The layout for an entire form (top errors, fields, hidden fields)
> in as_* style.
>
> In each of these templates, (b) will need to render instances of (a),
> and (c) will need to render multiple instances of (b) -- and not
> necessarily the same instances, either.
>
> Each of templates could be applied:
>
>  1) As a background system default - what you get if you don't override anything
>  2) As a specified default for the entire rendered page
>  3) As a specified default for a single form on a page
>  4) As a specified override for a field on a form (a/b) or a form as a whole (c)
>
> There's also the issue of doctypes; If I have a document that is html4
> document, I need to ensure I only get html4 snippets; ideally without
> having to specify html4 every time I use a snippet.
>
> If anybody has any other use cases or points worth considering, please speak up.

Hi,

I use templates for rendering of my forms (with exception of widgets -
but I would prefer to use templates also for them) so I like template-
based proposal. And I have some more notes based on my experiences:
1. Rendering of some parts of forms depends on current context -
fields are different in simple forms and in forms which are parts of
formsets. But widgets are generally same in each context.
2. There are more variants how to customize only parts of forms - you
can customize all widgets of given type in the form or only one
field.
3. It is quite difficult to create templates which are easy and
customizable at once - my templates work but I have problems to
comprehend them.

I would like to propose following approach:
1. There are not only two form renderers as in Carl's proposal
(render_form and render_field) - each component (formset, form,
fieldset, field, widget and errors) has its own renderer. (I do not
say if it could be filter or tag or if it could be one filter/tag for
each component or one generic one which would only called something as
render method of the component - it would be next step to decide it.)
2. Each renderer would only call render_to_string with list of
template names (defined in the following paragraph) and with the
component in template context.

List of templates would be created from the following items (it is not
so difficult as the description):
1. Settings FORMS_TEMPLATE_PREFIX - "django/forms" by default.
2. Formset's name - could be defined in python code or by template
author, for example "permission_list".
3. Form's name - could be defined in python code or by template
author, for example "contact_form".
4. Fieldset's name - could be defined in definition of the fieldset,
for example "personal_info".
5. Field's name - defined in definition of the field, for example
"username".
6. List of form styles - could be defined by templates' authors (step
by step), for example "table_form", "html, table_formset" or "xhtml,
div_form".
7. List of templates for each component - defined in the component's
class, for example "date_field.html, field.html" for DateField or
"email_widget.html, widget.html" for EmailWidget.

For example, EmailField "sender" in form with name "contact_form" in
form style "table_form" would use following list (in reality only one
template or a few templates would be defined):
- django/forms/contact_form/sender/table_form/email_field.html"
- django/forms/contact_form/sender/table_form/field.html"
- django/forms/contact_form/sender/email_field.html"
- django/forms/contact_form/sender/field.html"
- django/forms/contact_form/table_form/email_field.html"
- django/forms/contact_form/table_form/field.html"
- django/forms/contact_form/email_field.html"
- django/forms/contact_form/field.html"
- django/forms/sender/table_form/email_field.html"
- django/forms/sender/table_form/field.html"
- django/forms/sender/email_field.html"
- django/forms/sender/field.html"
- django/forms/table_form/email_field.html"
- django/forms/table_form/field.html"
- django/forms/email_field.html"
- django/forms/field.html"

Some use cases:
1. I want to use templates from Django, but with html: Use "html" form
style in your template.
2. I want to use custom date widget's renderer for all my forms:
Define template "django/forms/date_widget.html".
3. I want to use custom field's renderer for all my fields with
"table_form" style: Define template "django/forms/table_form/
field.html".
4. I want to use custom email widget's renderer for "contact_form":
Define template "django/forms/contact_form/email_widget.html".
5. I want to use custom field errors' renderer for "username" field in
"contact_form": Define template "django/forms/contact_form/username/
errors.html".
5. I want to use different email widgets' renderer for admin: Define
"django/forms/admin/email_widget.html".
6. I want to implement reusable widget "MyGreatWidget" defined in
xhtml variant be default, but with support for html: Define template
"django/forms/my_great_widget.html" and "django/forms/html/
my_great_widget.html".

petr.marhoun

unread,
Jul 18, 2010, 8:19:47 AM7/18/10
to Django developers
I think that the best practice is not to use css and js in the html
page, only in external files (but maybe it is too theoretical opinion,
impossible in reality). I would not like if an reusable widgets
library embed css or js in my pages. And if there is an requirement
that widget's specific css or js code could be in different request,
it is much more complicated - generation of forms could be very
dynamic and generally it is not possible to say which widgets are used
without repeating the process.

For me, the right reusable widgets library would:
- implement html code without any css or js,
- implement external css and/or js code - static or dynamic,
- embed some hints for js and/or css in html code (html class for
widget identification or possibly more complex information such as
html5 data-* attributes for situations as slug fields in admin),
- give me a note that I have to reference some js and/or css files.

Chase

unread,
Jul 28, 2010, 9:23:47 AM7/28/10
to Django developers
Don't forget support for HTML5 INPUT placeholders! Ideally these would
pull from a new attribute on the ModelForm object.

Would {% form %} output the <FORM> tag, as well as the CSRF token, too?

Chris

unread,
Aug 7, 2010, 9:49:55 AM8/7/10
to Django developers
I'm a big +1 on the rendering with templates, as well as multiple
template tags, I see something like:

{% form new_user_form as_p %}
{% fieldset user_info %}
{% field username %}
{% field password %}
{% endfieldset %}
{% endform %}

or something similar, you could possibly even specify attributes in
the field tag like:
{% field username class='required' %}

Which is something that I have a big problem with the way it is
curretly done ie:
username =
forms.TextField(widget=forms.TextInput(attrs={class:'required'})

It's too verbose, and it mixes presentation with code to an extent
that I'm uncomfortable with.

Chris
Reply all
Reply to author
Forward
0 new messages