[GSoC] Revised form rendering
Gregor Müllegger
It's exactly the same as below but formated nicely.
GSoC 2011 Proposal - Revised form rendering
============================================
Hi my name is Gregor Müllegger. I'm a Computer Science student in Germany at
the University of Augsburg currently in the fourth year of my studies. I first
came to django shortly before 0.96 was released and a lots of awesomeness was
introduced with the magic removal branch.
I'm also doing some django freelancing work since 2008 to finance my studies
and attended DjangoCon EU in 2011. This year I would like to apply to
Google Summer of Code helping to improve Django with something that bugs me
since quite a while: It's builtin ability to render a form straight away into
HTML.
Motiviation
-----------
Why would I like to change the current behaviour? There are some reasons for
this:
1. It's hard to change the default rendering.
It is very easy to use e.g. django's builtin rendering of a ``ul`` based
output like ``{{ myform.as_ul }}``. But what if you want to use your own
*layout* like ``{{ myform.as_dl }}``? You can create a new method on your
existing forms, but that involves writing python code (out of the designers
hand) and might not be possible for thirdparty forms.
2. Maybe the ``as_ul`` rendering is fine for me. But maybe I want to skip that
unnecessary field that the thirdparty form is providing. The solution would
be to write down every single field in your template, reimplementing the
``ul`` layout::
<li>{{ form.field.label_tag }}: {{ form.field }}
{{ form.field.help_text }} {{ form.field.errors }}</li>
{# skipping field2 here #}
<li>{{ form.field3.label_tag }}: {{ form.field3 }}
{{ form.field3.help_text }} {{ form.field3.errors }}</li>
...
We all love DRY, so this is not acceptable.
3. The designers I worked with are often interested on adding custom css class
or an attribute to a form field. Most of the time this is really a pain to
do if you don't have control over the python form code. Imagine a reusable
app that ships with urls, views, forms. To add a single class you would
need to: (a) overwrite the predefined url because you want (b) to specify
an additional parameter for the view which is (c) your custom subclass of
the thirdparty form::
class ChangedAttributeForm(ThirdPartyForm):
field = forms.CharField(max_length=50,
widget=TextInput(attrs={'class': 'fancy-text-input'}))
btw: This also violates the DRY principle since you have to redefine the
used field type, it's attributes (like ``max_length=50``) and the widget.
I want to tell the designer how he can do this without my help in the
template.
Goals I want to accomplish
--------------------------
After showing some of the problems that I see, are here the higher goals I
want to achieve during the summer:
1. All of the rendering formats should be extracted from django's python source
into a well defined structure of customizable templates.
2. Make it possible to reorder the use of form fields in the template without
needing to write down the complete form.
3. Support for *chrome* that can be added dynamically in the template to a set
of fields, changing some aspects of the form output.
4. Ensuring that the DRY principle is applyable to your form templates (in
case of reordering/skipping some fields, like shown above)
5. Ensuring total backwards compatibility for ``{{ form.as_p }}`` etc. but
deprecating it.
6. Converting the admin to use all of the new goodness. First, to make django
eating its own dogfood. Second, to prove that the stuff I have developed is
really a step forward and to show up big problems that occur in reallife
scenarios, not taken into account in the blueprint.
7. Documenting all the newly introduced templatetags, the builtin chromes,
how to write custom chrome, how to create your own form layout etc...
Let's get a bit more detailed. How do I want to implement these goals?
**1. No HTML in python source**
I will push the formating of ``as_table``, ``as_ul`` and ``as_p`` into a
specified template structure. A formating (e.g. ``ul``) will be called a
layout and will live in the template directory ``forms/layouts/<layout>/...``.
This directory will contain a single file (e.g. for the table layout)::
forms/layouts/table/row.html
Creating a new layout is as simple as putting a new directory into your
template path and adding one file. Why putting the file into a directory? The
reason is because we can make the layout even more customizable. You are able
in the ``row.html`` to ``{% include %}`` a ``errors.html`` and a
``helptext.html`` that can be overriden by the designer without completely
rewriting the whole layout.
Ok, but how will this look in the template where I use the form? For this we
need a new template tag::
{% form myform using layout "p" %}
-- or --
{% form myform using layout "my_custom_layout" %}
**2. Reordering/skipping fields**
Thats pretty straigh forward with the newly introduced template tag::
{% form myform using fields "firstname" "lastname" "country" %}
You can list any fields you want, skipping some of them, reordering them etc.
Here is a more advanced example with using a layout and injecting fields of a
second form into another one::
{% form myform using layout "p" and fields "firstname" "lastname" %}
{% form fancyform.favourite_color using layout "p" %}
{% form myform using layout "p" and fields "country" %}
What have we added to the syntax?
a. We can use a single field instead of a form as first argument
b. We can use multiple *rendering modifiers* after the keyword ``using``.
*Rendering modifiers* are the bits after ``using``. They have a name (like
``layout`` or ``fields``) and take as many arguments as they want. You can
seperate multiple modifiers with ``and``.
The goal will be to make these modifiers defineable. Which means that you
can create your own modifiers to support some of your extravagant form
rendering needs. To support this we will need to have a rendering modifier
*registry* or something similiar, where you can ideally load new ones with the
existing ``{% load %}`` tag. The same will apply for custom *chrome*,
described below.
**3. Chrome**
The principle of a *chrome* is a shameless plug of Russell's earlier
suggestions on a revised form rendering [1]. Let me quickly summarize what a
chrome might be able to do for you.
A chrome might be useful to change the rendering of a widget. For example you
have a simple ``<input type="text" />`` input for datetime field. But you want
to enhance it with a calendar popup (like in django's admin). You would simply
apply the ``calendar`` chrome to your field::
{% form myform.birthday using calendar %}
So what's the difference here between a chrome and a rendering modifier? The
difference is only visible in the python level since chromes will be way
easier to implement.
They opperate only on fields and widgets not on a complete form. They won't
need to hassle with argument parsing since they have a common syntax that
will be::
{% form <form instance> using <chrome name> [<list of arguments>]
[for <filtering fields>] %}
So a chrome is called with the ``<list of arguments>`` that are already
evaluated, previously beeing template variables or static strings. The chrome
is also called once for every field, which simplifies implementation again.
The template tag syntax also provides ways of limiting the use of the chrome
to a set of fields (but the chrome implementor doesn't need to care about
them). It's easier to explain this with some examples::
Will add the attribute required to _all_ fields of the form (no
field filters specified):
{% form myform using attribute "required" %}
Calendar chrome for two specified fields:
{% form myform using calendar for myform.birthday myform.member_since %}
Adding the class "error" to all fields that have errors:
{% form myform using class "error" for errors %}
Calendar for all forms.DateTimeField fields in all of the used forms:
{% formblock using calendar for type "DateTimeField" %}
{% form myform %}
{% form fancyform %}
{% form another_form_with_lots_of_datetime_fields %}
{% endformblock %}
Autocomplete email addresses from your addressbook for all fields
that are using the EmailInput:
{% form myform using autocomplete "/addressbook/emails/" for
widget "EmailInput" %}
A sample chrome implementation would look like::
def attribute(bound_field, name, value=None):
if value is None:
value = ''
bound_field.field.widget.attrs[name] = value
Look again at the example from above::
{% form myform using attribute "placeholder" "Type in your name
..." for myform.firstname myform.lastname %}
This template tag will call the ``attribute`` chrome on both the fields
``firstname`` and ``lastname`` like::
attribute(myform['firstname'], 'placeholder', 'Type in your name ...')
attribute(myform['lastname'], 'placeholder', 'Type in your name ...')
There will also be the possibility of template only chromes which means that
you don't need any python code for some simple modifications.
Template based chromes will live in ``forms/chromes/<chrome name>.html`` and
will get the same arguments as the proposed signature of the python function:
``bound_field`` and ``args``. This makes things possible like creating
Javascript triggers after the widget::
in the form template:
{% form myform.client using autocomplete "/customers/" %}
in forms/chromes/autocomplete.html:
{{ bound_field }}
{# ^--- will render the used widget as usual #}
<script>
... javascript triggers ...
autocomplete({ url: {{ args.0 }} });
</script>
**4. Keeping your form templates DRY**
The example in **2.** is already much better than the current situation but it
still violates the DRY principle somehow. We repeat ourselfs by listing the
used layout three times. We can do better by grouping the modifiers with a
*formblock*::
{% formblock using layout "p" %}
{% form myform using fields "firstname" "lastname" %}
{% form fancyform.favourite_color %}
{% form myform using fields "country" %}
{% endformblock %}
This will remember the modifiers and chromes that are used in the
``formblock`` and will apply them to all ``{% form %}`` tags that are used
inside.
**5. Backwards compatibility**
Backwards compatibility is a serious thing but straight forward in the case of
this proposal. We can fall back to use the internals of the ``{% form %}`` tag
while rendering the form via ``{{ myform }}`` or ``{{ myform.as_ul }}``. A bit
trickier is the use of ``{{ myform.field.label_tag }}`` and
``{{ myform.field.errors }}``. The proposal above doesn't include these cases.
But this is also possible to solve. Goal 1. suggests to refactor all HTML out
of the python source. This must include lables and errors as well. For this
case we would create some new templates::
forms/layouts/default/label.html
forms/layouts/default/errors.html
They get the bound field that is used passed in and can render there output like
the ``label_tag`` method and the ``errors`` attribute. In the template we would
use::
{% form myform.birthday display errors %}
instead of {{ myform.birthday.errors %}
{% form myform.birthday display label %}
instead of {{ myform.birthday.label_tag %}
Storing these templates in the layouts directory has also some nice side
effects. We can for example use some alternative styling of the labels and
errors based on the current layout::
{% formblock using layout "plain" %}
{% form myform.birthday display label field %}
^--- uses the "plain" layout to render the label and the
field definition
{% form myform.birthday display errors using layout "ul" %}
^--- the specified "ul" layout overwrites the "p" layout from the
formblock and displays a list of errors instead of errors
seperated by <br> that might be used in the "p" layout.
{% form myform.birthday display helptext %}
{% endfor %}
**6. Admin integration**
The admin integration will be a real fun part for me. I already worked many
many hours with customizing the admin. Many clients want another widget to be
used here, adding some style changes to a field there etc.
All the stuffs is already possible ofcourse through custom widgets but we
still need to overwrite the forms in the admin code. A cool thing would be to
make the use of chromes easier here.
We will add some hooks to the ``ModelAdmin`` class that allow you to add
chromes to fields, maybe in a way like ``formfield_overrides`` works but
propably in a more flexible manner. This is already easy to use for most
designers since reading the ``admin.py`` file is usually very straight forward
and only requires some basic knowledges about assignments, lists and dicts.
However it would be even cooler to do this in the template.
First we will create a custom form layout for the admin. It's easy to
overwrite as described above so that you can change the look of the field
rows, errors, helptext pretty easily. Additionally there will be a block in
the ``change_form.html`` template that can be overridden by the user (the
example is showing the template ``admin/<appname>/change_form.html``)::
{% extends "admin/change_form.html" %}
{% block form %}
{% form adminform using autocomplete "/customers/" for
adminform.client %}
{% endblock %}
Besides the integration, there will be some need to convert the existing
gimmicks like calendar popup, "add new" icon next to ForeingKey dropdowns,
etc. into chrome implementations.
**7. Documentation**
Lots of work but nothing to specify here...
Media aka. JS/CSS
-----------------
One of the other mainpoints in the discussions I reviewed for this proposal was
the use of JS and CSS files that must be maintained somehow to display them
how we already do through the media framework (e.g. ``{{ form.media }}``).
The problem with this is that with the new template tag we can change some
of the widgets in the template and introducing new dependencies. Thats why I
would like to have an alternative for the ``using`` argument in the
``{% form %}`` tag.
If ``using`` is replaced with ``configure``, the ``{% form %}`` tag will _not_
output the HTML in the current place. However it will record and remember the
usage of widgets and fields to determine which media files are required. An
example template would look like::
{% block extrahead %}
{% form myform configure widget "CalendarInput" for myform.birthday %}
^--- The new widget for the field birthday will be
recorded, but the form will not be rendered.
{% formmedia css for myform %}
^--- Outputting all necessary css files.
{% endblock %}
{% block content %}
{% form myform %}
^--- The form will be rendered as usual but with the
"CalendarInput" widget that was specified in the other tag.
{% endblock %}
{% block extrajs %}
{% formmedia js for myform %}
^--- Outputting all necessary js files at the end of the document.
{% endblock %}
A shortform for the ``{% formmedia %}`` tag is available if one likes to load
the css and javascript next to each other: ``{% formmedia for myform %}``
I don't know if backwards compatibility is also necessary here. If you use the
new ``{% form %}`` tag you will also need to update your ``{{ form.media }}``
statements.
However we could still provide backwards compatibility by overwriting the
``media`` attribute of the form instance that we have modified with the
``{% form ... configure .. %}`` tag.
Estimates
---------
That's it so far with the proposal. In the following I will go a bit into the
timeline that I have in mind for the implementation.
1st week: Examing what unittests are available for the current form rendering
and making sure they are stable for testing backwards compatibility
during the project.
2nd week: Converting the current layouts into template based renderers, ensuring
backwards compatibility.
Goal: no HTML should be left now in the python source.
3rd week: I will attend DjangoCon EU
4th week: Starting to write tests and implementing the {% form %} tag
to be able to
emulate all the rendering that is currently possible.
5th week: Implementing the necessary rendering modifiers like "fields"
(limiting the
form to some selected fields) and the API for chrome.
6th week: Building the registry that is necessary to register your own rendering
modifiers and chrome.
7th week: Taking care of the media handling.
Goal: Project should be feature complete.
8th week: Converting the admin to use the new form rendering and
providing hooks for
applying chrome to some fields.
9th week: Integrating lessons learned from the admin especially in the sense of
making it easy to package chrome with a reusable app.
Goal: Code should be ready to be used in sample projects
10th & 11th week: Documentation and bugfixes
12th week: Finalizing
Unfortunatelly university is running in germany during the time that I will
work on the project. However based on my experience with last years of
university I think that I can provide about 30 hours per week for work on the
project, discussing questions and suggestions on the mailing list and so on.
So thanks a lot for reading so far, I really appreciate it and would love to
hear your feedback. Please keep in mind that the syntax is product of my own
taste and I haven't shown it to someone else yet. I'm confident that it will
be usable for most template authors and is also very flexible on the python
side for customizations. But there might still be a lot of room for critiques
and situations that I have not considered yet.
Gregor
(A formated version of this proposal is available at [2])
Addendum
--------
Most of my inspiration came from the *Revised form rendering* thread [1]
kicked off by Russell last year after DjangoCon EU. Thanks also goes to Jannis
and Bruno for giving me some advice and making clear what the current state of
template based widgets is and much more.
My first thought was to include template based widgets also in this proposal.
But Bruno has already made a great effort with developing
**django-floppyforms** that might also get merged into django. He is already
working on a patch [3].
* [1] http://groups.google.com/group/django-developers/browse_thread/thread/cbb3aee22a0f8918
* [2] https://gist.github.com/898375
* [3] https://github.com/brutasse/django/compare/15667-template-widgets
Carl Meyer
As you've probably seen in past threads, this is an area where I'm quite
motivated to see some improvement. I think you've got quite a strong
proposal here in general, and you've clearly done your homework, so my
comments below dive directly into the details:
On 04/01/2011 11:57 AM, Gregor M�llegger wrote:
> I suggest reading this proposal online: https://gist.github.com/898375
> It's exactly the same as below but formated nicely.
>
>
> GSoC 2011 Proposal - Revised form rendering
> ============================================
>
> Hi my name is Gregor M�llegger. I'm a Computer Science student in Germany at
I agree with your Motivations section - in particular this final line,
which sums up the core motivation as I see it.
This all looks quite good to me.
> **2. Reordering/skipping fields**
>
> Thats pretty straigh forward with the newly introduced template tag::
>
> {% form myform using fields "firstname" "lastname" "country" %}
>
> You can list any fields you want, skipping some of them, reordering them etc.
> Here is a more advanced example with using a layout and injecting fields of a
> second form into another one::
>
> {% form myform using layout "p" and fields "firstname" "lastname" %}
> {% form fancyform.favourite_color using layout "p" %}
> {% form myform using layout "p" and fields "country" %}
>
> What have we added to the syntax?
>
> a. We can use a single field instead of a form as first argument
> b. We can use multiple *rendering modifiers* after the keyword ``using``.
>
> *Rendering modifiers* are the bits after ``using``. They have a name (like
> ``layout`` or ``fields``) and take as many arguments as they want. You can
> seperate multiple modifiers with ``and``.
>
> The goal will be to make these modifiers defineable. Which means that you
> can create your own modifiers to support some of your extravagant form
> rendering needs. To support this we will need to have a rendering modifier
> *registry* or something similiar, where you can ideally load new ones with the
> existing ``{% load %}`` tag. The same will apply for custom *chrome*,
> described below.
This last bit worries me; it has the potential to add unneeded
complexity. Do you have any actual use cases for the "custom rendering
modifier" that can't be handled adequately by your proposal otherwise?
The concept of what a "rendering modifier" actually _is_ and what it can
do both seem poorly defined here. I'd rather see this level of
flexibility left out (and potentially added later) than done without a
clear driving concept and motivation.
Chromes is the part of this proposal that I am least excited about. It
seems to me that we already _have_ a perfectly workable abstraction for
"a front-end UI widget" - that's a Widget. Once we have widgets
rendering via template's (Bruno's floppyforms integration work) I don't
see the value in having "widget" and "chrome" as separate abstractions.
If you need a custom calendar control, define a widget for it. If you
don't need to modify any of the Python behavior of the default Date
widget, then we just need a way in the template API to say "render the
widget for this field with this alternative widget template." With that,
plus a syntax in your template tag for passing arbitrary attrs, like
placeholder, to the widget for a field (and the widget template author
can make use of those attrs however they want, they don't all have to
become HTML attributes on an <input> tag), I think you can accomplish
everything done here with no need to introduce a new abstraction.
If we introduce chromes as a new concept, I'm concerned that in practice
they will usually be heavily tied to a particular widget anyway, and not
very composable. I.e. why would you ever use a "calendar" chrome with
anything other than a Date widget? And when would you ever use an
"autocomplete" chrome with, say, a textarea? Or a select box? IMO the
calendar chrome should simply be a calendar widget, and the autocomplete
chrome an autocomplete widget.
I agree with putting the error and label HTML in overridable templates,
but why is this template syntax change needed? It doesn't seem like a
clear improvement in syntax (it's more verbose, and less explicit), and
the existing "errors" and "label_tag" methods can render a template.
> Storing these templates in the layouts directory has also some nice side
> effects. We can for example use some alternative styling of the labels and
> errors based on the current layout::
>
> {% formblock using layout "plain" %}
> {% form myform.birthday display label field %}
> ^--- uses the "plain" layout to render the label and the
> field definition
> {% form myform.birthday display errors using layout "ul" %}
> ^--- the specified "ul" layout overwrites the "p" layout from the
> formblock and displays a list of errors instead of errors
> seperated by <br> that might be used in the "p" layout.
> {% form myform.birthday display helptext %}
> {% endfor %}
Of course, if you wanted to do this sort of customization, you'd need to
use the new {% form %} based syntax.
With the exception of s/chrome/widget/g, this section looks pretty
strong to me. I think converting the admin is a valuable minimum barrier
to prove the usefulness and flexibility of the new API; and should
result in more flexible and overridable admin templates as well.
This is a tricky piece, and I haven't seen any better proposal for
solving it than what you're proposing here.
> I don't know if backwards compatibility is also necessary here. If you use the
> new ``{% form %}`` tag you will also need to update your ``{{ form.media }}``
> statements.
> However we could still provide backwards compatibility by overwriting the
> ``media`` attribute of the form instance that we have modified with the
> ``{% form ... configure .. %}`` tag.
It shouldn't even be necessary to "overwrite" anything - as long as you
update the widget for that field on the form instance, I think {{
form.media }} should pick it up from the widget's Media automatically.
In fact that might mean the {% formmedia %} tag isn't needed at all, and
people can continue to use {{ form.media }} (and {{ form.media.css }}
and {{ form.media.js }}) as we do now.
> Estimates
> ---------
>
> That's it so far with the proposal. In the following I will go a bit into the
> timeline that I have in mind for the implementation.
>
> 1st week: Examing what unittests are available for the current form rendering
> and making sure they are stable for testing backwards compatibility
> during the project.
>
> 2nd week: Converting the current layouts into template based renderers, ensuring
> backwards compatibility.
>
> Goal: no HTML should be left now in the python source.
Yay!
> 3rd week: I will attend DjangoCon EU
>
> 4th week: Starting to write tests and implementing the {% form %} tag
> to be able to
> emulate all the rendering that is currently possible.
>
> 5th week: Implementing the necessary rendering modifiers like "fields"
> (limiting the
> form to some selected fields) and the API for chrome.
>
> 6th week: Building the registry that is necessary to register your own rendering
> modifiers and chrome.
Not sure this registry is actually needed...
> 7th week: Taking care of the media handling.
>
> Goal: Project should be feature complete.
>
> 8th week: Converting the admin to use the new form rendering and
> providing hooks for
> applying chrome to some fields.
I think a week for this might be ambitious.
> 9th week: Integrating lessons learned from the admin especially in the sense of
> making it easy to package chrome with a reusable app.
>
> Goal: Code should be ready to be used in sample projects
>
> 10th & 11th week: Documentation and bugfixes
Allow me to channel Jacob briefly: Documentation is like tests, and
should be written while (or before) writing code.
I'd love to see a proposal where the _first_ week or two is writing
documentation - I usually write my documentation first, and I think
documentation-driven development is a pretty good way to make sure your
API is something someone would actually want to use before you build it.
"If the implementation is hard to explain, it's a bad idea."
> 12th week: Finalizing
>
> Unfortunatelly university is running in germany during the time that I will
> work on the project. However based on my experience with last years of
> university I think that I can provide about 30 hours per week for work on the
> project, discussing questions and suggestions on the mailing list and so on.
>
>
> So thanks a lot for reading so far, I really appreciate it and would love to
> hear your feedback. Please keep in mind that the syntax is product of my own
> taste and I haven't shown it to someone else yet. I'm confident that it will
> be usable for most template authors and is also very flexible on the python
> side for customizations. But there might still be a lot of room for critiques
> and situations that I have not considered yet.
>
> Gregor
>
> (A formated version of this proposal is available at [2])
>
>
> Addendum
> --------
>
> Most of my inspiration came from the *Revised form rendering* thread [1]
> kicked off by Russell last year after DjangoCon EU. Thanks also goes to Jannis
> and Bruno for giving me some advice and making clear what the current state of
> template based widgets is and much more.
>
> My first thought was to include template based widgets also in this proposal.
> But Bruno has already made a great effort with developing
> **django-floppyforms** that might also get merged into django. He is already
> working on a patch [3].
Yes, I think template-based widgets are a high priority, and my review
of this proposal is assuming that they are already being done anyway.
>
> * [1] http://groups.google.com/group/django-developers/browse_thread/thread/cbb3aee22a0f8918
> * [2] https://gist.github.com/898375
> * [3] https://github.com/brutasse/django/compare/15667-template-widgets
Good proposal - I'm excited about the potential here.
Carl
Mikhail Korobov
On 2 апр, 01:17, Carl Meyer <c...@oddbird.net> wrote:
>
> > 3. The designers I worked with are often interested on adding custom css class
> > or an attribute to a form field. Most of the time this is really a pain to
> > do if you don't have control over the python form code. Imagine a reusable
> > app that ships with urls, views, forms. To add a single class you would
> > need to: (a) overwrite the predefined url because you want (b) to specify
> > an additional parameter for the view which is (c) your custom subclass of
> > the thirdparty form::
>
> > class ChangedAttributeForm(ThirdPartyForm):
> > field = forms.CharField(max_length=50,
> > widget=TextInput(attrs={'class': 'fancy-text-input'}))
>
> > btw: This also violates the DRY principle since you have to redefine the
> > used field type, it's attributes (like ``max_length=50``) and the widget.
>
> > I want to tell the designer how he can do this without my help in the
> > template.
>
> I agree with your Motivations section - in particular this final line,
> which sums up the core motivation as I see it.
>
https://bitbucket.org/kmike/django-widget-tweaks
Each widget is instantiated by field in 'as_widget' method now, and
the field passes 'attrs' argument to widget with extra attributes.
The idea behind django-widget-tweaks is simple: it provides template
filters that take field instance and wrap its 'as_widget' bound method
so it receives extra attributes (and pass these extra attributes to
widget). This way widget attributes (including input types and css
classes) can be altered in templates without touching python, e.g.:
{{ myform.field|add_class:"fancy-text-input" }}
{{ search_form.query|attr:"type:search" }}
Implementation: https://bitbucket.org/kmike/django-widget-tweaks/src/0e9bac3c71bd/widget_tweaks/templatetags/widget_tweaks.py
Carl Meyer
That's quite a neat app - I have some similar template filters
implemented in some of my own projects. It doesn't cover the full range
of Gregor's proposal -- but I actually think most of Gregor's proposal
would be possible to implement outside Django. Perhaps we should
consider whether it would be preferable to start it off that way, or
focus the proposal more on where Django itself really needs to change.
Personally, I think HTML generation in Python code is a bug, and I'd
like to see it eradicated from Django, so I'm pretty open to seeing this
stuff make it to trunk.
Carl
Gregor Müllegger
I haven't reached you in IRC before.
2011/4/1 Carl Meyer <ca...@oddbird.net>:
> Hi Gregor,
>
> As you've probably seen in past threads, this is an area where I'm quite
> motivated to see some improvement. I think you've got quite a strong
> proposal here in general, and you've clearly done your homework, so my
> comments below dive directly into the details:
(I've shorten my proposal a bit in the quotes to make the whole email more
readable)
> On 04/01/2011 11:57 AM, Gregor Müllegger wrote:
>> I suggest reading this proposal online: https://gist.github.com/898375
>> It's exactly the same as below but formated nicely.
>>
>>
>> GSoC 2011 Proposal - Revised form rendering
>> ============================================
>>
>> ...
>>
>> Motiviation
>> -----------
>>
>> ...
>>
>> I want to tell the designer how he can do this without my help in the
>> template.
>
> I agree with your Motivations section - in particular this final line,
> which sums up the core motivation as I see it.
>
>> Goals I want to accomplish
>> --------------------------
>>
>> ...
>>
>> **1. No HTML in python source**
>>
>> ...
>>
>
> This all looks quite good to me.
>
>> **2. Reordering/skipping fields**
>>
>> ...
>>
>> The goal will be to make these modifiers defineable. Which means that you
>> can create your own modifiers to support some of your extravagant form
>> rendering needs. To support this we will need to have a rendering modifier
>> *registry* or something similiar, where you can ideally load new ones with the
>> existing ``{% load %}`` tag. The same will apply for custom *chrome*,
>> described below.
>
> This last bit worries me; it has the potential to add unneeded
> complexity. Do you have any actual use cases for the "custom rendering
> modifier" that can't be handled adequately by your proposal otherwise?
> The concept of what a "rendering modifier" actually _is_ and what it can
> do both seem poorly defined here. I'd rather see this level of
> flexibility left out (and potentially added later) than done without a
> clear driving concept and motivation.
>
I understand your worries and I clearly don't want to invent a YAGNI case. But
I think it fits nicely into the proposal (of it's current state). The reasons
are chrome. Chrome are just a specialized versions of "rendering modifiers"
(btw: I don't like the name but it's the best I came up with yet). And if you
can make it possible to register your own chrome, then it might be only a very
small overhead (if any) to do the same for rendering modifiers.
I want to make clear what a rendering modifier would be on the python level. I
thought about a FormRenderer class that handles the rendering of the form
based on the some predefined rules like what default layout is used. How to
render widgets and so on.
Rendering modifiers are then decorated around the FormRenderer. An example:
In the template::
{% form myform using layout "p" and widget "CalendarInput" for
myform.birthday %}
The FormRenderer iterates over the fields in myform and wants to render them
with layout "default" and with the widgets defined in the form itself. It
asks then the form modifiers "layout" and "widget" that were used in the form
tag how they want to modify the rendering of a specific field. The "layout"
modifier would always return "use layout p", the "widget" modifier always
returns "use the default widget" except when FormRenderer reaches the birthday
field, then "widget" would return "use CalendarInput widget".
(Sorry for the very easy language, maybe my english is to limited to express
it in a not so verbose way.)
I haven't thought in detail about additional usecases for the rendering
modifiers that might life in the app ecosystem. But while writing this
response, there came some of the possibilities into my mind. Maybe there is
the potential for a doctype modifier::
{% form myform using doctype "html4" %}
This will add a new template directory to the template path while rendering
the form. So that it renders HTML4 compatible widgets if available in the new
template directory. And if not it falls back to the default templates used by
the new template based widgets.
Something more exotic (and complex to implement but still possible) is a
"validation" rendering modifier::
{% form myform using validation %}
This will walk through the form and outputs at the end some javascript that
validates the form on the client side based on the used form fields.
This will need a port of python validators into javascript, but that isn't
that complex (I already did a proof of concept for that kind of behaviour in a
template tag).
And thats something that wouldn't get into django core but could life nicely
in **django-clientsidevalidation** :-)
This is why I think that the registry for rendering modifiers is useful. We
would need a registry (i.e. a dict that holds the "name to modifier" mapping)
anyway. So why shouldn't we make it open to other developers? The power of
rendering modifiers would be huge. You could modify nearly any behaviour of
the form rendering.
>> **3. Chrome**
>>
>> ...
>>
>
> Chromes is the part of this proposal that I am least excited about. It
> seems to me that we already _have_ a perfectly workable abstraction for
> "a front-end UI widget" - that's a Widget. Once we have widgets
> rendering via template's (Bruno's floppyforms integration work) I don't
> see the value in having "widget" and "chrome" as separate abstractions.
>
> If you need a custom calendar control, define a widget for it. If you
> don't need to modify any of the Python behavior of the default Date
> widget, then we just need a way in the template API to say "render the
> widget for this field with this alternative widget template." With that,
> plus a syntax in your template tag for passing arbitrary attrs, like
> placeholder, to the widget for a field (and the widget template author
> can make use of those attrs however they want, they don't all have to
> become HTML attributes on an <input> tag), I think you can accomplish
> everything done here with no need to introduce a new abstraction.
>
> If we introduce chromes as a new concept, I'm concerned that in practice
> they will usually be heavily tied to a particular widget anyway, and not
> very composable. I.e. why would you ever use a "calendar" chrome with
> anything other than a Date widget? And when would you ever use an
> "autocomplete" chrome with, say, a textarea? Or a select box? IMO the
> calendar chrome should simply be a calendar widget, and the autocomplete
> chrome an autocomplete widget.
Fair point. The motivation behind chrome were modifiers that could work on
more than a single type of widgets and template based chrome that add HTML
before, after the widget -- totally controlled by the designer.
However by changing the widget template with something like ``{% form myform
using widget template "widgets/calendar.html" for myform.birthday %}`` this
will work in a similiar way as with chrome templates.
The more generic case, a chrome that can modify any widget, would be a
attribute changer, the chrome that adds a class, one that changes the input
type (e.g. from "text" to "email") and so on.
In an IRC discussion I had with Bruno yesterday, he saw the chrome as
possibility to change the widget rendering in the chrome template. Example::
in forms/chromes/mychrome.html
<html that is inserted before the widget>
{% widgetrender bound_field
using add_to_context "email" as input_type
and template "mywidget.html" %}
<html that is inserted after the widget>
But this isn't included in my proposal because that would duplicate logic from
the widget rendering itself.
And there is still another valid case for chrome. They can be stacked!
An autocomplete widget could operate on text inputs, on email inputs, on url
inputs, on search inputs ... (in the case you use HTML5 inputs). Additionally
another chrome could suppress the help_text defined in the field and the third
chrome might wrap the field with a <div> (or some other fancy decoration)::
{% form sendemailform.receiver
using autocomplete "/addressbook/emails"
and nohelptext <--- since "enter a valid
email address" is not
needed if you use the
autocomplete
and wrap_into_div %}
But, yes all of them can be done in the {% form %} tag as well through
simple rendering modifiers, like already said chromes are just like rendering
modifiers, they only have a simpler syntax to define. An analogy: Template tags
are difficult to write by subclassing template.Node. They get pretty easy for
simple cases if you use template.Library.simpletag to register a function.
>> **4. Keeping your form templates DRY**
>>
>> ...
>>
>> **5. Backwards compatibility**
>>
>> Backwards compatibility is a serious thing but straight forward in the case of
>> this proposal. We can fall back to use the internals of the ``{% form %}`` tag
>> while rendering the form via ``{{ myform }}`` or ``{{ myform.as_ul }}``. A bit
>> trickier is the use of ``{{ myform.field.label_tag }}`` and
>> ``{{ myform.field.errors }}``. The proposal above doesn't include these cases.
>>
>> But this is also possible to solve. Goal 1. suggests to refactor all HTML out
>> of the python source. This must include lables and errors as well. For this
>> case we would create some new templates::
>>
>> forms/layouts/default/label.html
>> forms/layouts/default/errors.html
>>
>> They get the bound field that is used passed in and can render there output like
>> the ``label_tag`` method and the ``errors`` attribute. In the template we would
>> use::
>>
>> {% form myform.birthday display errors %}
>> instead of {{ myform.birthday.errors %}
>>
>> {% form myform.birthday display label %}
>> instead of {{ myform.birthday.label_tag %}
>
> I agree with putting the error and label HTML in overridable templates,
> but why is this template syntax change needed? It doesn't seem like a
> clear improvement in syntax (it's more verbose, and less explicit), and
> the existing "errors" and "label_tag" methods can render a template.
Yeah, it will still work. Maybe with or without deprecation. But why should we
stop there with refactoring the form to know nothing about its HTML
representation? IMO it's better to remove these methods as well to stay
consistent.
>> Storing these templates in the layouts directory has also some nice side
>> effects. We can for example use some alternative styling of the labels and
>> errors based on the current layout::
>>
>> {% formblock using layout "plain" %}
>> {% form myform.birthday display label field %}
>> ^--- uses the "plain" layout to render the label and the
>> field definition
>> {% form myform.birthday display errors using layout "ul" %}
>> ^--- the specified "ul" layout overwrites the "p" layout from the
>> formblock and displays a list of errors instead of errors
>> seperated by <br> that might be used in the "p" layout.
>> {% form myform.birthday display helptext %}
>> {% endfor %}
>
> Of course, if you wanted to do this sort of customization, you'd need to
> use the new {% form %} based syntax.
Exactly :-) thats the most important reason why I would like to encourage the
new syntax. You could also simply drop some new templates into the layout
directory like "highlight_row.html" and use::
{% form registrationform using fields "username" "password" %}
{% form registrationform.accept_tos display highlight_row %}
This way you could highlight the accept terms checkbox row to make sure the
user doesn't forget to tick the box.
Yes this could be done with a new layout as well (having a
forms/layouts/highlight/row.html file)::
{% form registrationform using fields "username" "password" %}
{% form registrationform.accept_tos using layout "highlight_row" %}
But I think the "display" version is more readable and propably easier to
understand (I have no evidence or user research on this).
>> **6. Admin integration**
>>
>> ...
>>
>
> With the exception of s/chrome/widget/g, this section looks pretty
> strong to me. I think converting the admin is a valuable minimum barrier
> to prove the usefulness and flexibility of the new API; and should
> result in more flexible and overridable admin templates as well.
Any way we go, with chrome or widgets only, converting the admin widgets to
template based ones is still necessary.
>> **7. Documentation**
My intention was that the {% form ... configure ... %} tag only saves the
options that were made but doesn't apply them to the form, like changing a
widget. In that case we would need the {% formmedia %} tag that understands
the modifications that will be applied to the form rendering.
But thats all in all an implementation detail. Your version is cleaner and
less bug prone and I will go for that one if no other problem will come to my
mind.
>> Estimates
>> ---------
>>
>> That's it so far with the proposal. In the following I will go a bit into the
>> timeline that I have in mind for the implementation.
>>
>> 1st week: Examing what unittests are available for the current form rendering
>> and making sure they are stable for testing backwards compatibility
>> during the project.
>>
>> 2nd week: Converting the current layouts into template based renderers, ensuring
>> backwards compatibility.
>>
>> Goal: no HTML should be left now in the python source.
>
> Yay!
Party!
>> 3rd week: I will attend DjangoCon EU
>>
>> 4th week: Starting to write tests and implementing the {% form %} tag
>> to be able to
>> emulate all the rendering that is currently possible.
>>
>> 5th week: Implementing the necessary rendering modifiers like "fields"
>> (limiting the
>> form to some selected fields) and the API for chrome.
>>
>> 6th week: Building the registry that is necessary to register your own rendering
>> modifiers and chrome.
>
> Not sure this registry is actually needed...
See my response to it above.
>> 7th week: Taking care of the media handling.
>>
>> Goal: Project should be feature complete.
>>
>> 8th week: Converting the admin to use the new form rendering and
>> providing hooks for
>> applying chrome to some fields.
>
> I think a week for this might be ambitious.
Propably yes. I will rethink this and update my estimates. Thanks for the
hint.
>> 9th week: Integrating lessons learned from the admin especially in the sense of
>> making it easy to package chrome with a reusable app.
>>
>> Goal: Code should be ready to be used in sample projects
>>
>> 10th & 11th week: Documentation and bugfixes
>
> Allow me to channel Jacob briefly: Documentation is like tests, and
> should be written while (or before) writing code.
>
> I'd love to see a proposal where the _first_ week or two is writing
> documentation - I usually write my documentation first, and I think
> documentation-driven development is a pretty good way to make sure your
> API is something someone would actually want to use before you build it.
> "If the implementation is hard to explain, it's a bad idea."
Yep, that's a really good idea. I will take this into account when rethinking
the time estimates.
>> 12th week: Finalizing
>>
>> ...
>>
>> My first thought was to include template based widgets also in this proposal.
>> But Bruno has already made a great effort with developing
>> **django-floppyforms** that might also get merged into django. He is already
>> working on a patch [3].
>
> Yes, I think template-based widgets are a high priority, and my review
> of this proposal is assuming that they are already being done anyway.
>
> Good proposal - I'm excited about the potential here.
>
> Carl
Thanks a lot for the in details thoughts. I hope I can make it bulletproof
enough until the deadline next week.
Russell Keith-Magee
I think this would be a good way to structure the project -- i.e.,
keep a clear conceptual separation between hooks and configuration
options that need to be added to Django in order to make an external
form rendering library possible, and the library that actually
exploits those hooks and configuration options.
Alex took a similar approach with the NoSQL work last year -- there is
a fairly clear separation between the "query refactor" changes, and
the "MongoDB module" that exploits those changes.
I don't think this needs to change the scope of Gregor's proposal very
much. If we change the focus to be just on the "Django side", we run
the risk of architecture astronauting. Developing a non-trivial
implementation that demonstrates the need for any given refactor helps
to clarify exactly why any given change to Django's APIs are needed.
Yours,
Russ Magee %-)
Gregor Müllegger
> I think this would be a good way to structure the project -- i.e.,
> keep a clear conceptual separation between hooks and configuration
> options that need to be added to Django in order to make an external
> form rendering library possible, and the library that actually
> exploits those hooks and configuration options.
>
> Alex took a similar approach with the NoSQL work last year -- there is
> a fairly clear separation between the "query refactor" changes, and
> the "MongoDB module" that exploits those changes.
>
> I don't think this needs to change the scope of Gregor's proposal very
> much. If we change the focus to be just on the "Django side", we run
> the risk of architecture astronauting. Developing a non-trivial
> implementation that demonstrates the need for any given refactor helps
> to clarify exactly why any given change to Django's APIs are needed.
>
> Yours,
> Russ Magee %-)
I agree on this. The best practice seems to keep changes outside of django if
possible until proven useful. Which can be done cleanly with the templatetag
at least. I didn't integrate plans on what goes inside django and what will be
outside of it since I didn't know if GSoC projects are accepted that work most
of the time on an app than on django itself. But if thats possible like with
Alex's work last year, thats fine for me.
Let me split up quickly what might go into django directly and what could live
outside if my proposal gets accepted:
1. Layouts
The form layouts should be included in django itself, refactoring
{{ form.as_table }}, labels, errorlists etc to make django code HTML free :-)
2. The template tag
This one clearly goes outside of django. Including a template tag is pretty
easy. Merging this in later would be also a no brainer.
3. Rendering modifiers
Thats also something that will go into an app. It's tight pretty closely to
the template tag. But this might require some internal changes to forms to
support all of the aspects described in the proposal (I don't see any
necessary changes yet, but it's like Russell describes, we don't know what
needs to be done until we have a full featured, non-trivial implementation).
4. Changes to widgets
I could see some additional APIs to widgets that get included into django
directly. Like a clean API to add attributes (instead of modifying attrs
directly), an API to add classes, an API to change the template that is used
to render the widget ...
All thirdparty apps that provide additional form features like
"django-widget-tweaks" would benefit.
5. Admin integration
This is a big point that is difficult to do. Creating a "admin" form layout is
still possible. But using the template tag and its mechanics inside of the
admin might not be useful until the template tag is merged into django.
However we might use the admin just as proof of concept, developing
integration of the app in another branch of django. But thats not ideal IMO.
6. Documentation
Writing documentation is ofcourse straight forward for the stuff that goes
into django directly. The thirdparty stuff will be documented like any other
reusable app. I don't see very big problems there. It might just generate some
extra work merging it in afterwards if the app code is accepted for django
core. But thats really the tiniest problem and acceptable if we can produce a
more mature product.
Gregor
Russell Keith-Magee
> I suggest reading this proposal online: https://gist.github.com/898375
> It's exactly the same as below but formated nicely.
>
>
> GSoC 2011 Proposal - Revised form rendering
> ============================================
>
> Hi my name is Gregor Müllegger. I'm a Computer Science student in Germany at
> the University of Augsburg currently in the fourth year of my studies. I first
> came to django shortly before 0.96 was released and a lots of awesomeness was
> introduced with the magic removal branch.
Hi Gregor,
Firstly, I'd like to echo Carl's sentiments -- this is a strong
proposal that shows you've researched the history of Django
discussions on this topic, and given it some thought of your own.
Carl has already raised several of the points that I noticed on my
read through. However, I do have some of my own queries.
Syntax
~~~~~~
You've proposed a syntax that goes something like:
{% form myform using layout "p" and fields "firstname" "lastname" %}
{% form fancyform.favourite_color using layout "p" %}
I have two questions about this syntax.
Firstly, while it looks fine for a small example, I can see how it
would rapidly deteriorate if you have lots of fields, or lots of
custom field requirements for each field. Django's template language
doesn't allow you to split tags over multiple lines, so what happens
when my tag runs over 80 characters (this simple example is already a
69 characters)?
Secondly, you appear to be using a single template tag for rendering
both forms *and* fields. Is there a motivation behind this?
Rendering modifiers
~~~~~~~~~~~~~~~~~~~~
I share Carl's concern about exactly how and why these are necessary;
even after your most recent post, I'm still not completely clear on
what they mean in practice.
On the one hand, I can see that there may be a use case for passing
arguments into the template rendering process. However, you seem to be
proposing a mechanism by which developers could define Python code
that alter widget rendering by performing such tasks as:
* selecting a different rendering directory
* changing the template that is used for a widget
* changing the layout scheme
* prepending/appending content onto a widget
I think I understand your motivation here, but my inclination is that
this will end up being a very complex system that won't end up being
useful in practice.
For example -- consider the case of validation. In an ideal world, it
would be nice to be able to just throw a switch and say "turn on
client side validation for this field", and then have that validation
work regardless of the rendering of the field itself. However, in
practice, it seems to me that the implementation of validation will be
tightly bound to the rendering of the field itself -- i.e, your widget
template would really need to be a "HTML4 text input with validation",
not a "text widget" with the HTML4 option and the validation option
enabled.
I can certainly see value in allowing template designers to pass in
keyword arguments when rendering fields on a template, allowing chrome
developers to define templates that implement various features (e.g.,
define a TextInput template that optionally has validation). However,
making this a complex registry and code solution seems like overkill
to me.
Form rows
~~~~~~~~~
If you look at the existing as_* form layout tools, there is a
distinction between gross form layout, and the layout of an individual
form row.
{{ form_header }}
{% for field in form %}
{{ pre_field }}}
{{ field }}
{{ post_field }}
{% endfor %}
{{ form_footer }}
Conceptually, the rendering for pre/post field could vary depending on
the field. How is this handled in your proposal? Is it part of the
form renderer? Or part of the widget rendering? This overlaps with
your discussion about error templates -- since errors (and their
rendering) is one of the major pre/post rendering activities for both
forms and fields.
CSS/Media
~~~~~~~~~
I'm not wild about the idea of having to include the same form content
twice in order to get CSS and JS included as where it is required.
I can see two options here.
Firstly, follow the lead of the {% cycle %} tag -- use the {% form %}
tag to define what the form will look like, but not actually render
where it is defined; assign the rendered properties to a context
variable, then reference that context variable when you want to
actually render something related to the form.
{% form myform <some content> as myform_rendered %}
{{ myform_rendered.media.css }}
{{ myform_rendered.media.js }}
{{ myform_rendered.html }}
This would be fairly easy to implement, but I'm not sure it would be
especially flexible.
Secondly, it may be possible to play a trick on the template compiler.
Jonas Obrist suggested this trick to me at Djangocon (US) last year;
however, I haven't had a chance to dig into it any more.
The trick goes something like this:
When you parse a template tag like an {% if %}, the template parser
scans forward until it reaches an {% endif %}. All the node content
that is discovered is then provided to the if node as the subtemplate
for the if.
However, there's no specific requirement that you need to parse until
you reach a specific tag -- you could just scan to the end of the
document, and treat the rest of the document as if it were the nested
content of an if that runs to the end of the page.
So -- you should be able to write a "CSS" template tag that reads to
the end of the template, and in the process, discovers all the {% form
%} nodes on the template. From that, you can determine all the CSS
requirements that your HTML document has.
It requires a bit of a twist in the document tree -- the entire
document effectively becomes a child of the CSS node in the template
-- but it should allow you to get access to form rendering
requirements before the form is actually rendered on the page.
Yours,
Russ Magee %-)
bur...@gmail.com
<rus...@keith-magee.com> wrote:
> Secondly, it may be possible to play a trick on the template compiler.
> Jonas Obrist suggested this trick to me at Djangocon (US) last year;
> however, I haven't had a chance to dig into it any more.
>
> The trick goes something like this:
>
> When you parse a template tag like an {% if %}, the template parser
> scans forward until it reaches an {% endif %}. All the node content
> that is discovered is then provided to the if node as the subtemplate
> for the if.
>
> However, there's no specific requirement that you need to parse until
> you reach a specific tag -- you could just scan to the end of the
> document, and treat the rest of the document as if it were the nested
> content of an if that runs to the end of the page.
>
> So -- you should be able to write a "CSS" template tag that reads to
> the end of the template, and in the process, discovers all the {% form
> %} nodes on the template. From that, you can determine all the CSS
> requirements that your HTML document has.
>
> It requires a bit of a twist in the document tree -- the entire
> document effectively becomes a child of the CSS node in the template
> -- but it should allow you to get access to form rendering
> requirements before the form is actually rendered on the page.
Hi Russell,
Maybe a bit different thing, but if we're talking about forms media
customization,
I'd love the following solution implemented for css and js:
Tag {% media for form1 %} or {% css for form1 %} which would remember
all previous css/js references, displaying only ones not rendered yet.
For example, lots of my widgets are referring jquery js and css, and
there's no need to load jquery 10 times.
And if I loaded jquery before forms, I would like to be able to say
{% js "{{STATIC_URL}}/media/jquery-1.4.2.min.js" %} or {% css
"/media/css/jquery-tweaks.css" %}
Gregor,
Regarding proposal itself,
the idea to make django form rendering based on templates is good, but
I think it should be simple, modular and not much verbose.
I'd like more to see more modular, but easier syntax:
1) If you have to render form in your custom way, you have to write
all form fields (you'll tweak their order and widgets anyway later, so
it's a plus).
2) If you have to render field in your custom way, you have to write
your own widget or widget template.
3) Any advanced form customization can be made using templates.
Something like this:
{% renderform form1 using layout "uni_form" %} {# errors output could
be changed with layout templates only #}
{# will use template layouts/app1/uni_form/charfield/x.html if it
was charfield. #}
{% render field1 using widget template "x" set class="field1" %}
{% render field3 %}
{% render field2 %}
{% endrenderform %}
--
Best regards, Yuri V. Baburov, Skype: yuri.baburov, MSN: bu...@live.com
Gregor Müllegger
thanks for your toughts.
2011/4/2 bur...@gmail.com <bur...@gmail.com>:
> Gregor,
>
> Regarding proposal itself,
>
> the idea to make django form rendering based on templates is good, but
> I think it should be simple, modular and not much verbose.
> I'd like more to see more modular, but easier syntax:
> 1) If you have to render form in your custom way, you have to write
> all form fields (you'll tweak their order and widgets anyway later, so
> it's a plus).
> 2) If you have to render field in your custom way, you have to write
> your own widget or widget template.
> 3) Any advanced form customization can be made using templates.
>
> Something like this:
> {% renderform form1 using layout "uni_form" %} {# errors output could
> be changed with layout templates only #}
> {# will use template layouts/app1/uni_form/charfield/x.html if it
> was charfield. #}
> {% render field1 using widget template "x" set class="field1" %}
> {% render field3 %}
> {% render field2 %}
> {% endrenderform %}
If I get your ideas right, then my proposed {% form %} and {% formblock %}
tag already supports what you suggested. I try to rewrite your example with
my proposed syntax:
{% formblock form1 using layout "uni_form" %}
{% form form1.field1 using widget template "x" and addclass "field1" %}
{% form form1 using fields "field3" "field2" %}
{% endformblock %}
The things that are different to your example are just some minor syntax
things. Additionally, the "rendering modifiers" (in this case "widget" and
"addclass") are not specified yet and I don't settled on a fixed set yet. I
would like to defer this until a later point because thats something that need
a broader discussion and IMO shouldn't take place in this thread, discussing
the more basic concerns regarding my proposal.
Or did I misunderstand what you tried to express? If thats the case, sorry for
that and please clarify that for me. I would appreciate your feedback.
(btw: a difference to your example is that I use the form tag to render forms
and single fields. I will include why I choose to do this in my response later
to Russell's post -- sorry I need to get offline but will respond again in the
next few hours)
Gregor Müllegger
2011/4/2 Russell Keith-Magee <rus...@keith-magee.com>:
(Having the possibility of multiline tags would be nice in many other cases as
well ... but that's another topic :-)
I don't see the use of more than two or three modifiers in a template tag in a
day to day use. However it's ofcourse possible to produce complex statements
that would need to span multiple lines. And it's already possible with the
syntax described in the proposal.
You can use a {% formblock %} to extract the modifiers out of the form tag:
{% formblock using layout "uni_form" %}
{% formblock using widget template "calendar" for myform.birthday %}
{% form myform using fields "firstname" "lastname" "birthday" %}
{% endformblock %}
{% endformblock %}
Some might not like the syntax since you would encourage another level of
indention and another two lines for the closing tags. But you can use the
"configure" statement as described in the media handling section of my
proposal:
{% form myform configure layout "uni_form" %}
{% form myform configure widget template "calendar" for myform.birthday %}
{% form myform using fields "firstname" "lastname" "birthday" %}
> Secondly, you appear to be using a single template tag for rendering
> both forms *and* fields. Is there a motivation behind this?
Yes there is :-) The motiviation is, that {% form myform.birthday %} is just a
shorthand for {% form myform using fields "birthday" %}. You can also use it
to iterate over a form (like in your example below) and outputting it field by
field:
{% for field in myform %}
{% form field %}
{% endfor %}
I also don't see a need for an extra {% formfield %} tag. I don't see any
cases in which a {% formfield %} would be shorter, more flexible, easier to
use. It might be more readable, but I think it would end up as a stripped down
copy/alias of the form tag.
> Rendering modifiers
> ~~~~~~~~~~~~~~~~~~~~
>
> I share Carl's concern about exactly how and why these are necessary;
> even after your most recent post, I'm still not completely clear on
> what they mean in practice.
>
> On the one hand, I can see that there may be a use case for passing
> arguments into the template rendering process. However, you seem to be
> proposing a mechanism by which developers could define Python code
> that alter widget rendering by performing such tasks as:
>
> * selecting a different rendering directory
> * changing the template that is used for a widget
> * changing the layout scheme
> * prepending/appending content onto a widget
>
> I think I understand your motivation here, but my inclination is that
> this will end up being a very complex system that won't end up being
> useful in practice.
The rendering modifiers came up in the proposal because I thought about the
possible implementations of the {% form ... using ... and ... %} syntax. I had
the idea that these modifiers after "using" could be seperated from the tag
itself. I had the analogy to queryset methods in mind. E.g. the filter() method
changes the internal state of the queryset. The same would be true for
rendering modifiers, analogy: the "fields" modifiers limit the rendered
fields. These modifiers could be implemented easily as methods on the template
tag.
But with querysets you can extend the QuerySet base class and add other
convenience methods. I wanted the same for the form rendering. Why shouldn't
it be possible not only to use the rendering modifiers shipped with the form
tag but also "subclassing" it (in analogy to queryset) and extending it.
Subclassing isn't an option here because you cannot inject a new manager
into the form tag like you do with models. So I came up with the registry.
Ofcourse we don't need the registry in the first place and could skip it until
we prove that it's requested by the users of the new form rendering machinery.
> For example -- consider the case of validation. In an ideal world, it
> would be nice to be able to just throw a switch and say "turn on
> client side validation for this field", and then have that validation
> work regardless of the rendering of the field itself. However, in
> practice, it seems to me that the implementation of validation will be
> tightly bound to the rendering of the field itself -- i.e, your widget
> template would really need to be a "HTML4 text input with validation",
> not a "text widget" with the HTML4 option and the validation option
> enabled.
Yes, the validation would need to know about all the things in use, like what
does the Form.clean method do? How is the current field validated? What
special cases are used in the widget? But that would be the problem of the
clientside validation framework. I have written a very basic proposal how I
would do this in the django-rays mailing list a year ago. But let's just
declare the validation example as a from me badly chosen one. It just came to
my mind why I would like to have custom rendering modifiers.
> I can certainly see value in allowing template designers to pass in
> keyword arguments when rendering fields on a template, allowing chrome
> developers to define templates that implement various features (e.g.,
> define a TextInput template that optionally has validation). However,
> making this a complex registry and code solution seems like overkill
> to me.
I don't think it would be complex. The implementation described above would
use modifiers implemented as methods on the template tag, that are called with
the arguments given in the template. We could use a simple dict lookup instead
of a "method"-lookup:
MODIFIERS = {
'fields': limit_fields,
'widget': change_widget,
'layout': change_layout,
...
}
class FormTag(template.Node):
...
def render(self, context):
form = self.form_variable.resolve(context)
...
for modifier in self.modifiers:
modifier = MODIFIERS[modifier.name]
form = modifier(form, modifier.arguments)
...
Having an API to add new keys to the MODIFIERS dict wouldn't be overkill and
much to maintain IMO. Loading new modifiers with the {% load %} tag would be
slightly more complex, I agree. But I haven't investigated on that yet what
code changes would be necessary in the template library.
But like I already wrote above: I see your concerns with adding yet another
registry that might not be used very often. I would agree on dropping that
feature if you think its better to skip this until users demand it afterwards.
> Form rows
> ~~~~~~~~~
>
> If you look at the existing as_* form layout tools, there is a
> distinction between gross form layout, and the layout of an individual
> form row.
>
> {{ form_header }}
> {% for field in form %}
> {{ pre_field }}}
> {{ field }}
> {{ post_field }}
> {% endfor %}
> {{ form_footer }}
>
> Conceptually, the rendering for pre/post field could vary depending on
> the field. How is this handled in your proposal? Is it part of the
> form renderer? Or part of the widget rendering? This overlaps with
> your discussion about error templates -- since errors (and their
> rendering) is one of the major pre/post rendering activities for both
> forms and fields.
I'm not sure if I get your example right. Is it representing the output of a
"as_*" method? I looked into the django/forms/forms.py code again and I'm
quite sure that there is no header or footer of a form rendered. I also don't
see many usecases of including a header and a footer into the layout (like a
wrapping <table> or <ul> element) since this is very different from form to
form. One might want the wrapping <table> the other wants to put two forms
into one <table> element etc.
The rest of the example in between would be rendered in the "row.html"
template of the form layout. E.g. the forms/layouts/table/row.html might look
like:
<tr><th>{{ field.label_tag }}</th>
<td>
{{ field.errors }}
{{ field }}
{{ field.help_text }}
</td></tr>
So you could define any pre_field and post_field stuff you want. I have also
not found a case in which the current rendering makes decisions based on the
field type. I think the admin does it (e.g. for checkboxes) but also in the
template level? Maybe you can give me a more specific hint on what you mean.
Basically changing the output could be ofcourse influenced by some if tags in
the "row.html" template.
> CSS/Media
> ~~~~~~~~~
>
> I'm not wild about the idea of having to include the same form content
> twice in order to get CSS and JS included as where it is required.
What do you mean by including the same form content twice? Do you mean the
duplicate use of the form tag? I thought I had solved this via the "configure"
statement. This will record all the used options so that you don't have to
duplicate the changes made to the form.
> I can see two options here.
>
> Firstly, follow the lead of the {% cycle %} tag -- use the {% form %}
> tag to define what the form will look like, but not actually render
> where it is defined; assign the rendered properties to a context
> variable, then reference that context variable when you want to
> actually render something related to the form.
>
> {% form myform <some content> as myform_rendered %}
>
> {{ myform_rendered.media.css }}
>
> {{ myform_rendered.media.js }}
>
> {{ myform_rendered.html }}
>
> This would be fairly easy to implement, but I'm not sure it would be
> especially flexible.
I think I have described your first proposed way with the "configure"
statement in the form tag.
{% form myform.birthday configure widget template "calendar" %}
will use the calendar widget template for the field birthday but doesn't
output the field, remembering the configuration for when {% form myform %}
is used.
Rendering the form into a variable is also an alternative. But I'm not totally
happy with it and there is a reason why I didn't took this way. My idea of the
form tag included that the form rendering would be pushed down to the latest
possible point (like with the laziness of querysets). The problem with
rendering it before you output the CSS links is that you must render the form in
the head of the page to get the final CSS requirements. But that would limit
the possibilites to change the rendering later. Take this example:
{% block extrahead %}
{% get_comment_form for object as commentform %}
{% form commentform as commentform_rendered %}
{{ commentform_rendered.media.css }}
{% endblock %}
{% block content %}
{% include "comments.html" %}
^--- I for myself have often a default template that contains
rendering the comment form, showing latest comments etc that I
can include on any page to show comments.
{% endblock %}
in comment_form.html
... I could only use the rendered {{ commentform_rendered }} here and
couldn't even change the order of the field:
{% form commentform using fields "name" "email" "comment" %}
^--- skipping the "url" field here ... not possible with a prerendered
form
> Secondly, it may be possible to play a trick on the template compiler.
> Jonas Obrist suggested this trick to me at Djangocon (US) last year;
> however, I haven't had a chance to dig into it any more.
>
> The trick goes something like this:
>
> When you parse a template tag like an {% if %}, the template parser
> scans forward until it reaches an {% endif %}. All the node content
> that is discovered is then provided to the if node as the subtemplate
> for the if.
>
> However, there's no specific requirement that you need to parse until
> you reach a specific tag -- you could just scan to the end of the
> document, and treat the rest of the document as if it were the nested
> content of an if that runs to the end of the page.
>
> So -- you should be able to write a "CSS" template tag that reads to
> the end of the template, and in the process, discovers all the {% form
> %} nodes on the template. From that, you can determine all the CSS
> requirements that your HTML document has.
>
> It requires a bit of a twist in the document tree -- the entire
> document effectively becomes a child of the CSS node in the template
> -- but it should allow you to get access to form rendering
> requirements before the form is actually rendered on the page.
That sounds HUGE and pretty cool. I had something like this in mind but
skipped the idea quickly because I thought that it's not be possible.
Especially with template inheritance. But if thats really possible
(theoretically) ... I will at least try it out since it would solve the media
handling problem in a very elegant way for the user.
Thanks for mentioning it. This makes me a bit enthusastic because my initial
idea was not a complete bullshit :-)
>
> Yours,
> Russ Magee %-)
Gregor
Chris Beaven
Daniel Greenfeld
Gregor Müllegger
> Anyway, as the current lead on Django Uni-Form I think its great that Gregor
> is picking up the torch for refactoring form rendering in Django 1.40. He's
> obviously done his homework and put a lot of thought into this critical part
> of Django.
> I'm not a core developer my vote doesn't count, but I'm giving it anyway. +1
> from me.
> Daniel Greenfeld
Hi Daniel, I really appreciate your support. It shows that I might be on the
right path for a successfull refactoring. Thanks.
@all:
I've updated my proposal. Mainly the media tag part and the timeline estimates
(nothing really exciting for the moment). A revision of the chrome section
will follow tomorrow. Have a look at the proposal online [1] or look below,
I've included the changed bits:
GSoC 2011 Proposal - Revised form rendering
============================================
...
Media aka. JS/CSS
-----------------
One of the other mainpoints in the discussions I reviewed for this proposal was
the use of JS and CSS files that must be maintained somehow to display them
how we already do through the media framework (e.g. ``{{ form.media }}``).
The problem with this is that with the new template tag we can change some
of the widgets in the template and introducing new dependencies. Thats why I
would like to have an alternative for the ``using`` argument in the
``{% form %}`` tag.
If ``using`` is replaced with ``configure``, the ``{% form %}`` tag will _not_
output the HTML in the current place. However it will record and remember the
usage of widgets and fields to determine which media files are required. An
example template would look like::
{% block extrahead %}
{% form myform configure widget "CalendarInput" for myform.birthday %}
^--- The new widget for the field birthday will be
recorded, but the form will not be rendered.
{{ myform.media.css }}
^--- Outputting all necessary css files.
{% endblock %}
{% block content %}
{% form myform %}
^--- The form will be rendered as usual but with the
"CalendarInput" widget that was specified in the other tag.
{% endblock %}
{% block footer %}
{{ myform.media.js }}
^--- Outputting all necessary js files at the end of the document.
{% endblock %}
I will also check the possibility and difficulty of a new ``{% formmedia %}``
tag that hooks into the template parsing system, reading until the end of the
template and analyzing the use of the ``{% form %}`` tag. This way it could
determine all changes that will be applied to the form before it gets
rendered, including all the necessary CSS dependencies that needs to be
imported in the header of the page.
It is not clarified yet, if the ``{% formmedia %}`` is possible at all with
the current template parsing implementation. There might be some risks that
need to be sorted out before starting with the implementation:
* By parsing from the ``{% formmedia %}`` tag until the end of the template
might result in that all content after this tag is represented as a child
node of it. What side effects are implied? Does it produce backwards
incompatibilities with thirdparty template tags?
* What happens if the ``{% form %}`` tag is changing the widget of the form
based on a context variable?
Estimates
---------
...
1st week: Starting to layout the documentation. The form tag syntax based on
discussions from the mailing list should already be finalized.
2nd week: Examing what unittests are available for the current form rendering
and making sure they are stable for testing backwards compatibility during the
project.
3rd week: I will attend DjangoCon EU, hopefully giving a talk about the
revised form rendering and collecting more feedback in an open space.
4th week: Converting the current layouts into template based renderers, ensuring
backwards compatibility.
Goal: no HTML should be left now in the python source.
5th week: Mainly working on documentation and integrating the very last changes
based on discussions from DjangoCon EU.
Goal: All public APIs should be stable.
6th week: Starting to write tests and implementing the ``{% form %}`` tag to
be able to emulate all the rendering that is currently possible.
7th week: Implementing the necessary rendering modifiers like "fields",
"layout" etc. and the API for chrome.
8th week: Implementing the ``{% formmedia %}`` tag.
Goal: Project should be feature complete.
9th - 11th week:
* Validating backwards compatibility for the ``{% formmedia %}`` parsing
implementation with thirdparty modules (see Media section).
* Converting the admin to use the new form rendering.
* Integrating lessons learned from the admin.
* Bugfixes and regression tests for problems that showed up in the work with
the admin.
Goal: Code should be ready to be used in sample projects.
12th week: Finalizing, bugfixes and tweaking the documentation.
...
Gregor
Russell Keith-Magee
> Hi,
>
> 2011/4/2 Russell Keith-Magee <rus...@keith-magee.com>:
>> On Fri, Apr 1, 2011 at 11:57 PM, Gregor Müllegger <gre...@muellegger.de> wrote:
>> Firstly, while it looks fine for a small example, I can see how it
>> would rapidly deteriorate if you have lots of fields, or lots of
>> custom field requirements for each field. Django's template language
>> doesn't allow you to split tags over multiple lines, so what happens
>> when my tag runs over 80 characters (this simple example is already a
>> 69 characters)?
>
> (Having the possibility of multiline tags would be nice in many other cases as
> well ... but that's another topic :-)
I'm not convinced it would be nice. Template tags are intended to be
short and pithy, not space in which you can define your own
domain-specific language. My feeling is that if a template tag
proposal needs to invoke the "if only tags could span multiple lines"
clause, the tag is already doing too much in it's definition.
The one exception I will grant to this is comments -- requiring {%
comment %} {% endcomment %}, or {# #} on every line is just daft,
IMHO. However, it's a daftness that hasn't progressed into enough of
an itch to drive me to scratch it myself. It's also unrelated to your
proposal :-)
> I don't see the use of more than two or three modifiers in a template tag in a
> day to day use.
Well.. I gave you an example -- a form with 10 fields, and I want to
render 8 of them.
> However it's of course possible to produce complex statements
> that would need to span multiple lines. And it's already possible with the
> syntax described in the proposal.
>
> You can use a {% formblock %} to extract the modifiers out of the form tag:
>
> {% formblock using layout "uni_form" %}
> {% formblock using widget template "calendar" for myform.birthday %}
> {% form myform using fields "firstname" "lastname" "birthday" %}
> {% endformblock %}
> {% endformblock %}
>
> Some might not like the syntax since you would encourage another level of
> indention and another two lines for the closing tags. But you can use the
> "configure" statement as described in the media handling section of my
> proposal:
>
> {% form myform configure layout "uni_form" %}
> {% form myform configure widget template "calendar" for myform.birthday %}
> {% form myform using fields "firstname" "lastname" "birthday" %}
I hadn't quite grasped how you were using formblock in your previous
example. I now see what you're trying to do, and it makes a lot more
sense. However, I'm not completely sold on certain aspects of this
syntax.
In particular, I have two objections:
* it feels like it's trying to cram too much into a single template tag.
* it requires a lot of duplication of "form myform configure" or
"formblock myform using"
Consider a slight variation that uses more tags:
{% form myform configure %}
{% layout "uni_form" %}
{% widget "calendar" for DateField %}
{% autocomplete "lastname" from "/users/lastnames" %}
{% endform %}
Then, when it comes time to render:
{% form myform using %}
{% field "firstname" %}
<p>And some HTML decoration</p>
{% field "lastname" %}
{% endform %}
or, using an iterative approach:
{% form myform using %}
{% for field in myform %}
{% field field %}
{% endfor %}
{% endform %}
or, using default layout:
{% form myform %}
This approach:
* cuts down on the duplication, either in the form of indented form
blocks, or in duplicated 'tag opening' content
* avoids the need for a DSL embedded in the tag definition
* avoids potential grammar ambiguities -- the less grammar there is
in a tag, the less likely there will be edge cases in the parsing.
>> Secondly, you appear to be using a single template tag for rendering
>> both forms *and* fields. Is there a motivation behind this?
>
> Yes there is :-) The motiviation is, that {% form myform.birthday %} is just a
> shorthand for {% form myform using fields "birthday" %}. You can also use it
> to iterate over a form (like in your example below) and outputting it field by
> field:
>
> {% for field in myform %}
> {% form field %}
> {% endfor %}
>
> I also don't see a need for an extra {% formfield %} tag. I don't see any
> cases in which a {% formfield %} would be shorter, more flexible, easier to
> use. It might be more readable, but I think it would end up as a stripped down
> copy/alias of the form tag.
See my previous comments about cramming everything into a single tag.
Also, there's an implementation issue here; if the first argument to
form can be a field or a form, then your implementation will need to
have logic to differentiate the two -- either isinstance checks, or
duck-type checks. Either way, it's a problem that is avoided by being
explicit: {% form %} operates on forms, {% field %} operates on
fields.
>> Rendering modifiers
>> ~~~~~~~~~~~~~~~~~~~~
>>
> But like I already wrote above: I see your concerns with adding yet another
> registry that might not be used very often. I would agree on dropping that
> feature if you think its better to skip this until users demand it afterwards.
I think this might be the better approach. Your proposal is already
fairly ambitious, especially if it's going to include a reworking of
admin's rendering. And, to that end, reworking admin's rendering
should prove a useful testbed for whether it is needed at all.
Also -- some food for thought; the reason that a registry is needed
here is because you're defining an extension api for a monolithic {%
form %} tag. If the rendering strategy used multiple tags, modifiers
could also be defined as template tags:
{% form myform %}
{% mymodifier foo %}
{% endform %}
Then you don't need a separate registry; a "modifier" just becomes a
template tag operating on the API exposed by the form rendering API
(presumably a set of known objects in the local context of the form
tag). If an end user wants a rendering modifier, they just define
their own template tag and use it in a {% form configure %} block.
I wasn't thinking so much about the <table> rendering, as some of the
other non-row rendering details.
For example, non-form errors currently have a rendering style that is
defined in code (although they are programmatically exposed in a way
that makes them compatible with templates). This is something that
isn't definable on a per-row basis, but it is tied to a form.
A second example -- handling of hidden fields. Again, these aren't
strictly row-based, but they do impact on the rendering of the form as
a whole.
> The rest of the example in between would be rendered in the "row.html"
> template of the form layout. E.g. the forms/layouts/table/row.html might look
> like:
>
> <tr><th>{{ field.label_tag }}</th>
> <td>
> {{ field.errors }}
> {{ field }}
> {{ field.help_text }}
> </td></tr>
>
> So you could define any pre_field and post_field stuff you want. I have also
> not found a case in which the current rendering makes decisions based on the
> field type. I think the admin does it (e.g. for checkboxes) but also in the
> template level? Maybe you can give me a more specific hint on what you mean.
Exceptions in radiobutton and checkbox rendering have historically
been the thorn in my side whenever I look at Django's form library, so
I'm just trying to think outside the box and think of the ways people
may want to twist form rendering. I don't really have a specific use
case in mind here.
>> CSS/Media
>> ~~~~~~~~~
>>
>> I'm not wild about the idea of having to include the same form content
>> twice in order to get CSS and JS included as where it is required.
>
> What do you mean by including the same form content twice? Do you mean the
> duplicate use of the form tag? I thought I had solved this via the "configure"
> statement. This will record all the used options so that you don't have to
> duplicate the changes made to the form.
Understood -- the flaw was in my understanding of your proposal.
>> Secondly, it may be possible to play a trick on the template compiler.
>> Jonas Obrist suggested this trick to me at Djangocon (US) last year;
>> however, I haven't had a chance to dig into it any more.
...
> That sounds HUGE and pretty cool. I had something like this in mind but
> skipped the idea quickly because I thought that it's not be possible.
> Especially with template inheritance. But if thats really possible
> (theoretically) ... I will at least try it out since it would solve the media
> handling problem in a very elegant way for the user.
Its certainly worth a couple of days investigation to see if it's
possible or plausible.
Two more thoughts that have occurred to me:
Firstly, from the point of view of proving the viability of your API,
it might be worth collecting a selection of pathological use cases
that can be used as a testbed. For example:
* A form with 30 widgets
* A form with only hidden widgets
* A form with multiple date widgets, all using a calendar widget
except for the last one
* A page that contains two forms
In effect, this is a set of conceptual test cases for your API -- in
fact, it might even form the basis for your programatic test suite.
Secondly: formsets. Are you intending to tackle formsets in this
proposal? It's ok if you don't -- omitting formsets would be a good
way to limit scope for the purposes of GSoC. However, I suspect the
formset use case will ultimately be important for trunk inclusion.
Even if you're not intending to tackle formsets, it's probably worth
mentioning them in your proposal.
In summary: I think you've got a strong proposal here. I've also taken
a look at your new proposal timeline, and it looks good to me. There
are certainly details to work out, and although they may appear
significant on first impression, I think they're mostly cosmetic. A
decision about using nested multiple tags or a single tag will
certainly affect your implementation, but doesn't fundamentally alter
the idea that you're driving at here: customizing the form rendering
using templates. As long as your submission acknowledges the fact that
there are some API issues still need to be resolved, I think your
proposal is in reasonably good shape.
Good luck with your final submission!
Yours
Russ Magee %-)
Gregor Müllegger
deadline. I'm not happy with it but I need to tackle some personal issues at
once.
2011/4/5 Russell Keith-Magee <rus...@keith-magee.com>:
>
> ...
>
> In particular, I have two objections:
> * it feels like it's trying to cram too much into a single template tag.
> * it requires a lot of duplication of "form myform configure" or
> "formblock myform using"
>
> Consider a slight variation that uses more tags:
>
> {% form myform configure %}
> {% layout "uni_form" %}
> {% widget "calendar" for DateField %}
> {% autocomplete "lastname" from "/users/lastnames" %}
> {% endform %}
Yes that looks clean and easy to follow. I just don't like the lone keyword
'configure' (or 'using') at the end of the tag. But I guess we need it to
determine if we search for a closing endform tag or if it's a standalone one
(or we use the 'trick' you described for the media handling).
> Then, when it comes time to render:
>
> {% form myform using %}
> {% field "firstname" %}
> <p>And some HTML decoration</p>
> {% field "lastname" %}
> {% endform %}
Looks good. A problem would be with using multiple forms:
{% form myform using %}
{% form another_form using %}
{% field "name" %}
{% endform %}
{% endform %}
Is "name" a field of "myform" or "another_form"? But thats easy to work around
by using {% field myform.name %} for example.
> or, using an iterative approach:
>
> {% form myform using %}
> {% for field in myform %}
> {% field field %}
> {% endfor %}
> {% endform %}
I wonder how it would look like to change the layout and outputting the whole
form like {{ form.as_ul }}. Your example is for more than basic scenarios good
but very verbose for just changing the layout. Maybe using the form tag would
inherit the configuration from the wrapping form tag::
{% form myform using %}
{% layout "ul" %}
{% form myform %}
{% endform %}
Skipping the wrapping {% form myform using %} is also possible, layout would
configure the whole template by setting a context variable. However this might
have sideeffects on included templates etc. Explicit is usually better than
implicit. But we will find a way to deal with it.
> or, using default layout:
>
> {% form myform %}
>
> This approach:
> * cuts down on the duplication, either in the form of indented form
> blocks, or in duplicated 'tag opening' content
> * avoids the need for a DSL embedded in the tag definition
> * avoids potential grammar ambiguities -- the less grammar there is
> in a tag, the less likely there will be edge cases in the parsing.
>
> ...
>
>>> Rendering modifiers
>>> ~~~~~~~~~~~~~~~~~~~~
>>>
>> But like I already wrote above: I see your concerns with adding yet another
>> registry that might not be used very often. I would agree on dropping that
>> feature if you think its better to skip this until users demand it afterwards.
>
> I think this might be the better approach. Your proposal is already
> fairly ambitious, especially if it's going to include a reworking of
> admin's rendering. And, to that end, reworking admin's rendering
> should prove a useful testbed for whether it is needed at all.
>
> Also -- some food for thought; the reason that a registry is needed
> here is because you're defining an extension api for a monolithic {%
> form %} tag. If the rendering strategy used multiple tags, modifiers
> could also be defined as template tags:
>
> {% form myform %}
> {% mymodifier foo %}
> {% endform %}
>
> Then you don't need a separate registry; a "modifier" just becomes a
> template tag operating on the API exposed by the form rendering API
> (presumably a set of known objects in the local context of the form
> tag). If an end user wants a rendering modifier, they just define
> their own template tag and use it in a {% form configure %} block.
Ha! That makes me happy to have a possibility for custom behaviour in the form
rendering and you happy because we don't need another abstraction. I like this
approach and will go for it.
Ok I see. It would be very easy to implement support for all this. We just
choose different templates depending on if we want to output the whole form.
For a single row (using {% field ... %}) we will use
forms/layouts/.../row.html when using {% form myform %} we will use
forms/layouts/.../form.html which makes use again of row.html internally.
>
> ...
>
> Two more thoughts that have occurred to me:
>
> Firstly, from the point of view of proving the viability of your API,
> it might be worth collecting a selection of pathological use cases
> that can be used as a testbed. For example:
>
> * A form with 30 widgets
> * A form with only hidden widgets
> * A form with multiple date widgets, all using a calendar widget
> except for the last one
> * A page that contains two forms
>
> In effect, this is a set of conceptual test cases for your API -- in
> fact, it might even form the basis for your programatic test suite.
Good idea. I did some of this "testing" already with my first proposed syntax.
Looking how it would behave in certain conditions. I will start collecting
more of these test cases and will play with them to see how the different
syntaxes feel in complex situations.
This is something good I can do until the coding starts (if I get accepted),
providing evidence that the finally choosen syntax is the best one.
> Secondly: formsets. Are you intending to tackle formsets in this
> proposal? It's ok if you don't -- omitting formsets would be a good
> way to limit scope for the purposes of GSoC. However, I suspect the
> formset use case will ultimately be important for trunk inclusion.
> Even if you're not intending to tackle formsets, it's probably worth
> mentioning them in your proposal.
I have not thought about formsets yet. But in my understanding there is no
real difference between a formset and using multiple forms manually. In a
formset you would just iterate over them instead of using them explicitly. The
only difference is the management form. We will find a way to nicely integrate
this. For example outputting it in the {% endform %} block or something
similiar. I could also imagine a "rendering modifier" that outputs a "add
another form" handler like we have it in the admin.
I will give formsets a thought but would like to keep them out of the actual
proposal just because of the time constraints. I will have them in mind so
that we can integrate them nicely later on.
> In summary: I think you've got a strong proposal here. I've also taken
> a look at your new proposal timeline, and it looks good to me. There
> are certainly details to work out, and although they may appear
> significant on first impression, I think they're mostly cosmetic. A
> decision about using nested multiple tags or a single tag will
> certainly affect your implementation, but doesn't fundamentally alter
> the idea that you're driving at here: customizing the form rendering
> using templates. As long as your submission acknowledges the fact that
> there are some API issues still need to be resolved, I think your
> proposal is in reasonably good shape.
>
> Good luck with your final submission!
>
> Yours
> Russ Magee %-)
Thanks a lot Russ. I'm very concerned that I had nearly no time in the last
days to refine my proposal but the deadline is pretty close now. I will invest
all the time I have to make the last changes, but all in all I hope this
doesn't affect my chances, since this thread is a good reference on where we
will finally go.
Gregor
Gregor Müllegger
the proposal is very much unchanged to the first draft, however I've included
passages that indicate that the syntax might change in further discussions
about this topic and that the concept of a chrome will be propably dropped in
favour of more flexible template based widgets -- I didn't wanted to
include a new detailed view on this topic without knowing how the final
implementation for template based widgets will look like.
For all that are interested, here are some links with more information about
my submission:
1. The official submission on the google-melange.com homepage:
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/gregmuellegger/1
2. The latest revision (and its change history) of the proposal on github
(maybe also nicer formated):
https://gist.github.com/898375
I will get back to the list kicking of some more discussions about the syntax
next week and doing some syntax "testing" like we previously discussed in this
thread.
Gregor