peppercorn
Chris McDonough
http://www.plope.com/peppercorn
It's a structured form data decoding system which replaces the need for
something like "dottedish".
--
Chris McDonough
Agendaless Consulting, Fredericksburg VA
The repoze.bfg Web Application Framework Book: http://bfg.repoze.org/book
Tim Parkin
--
You received this message because you are subscribed to the Google Groups "ish.io" group.
To post to this group, send an email to is...@googlegroups.com.
To unsubscribe from this group, send email to ishio+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ishio?hl=en-GB.
Chris McDonough
> Hi Chris,
>
> What you've done is undoubtedly ingenious and if we treat html as just
> an encoding stream, it makes a lot of sense. However, I'm not quite sure
> what the advantage is over using a dotted key approach? Personally I'd
> prefer to see the structure of my data easily in both html and request
> as well as in application code and dotted keys solve that problem in a
> way that is clear to the user of html, request data and application
> code. Dotted keys are also :-
>
> 1) useful in javascript/css as hooks for styling/progressive enhancement
> (and are used by formish's js in reordering/deleting but see the bottom
> pps for how we're thinking about using form ordering for this).
>
> 2) used when assigning widgets/defaults/etc to form elements
Yes, I agree that it's super-convenient to be able to say:
form.widgets = {'names.*':SomeWidget()}
I'm just not sure the dotted names belong in the form rendering. It sure makes
life easier when they're not in the form rendering, anyway. I don't think they
need to be present in the form rendering to retain this feature, though.
> I don't particularly like the hidden field markers that are used to
> denote start and end sections of sub-sections. It seems like unnecessary
> clutter in the html to me (personal opinion) and would mean many
> progressive enhancement JS would not be straightforward to write.
Clutter and style issues aside, can you give me an example of progressive
enhancement js that would fail or be difficult when hidden fields are present
in the form? I just don't know enough about it.
> re: a new form framework/library; I'd be interested in knowing what
> aspects of formish you don't like that make you want to build a new form
> framework? I'd prefer to work with you to make formish better than have
> another form library appear but I understand that if the approach you
> want to take isn't compatible there isn't much that can be done about
> that. Perhaps you could say what your goals would be for a new form
> library and we could chat about it?
Well, one thing is getting rid of the need for much of the code to support
"add_new_item" where sequence items are involved:
http://github.com/ish/formish/blob/master/formish/js/formish.js
Much of what this JS does when form elements are not numbered can be done like
this:
http://svn.repoze.org/deform/trunk/deform/static/scripts/deform.js
But as you say in the PPS, you're already on top of that.
Another thing is that in general, there's a pretty large amount of formish
code, and it's often difficult to follow due to the need to convert dotted
names and the existence of transparent proxies.
Some portion of it could go away if something like peppercorn were used instead
of relying so heavily on dotted names. I think some other portion of it could
go away if it didn't use ObjectWrapper proxies, or at very least it would be
easier to debug when things go wrong.
Other minor things:
- The multi file upload widget protocol is pretty baroque (even if the design
idea is brilliant).
- The way widgets are mapped to templates is pretty baroque (again, even if the
design idea is brilliant).
A very, very minor thing:
- PEP8 is a good thing. I beg you, please... ;-)
Formish also doesn't have enough test coverage, so it's difficult to contribute
to without inadvertently breaking something. This could be remedied, but every
time I start the job I bounce off of it. Writing tests is a tough job to get
spun up on when you didn't write the code to start with, so it's a hard job to
outsource; it's also not a lot of fun.
The docs are much better than most open source projects (particularly the
testing site), but they're still not adequate; it's hard to spin people up on
formish by pointing them at existing docs. I know this because I've had to do it.
In summary, most of the design of Formish is completely brilliant (it's still
the best form system out there), but the implementation, tests, and
documentation could use some loving.
To that end, although I'm not 100% certain this will be followed to its logical
conclusion, I've been working on http://svn.repoze.org/deform/trunk/ . It uses
peppercorn for form deserialization, and colander as a schema library. It
steals almost all of formish's design, but does not reuse any of its code.
This is just the easiest course of action for me personally.
If we were in the same room together for a week or so, I suspect we'd wind up
with a similar package, but instead of the result being called "deform" it
might be called "formish2". But we probably all lack the bandwidth to make
that possible.
>
> Tim
>
> p.s. I've built a replacement for convertish that handles mapping and
> serialisation, it's in a convertish branch on github at the moment and
> I'm in the process of rewriting the recursive procedures as a walk
> algorithm (with Matt's help).
Darn.. so have I.. http://docs.repoze.org/colander/
> p.p.s. After reading your blog post we've been thinking about using the
> ordering in the form to provide information about reordering instead of
> using javascript to re-order sequences. This would mean simple
> reordering, additions and deletions could be handled without javascript.
Yes, this seems to make sense.
Tim Parkin
On 3/23/10 8:58 AM, Tim Parkin wrote:
.. Dotted keys are also :-
1) useful in javascript/css as hooks for styling/progressive enhancement
(and are used by formish's js in reordering/deleting but see the bottom
pps for how we're thinking about using form ordering for this).
2) used when assigning widgets/defaults/etc to form elements
Yes, I agree that it's super-convenient to be able to say:
form.widgets = {'names.*':SomeWidget()}
I'm just not sure the dotted names belong in the form rendering. It sure makes life easier when they're not in the form rendering, anyway. I don't think they need to be present in the form rendering to retain this feature, though.
Clutter and style issues aside, can you give me an example of progressive enhancement js that would fail or be difficult when hidden fields are present in the form? I just don't know enough about it.I don't particularly like the hidden field markers that are used to
denote start and end sections of sub-sections. It seems like unnecessary
clutter in the html to me (personal opinion) and would mean many
progressive enhancement JS would not be straightforward to write.
Well, one thing is getting rid of the need for much of the code to support "add_new_item" where sequence items are involved:
re: a new form framework/library; I'd be interested in knowing what
aspects of formish you don't like that make you want to build a new form
framework? I'd prefer to work with you to make formish better than have
another form library appear but I understand that if the approach you
want to take isn't compatible there isn't much that can be done about
that. Perhaps you could say what your goals would be for a new form
library and we could chat about it?
http://github.com/ish/formish/blob/master/formish/js/formish.js
Much of what this JS does when form elements are not numbered can be done like this:
http://svn.repoze.org/deform/trunk/deform/static/scripts/deform.js
But as you say in the PPS, you're already on top of that.
Another thing is that in general, there's a pretty large amount of formish code, and it's often difficult to follow due to the need to convert dotted names and the existence of transparent proxies.
Some portion of it could go away if something like peppercorn were used instead of relying so heavily on dotted names. I think some other portion of it could go away if it didn't use ObjectWrapper proxies, or at very least it would be easier to debug when things go wrong.
Other minor things:
- The multi file upload widget protocol is pretty baroque (even if the design idea is brilliant).
- The way widgets are mapped to templates is pretty baroque (again, even if the design idea is brilliant).
A very, very minor thing:
- PEP8 is a good thing. I beg you, please... ;-)
Formish also doesn't have enough test coverage, so it's difficult to contribute to without inadvertently breaking something. This could be remedied, but every time I start the job I bounce off of it. Writing tests is a tough job to get spun up on when you didn't write the code to start with, so it's a hard job to outsource; it's also not a lot of fun.
The docs are much better than most open source projects (particularly the testing site), but they're still not adequate; it's hard to spin people up on formish by pointing them at existing docs. I know this because I've had to do it.
In summary, most of the design of Formish is completely brilliant (it's still the best form system out there), but the implementation, tests, and documentation could use some loving.
To that end, although I'm not 100% certain this will be followed to its logical conclusion, I've been working on http://svn.repoze.org/deform/trunk/ . It uses peppercorn for form deserialization, and colander as a schema library. It steals almost all of formish's design, but does not reuse any of its code. This is just the easiest course of action for me personally.
If we were in the same room together for a week or so, I suspect we'd wind up with a similar package, but instead of the result being called "deform" it might be called "formish2". But we probably all lack the bandwidth to make that possible.
Chris McDonough
> Clutter and style issues aside, can you give me an example of
> progressive enhancement js that would fail or be difficult when
> hidden fields are present in the form? I just don't know enough
> about it.
>
>
> Anything that wants to index specific fields based on names (i.e. it's
> easier and more explicit/readable to say
> $("input[name='course.start']").datepicker() than to try to index the
> third input with the name 'start' (The latter is not impossible though).
>
> I also use the dotted class names for styling tweaks quite often (e.g.
> "form .profile.address.n.postcode { background-color: #F00; }"
Given that this sort of tweakage needs to be done on a per-form-rendering basis
anyway, and can't be done generally (I think):
- In the case there are no sequences in the schema, you might just name each of
the schema elements uniquely, resulting in each form input element having a
unique name. It would then work exactly like it does now. This is probably
the 90% case.
- In the exceptional case that schema element names do overlap, I'm guessing it
won't be that much more painful to handle the exceptions using "dead reckoning"
via CSS/JQuery selector. It seems like this is what these syntaxes were made for.
- In the case there are sequences in the form rendering, it just can't be an
extremely common use case to be able to style one widget in the sequence
randomly . In particular, I can't imagine wanting to style "form
.profile.address.6.postcode" uniquely extremely often. OTOH, I can imagine
wanting to style the very first postcode or the very last postcode. But this
task is just as hard when the items are numbered as when they're not, right?
> My personal background is on the web/html side so that is where I like
> things to be 'purer' (it's the end product in my opinion and so should
> not be compromised just to make the code more elegant).
Ugh. There's nothing pure at all about HTML form generation, least of all the
HTML that gets generated. If I can save five hundred lines of code on the
Python side, and all I need to do is make more advanced use of common CSS
selectors on the browser side as a consequence, there's no question whatsoever
that I'm going to do that. We problably strongly disagree about that, though,
because I'm way more keen to reduce the amount of Python code on the server
side than I am in making it easy to tweak adhoc display properties of generated
form renderings. OTOH, I don't want to make it a total pain in the ass for
designers to tweak either, so it's a balancing act I guess.
> Another thing is that in general, there's a pretty large amount of
> formish code, and it's often difficult to follow due to the need to
> convert dotted names and the existence of transparent proxies.
>
> Which transparent proxies are the most 'difficult' and where do the
> dotted names make things difficult to understand?
A RenderableProperty that generates a configurable RenderableObjectWrapper when
an attribute is accessed on an unrelated object. A CollectionFieldsWrapper
that gets returned when you access "fields" as a property. A BoundWidget that
gets generated when you access a widget via a form (also via a property). All
of this stuff has a non-trivial cognitive cost. It's also computationally
expensive (to the extent it matters).
Random bitching: many of the docstrings in the formish code don't actually
match the code they document. Since it's often difficult to know how it's
meant to work, it's hard to fix them.
Anyway, formgen is a hard problem, and any code that solves it is going to be
rough reading. And formish does a damn fine job at it, much better than other
systems. But its usage of properties and wrappers to give things a nicer APIs
is a net lose AFAICT. I'd rather have a more straightforward implementation
where maybe I need to call a method instead of accessing a property than need
to trace through all this code when something goes wrong. Likewise for
wrappers.. I'd rather just figure out a pattern where I didn't need the
wrappers than use transparent wrappers. It's conceivable that the
formish.forms code might be fully understood by someone that didn't write it.
It's just not really a given.
> I'm probably an
> exception but it is'nt obvious to me where the conversions to/from
> dotted dicts could be problematical as most bits of data get converted
> as they arrive and then back when returned I think.. if it's not clear,
> I'm happy to simplify code but I'll need pointing at it if possible :-)
I think the code in deform is about as good a stab at an explanation of how I'd
make it easier to read and understand as any individual fixes I could enumerate
here. But here's some suggestions:
- Formish probably shouldn't depend on WebOb. Instead it would be more
flexible and useful if the app developer code that used formish was responsible
for feeding it raw form fields. Then even frameworks that didn't use WebOb
could use it without writing more wrappers.
- Fewer wrappers. Deform does have wrappers (widgets wrap schema objects), but
they are not transparent; they're opaque. And there's only one kind of wrapper
so far.
- Fewer properties. Reading code that accesses properties often fools you into
believing things that are not true; "this attribute access is cheap", "if I
have a problem here, debugging it will be easy!" Properties and decorators
often just hide complexity rather than doing away with it.
- A form should really just be a big widget (IMO).
- Collection.bind is mindbending. The concept of a collection as separate
entity from a form or a widget or a schema node needs justification. And how
is a Collection different than a Group? Ugh.
- Why am I always doing things like "${field.value[0]}" in templates? I just
want the value. There's some sacrifice I'm paying for here, I just don't know
what it is.
- Even though formish tries hard to give you back and nest fundamental data
structures like mappings and seqences, there's *lots* of code in formish.forms
that lets dottedish and dotted names bleed through. Anything that calls
"starify", anything that does something like "if '*' in k:" and similar, etc.
I'd personally rather that some lower-level component handled mapping the form
data to a nested structure so this stuff didn't need to be sprinkled around at
this level, just to make that code more understandable. This is really just a
factoring thing, not a design issue at all. It just makes things hard to read
and follow.
> Some portion of it could go away if something like peppercorn were
> used instead of relying so heavily on dotted names. I think some
> other portion of it could go away if it didn't use ObjectWrapper
> proxies, or at very least it would be easier to debug when things go
> wrong.
>
> Can you give me an example of sorts of things you were customising and
> having difficulty debugging.. It's really good that you're raising
> issues with readability etc as when it's your own code you can't see
> issues like this so it's really good to have some direction!
See above.
>
> Other minor things:
>
> - The multi file upload widget protocol is pretty baroque (even if
> the design idea is brilliant).
>
> Persistent file uploads are hairy :-) You should have seen the code
> before we spent a lot of time cleaning it up!
We actually stopped using your implementation (after learning lots from it),
and replaced it with one of our own.
>
> - The way widgets are mapped to templates is pretty baroque (again,
> even if the design idea is brilliant).
>
> You'll have to expand on this one.. do you mean the 'fallback' stuff?
No...
class Sequence(Collection):
...
template = 'sequence'
What does that mean? Which template is that pointing at? And why is the text
input widget's template named 'field.Input'? How does that map to an actual
template in the templates directory?
These are all rhetorical questions, because I figured all the issues out by
reading the code. But there's an indirection there that just isnt documented.
And it's not obvious.
> The docs are much better than most open source projects
> (particularly the testing site), but they're still not adequate;
> it's hard to spin people up on formish by pointing them at existing
> docs. I know this because I've had to do it.
>
> If you could point me at what parts of formish most need additional
> documentation, I'll take a go. Same goes for tests.
Tests are easy. Just get 100% coverage as reported by coverage.py and you're
probably in the right ballpark. 100% coverage is required, but not necessarily
sufficient to make sure that when people contribute they know they haven't
broken anything major. But it's an easy objective metric.
Docs are harder. But I'd think of yourself as someone who hasn't used it
before and needs to use it in anger. Not just the easy stuff. The hard,
boring stuff. Like mapping widgets to templates, customizing existing
templates, adding a template directory, creating a new schema type, creating a
new widget type, creating a new converter, mapping a schema type to a different
converter, mapping a new schema type to an existing converter. And so on. I
mean.. basically write docs that you'd like to read if you needed to use the
thing in anger and you weren't necessarily passionate about the problem.
> In summary, most of the design of Formish is completely brilliant
> (it's still the best form system out there), but the implementation,
> tests, and documentation could use some loving.
>
> I'd be happy to work on the latter but changing the implementation to
> something like peppercorn is almost definitely not going to happen as it
> would mean rewriting most of formish.
I figured as much, and that's fine.
> I think if that is a requirement
> then a new library would essential as the two approaches aren't
> compatible (as far as I can tell - then again, like I said earlier, I'm
> not really a programmer at heart so I may not be best qualified to judge).
They're not compatible without a lot of bandwidth to understand the issues and
time to arrive at "win win" compromises. I don't know that any of us have time
for that, especially given that you're already satisfied with formish.
> To that end, although I'm not 100% certain this will be followed to
> its logical conclusion, I've been working on
> http://svn.repoze.org/deform/trunk/ . It uses peppercorn for form
> deserialization, and colander as a schema library. It steals almost
> all of formish's design, but does not reuse any of its code. This is
> just the easiest course of action for me personally.
>
> If the best we can manage is a little cross fertilisation then that is
> better than nothing...
I think so. Reuse is overrated sometimes. ;-)
Chris McDonough
the meantime, I've gone into full-on reimplementation mode in "deform":
http://docs.repoze.org/deform/index.html
Obviously the docs need lots of work, and that's what I'll be concentrating on
for the next few days.
If you're not too busy and you think it's worthwhile, I'd like to hear your
take on the design and the code: http://svn.repoze.org/deform/trunk/
Now that I'm well down the rabbit hole, I feel strongly that something along
these lines is about the right way to go. As a "hail mary", I'm hoping to be
able to convince you guys of the same, because I'd really rather not go it alone.
Salient points:
- Deform currently does most of what formish does (basically everything except
for the DateParts widget and readonly widgets, which I haven't gotten around to
implementing yet), but it's only about 1/4 the amount of Python code; it also
contains less template code.
- The API is very close to Formish's. I could make it almost exact, I think,
save for the schema construction differences, which probably can't be helped.
- No cooperation is needed between widget renderings and sequences; any widget
can be embedded in any sequence.
- It's reasonably fast. A relatively complex form rendering runs at about
200-300 requests per second on my MacBook Pro.
- Mako can be plugged in by implementing another "renderer".
- It has 100% test coverage. Its dependencies also have near-100% test
coverage (peppercorn has 100%, colander has 99% .. missing one dumb line, and
Chameleon has ~90%).
In any case, I know it's a long shot, but nothing asked, nothing gained...
Hope you're good!
- C
Tim Parkin
Tim Parkin
Chris McDonough
Yes, I belive it would be possible to make the deform Python API very close to
the Formish API. In particular, I think we could probably make the "formish"
package a wrapper around deform that translates most API calls to their deform
equivalents. Likewise for translating many Schemaish schemas to Colander
schemas once used as a "formish" schema (when passed to formish.form.Form(...)).
Being able to reuse Formish widget template code is unlikely or at least the
amount of work required for reuse would exceed the work required to rewrite all
widgets in existence as far as I can tell.
On 3/29/10 8:42 AM, Tim Parkin wrote:
> Just had an idea...
>
> re: the form naming.. If we consult with some professional html building
> and web accessibility people and survey them to see what they think,
> would you be happy to change things depending on the repsonse? (i.e. if
> they say they would like dotted hooks of some type then go with that or
> if they say they are happy with hidden fields and single level names,
> then go with that?)
I'll want to use something that uses peppercorn personally, whatever the web
accessibility people might say, because it's clearly, absolutely, inarguably,
by god the right thing. <wink>
But for the nonbelievers, we *could* make this pluggable. There's exactly one
place in the code that parses the form post data and turns it into a "pstruct"
(which stands for "peppercorn structure", but it's just a name really).
For instance, currently, if the following set of controls present within a form
submission presented to deform:
__start__ people:sequence
__start__ person:mapping
first Bob
last Dobbins
__end__
__start__ person:mapping
first Fred
last Flintstone
__end__
__end__
When something like "form.validate(request.POST.items())" is called, a data
structure like this is created by the system:
{'people':[
{'first':'Bob', 'last':'Dobbins'},
{'first':'Fred', 'last':'Flintstone'}
]
}
E.g. formencode.variabledecode is happy to make the same transformation based
on dotted names instead. For example, if you pass this to
formencode.variabldecode.variable_decode()...
people-1.first Bob
people-1.last Dobbins
people-2.first Fred
people-3.last Flintstone
It will result in the same dictionary:
{'people':[
{'first':'Bob', 'last':'Dobbins'},
{'first':'Fred', 'last':'Flintstone'}
]
}
Making this particular thing pluggable would mean:
- an entirely different set of widgets would need to be used when
variabledecode-style is used vs. when peppercorn-style is used, because the
widgets will need to generate different HTML. It's just not worth it (AFAICT)
to try to generalize these differences away into a common widget set. A set of
widget templates that tried to do both would be too complicated to be worth using.
- When serializing a form, the system would need to provide an API to generate
a dotted name from the current field.
- The variabledecode-style sequence widget will requires a bunch of hairy
javascript for dealing with sequence manipulation.
I don't think any of these things is impossible.
On the other hand, if we can avoid it being a necessity, I'd like to. ;-) The
peppercorn deserialization stuff is good exactly *because* it avoids the need
to write code that deals with sequence numbering. This is, AFAICT, a major
reason as to why the codebase is much smaller. And in my testing so far, it's
easy enough to style via CSS. You didn't respond to my counterarguments about
this in the second-prior email I sent, and I'd like to understand exactly why
you think styling this stuff will be hard. I've had no problems at all so far.
Often things can be made better with minor spelling and appearance tweaks.
Would it help to change peppercorn to expect the following kinds of controls
back from an HTTP post?
people [
person {
first Bob
last Dobbins
}
person {
first Fred
last Flintstone
}
]
In other words, the beginning of a sequence would be spelled
<input type="hidden" name="sequencename ["/>
And the beginning of a mapping would be spelled:
<input type="hidden" name="mappingname {"/>
The end of a sequence would be spelled:
<input type="hidden" name="]"/>
The end of a mapping would be spelled:
<input type="hidden" name="}"/>
This looks a lot nicer, and might be easier to swallow as elements in the input
stream.
> I really would like to cooperate but only if the result is something
> that myself and other web development people would be happy using..
I'm sure that most people who deal heavily with CSS styling of forms will of
course hate it, because it means they need to learn something, and learning
things is the enemy of most people. So I don't think we need to get a focus
group together: we already know people naturally hate everything except what
they're using right now. So I'm not willing to ditch peppercorn-style form
renderings on that basis. I am willing to consider making it pluggable tho.
OTOH, as far as I can tell, designers *already* hate generated forms. As long
as sufficient documentation exists for styling, it seems pretty dubious that
people will *actually* have much more of a problem styling stuff with
peppercorn structure in it than stuff without it. Can you offer a concrete
common case where some particular styling would be much harder to do in a form
rendered with peppercorn structure in it than without?
> The granular form rendering part is really important also. Having worked
> in various environments, the death of a form library is judged by how
> flexible it is to the arrival of html from web department (which itself
> is dependant on how the UX/design department want to do things).
This isn't really an architecture issue. For example, when I made Chameleon
templates for Formish, I collapsed most of the little microtemplates for each
widget from the Mako original into one larger Chameleon one because we didn't
need the granularity of replacing the microtemplates individually. It's really
just a matter of how you write the templates themselves, and the granularity
can be dialed up and down as necessary.
> A lot of work went into getting formish so that it would satisfy these
> requirements (hence the callable 'templated' attributes on form elements
> which allow this).
Deform allows for the same by making a "renderer" available to the main widget
template. Of course it's all a matter of factoring too; you can write your own
widgets which have methods that render templates themselves and have the main
widget template call these methods. Not really architecture, though.
Tim Parkin
Quick question on the form naming you are using. How would you assign the label 'for' target given that the name's of each field are not unique? (e.g. if you had four 'date's which each had the name 'date' then you would still have to apply a unique id to the input field in order to target it with the label tag). I know you can wrap the form input with the label but many people don't recommend it as some accessibility browsers (and ie6) don't support clicking labels unless they have an id associated.
Tim
Tim Parkin
(responding to your earlier email here)...
Yes, I belive it would be possible to make the deform Python API very close to the Formish API. In particular, I think we could probably make the "formish" package a wrapper around deform that translates most API calls to their deform equivalents. Likewise for translating many Schemaish schemas to Colander schemas once used as a "formish" schema (when passed to formish.form.Form(...)).
Being able to reuse Formish widget template code is unlikely or at least the amount of work required for reuse would exceed the work required to rewrite all widgets in existence as far as I can tell.
I'll want to use something that uses peppercorn personally, whatever the web accessibility people might say, because it's clearly, absolutely, inarguably, by god the right thing. <wink>
On 3/29/10 8:42 AM, Tim Parkin wrote:
Just had an idea...
re: the form naming.. If we consult with some professional html building
and web accessibility people and survey them to see what they think,
would you be happy to change things depending on the repsonse? (i.e. if
they say they would like dotted hooks of some type then go with that or
if they say they are happy with hidden fields and single level names,
then go with that?)
But for the nonbelievers, we *could* make this pluggable. There's exactly one place in the code that parses the form post data and turns it into a "pstruct" (which stands for "peppercorn structure", but it's just a name really).
- When serializing a form, the system would need to provide an API to generate a dotted name from the current field.
- The variabledecode-style sequence widget will requires a bunch of hairy javascript for dealing with sequence manipulation.
On the other hand, if we can avoid it being a necessity, I'd like to. ;-) The peppercorn deserialization stuff is good exactly *because* it avoids the need to write code that deals with sequence numbering. This is, AFAICT, a major reason as to why the codebase is much smaller. And in my testing so far, it's easy enough to style via CSS. You didn't respond to my counterarguments about this in the second-prior email I sent, and I'd like to understand exactly why you think styling this stuff will be hard. I've had no problems at all so far.
Often things can be made better with minor spelling and appearance tweaks. Would it help to change peppercorn to expect the following kinds of controls back from an HTTP post?
people [
person {
first Bob
last Dobbins
}
person {
first Fred
last Flintstone
}
]
In other words, the beginning of a sequence would be spelled
<input type="hidden" name="sequencename ["/>
And the beginning of a mapping would be spelled:
<input type="hidden" name="mappingname {"/>
The end of a sequence would be spelled:
<input type="hidden" name="]"/>
The end of a mapping would be spelled:
<input type="hidden" name="}"/>
This looks a lot nicer, and might be easier to swallow as elements in the input stream.
I'm sure that most people who deal heavily with CSS styling of forms will of course hate it, because it means they need to learn something, and learning things is the enemy of most people. So I don't think we need to get a focus group together: we already know people naturally hate everything except what they're using right now. So I'm not willing to ditch peppercorn-style form renderings on that basis. I am willing to consider making it pluggable tho.
I really would like to cooperate but only if the result is something
that myself and other web development people would be happy using..
OTOH, as far as I can tell, designers *already* hate generated forms. As long as sufficient documentation exists for styling, it seems pretty dubious that people will *actually* have much more of a problem styling stuff with peppercorn structure in it than stuff without it. Can you offer a concrete common case where some particular styling would be much harder to do in a form rendered with peppercorn structure in it than without?
This isn't really an architecture issue. For example, when I made Chameleon templates for Formish, I collapsed most of the little microtemplates for each widget from the Mako original into one larger Chameleon one because we didn't need the granularity of replacing the microtemplates individually. It's really just a matter of how you write the templates themselves, and the granularity can be dialed up and down as necessary.
The granular form rendering part is really important also. Having worked
in various environments, the death of a form library is judged by how
flexible it is to the arrival of html from web department (which itself
is dependant on how the UX/design department want to do things).
Deform allows for the same by making a "renderer" available to the main widget template. Of course it's all a matter of factoring too; you can write your own widgets which have methods that render templates themselves and have the main widget template call these methods. Not really architecture, though.
A lot of work went into getting formish so that it would satisfy these
requirements (hence the callable 'templated' attributes on form elements
which allow this).
Chris McDonough
> On Mon, Mar 29, 2010 at 7:19 PM, Chris McDonough <chr...@plope.com
> <mailto:chr...@plope.com>> wrote:
>
> (responding to your earlier email here)...
>
> Yes, I belive it would be possible to make the deform Python API
> very close to the Formish API. In particular, I think we could
> probably make the "formish" package a wrapper around deform that
> translates most API calls to their deform equivalents. Likewise for
> translating many Schemaish schemas to Colander schemas once used as
> a "formish" schema (when passed to formish.form.Form(...)).
>
> Being able to reuse Formish widget template code is unlikely or at
> least the amount of work required for reuse would exceed the work
> required to rewrite all widgets in existence as far as I can tell.
>
> Don't mind rewriting templates as long as it is possible to do so - at
> the moment I've got some doubts that it actually is possible to do so
> (e.g. I can't work out how you can manage to have a date widget with dd,
> mm and yy parts which map to a python date (or datetime for that
> matter). It seems widgets serialisation/deserialisation only works as
> string serialisation.
I'll write a date parts widget tonight to show it can be done. I probably
should have done it before sending my preivous email, given you used it as an
example previously.
But if you care to imagine in the meantime, both the current
CheckedPasswordWidget and the FileUploadWidget in deform actually map multiple
input values to a single output value.
Deserialization of form submission by Peppercorn data generates a "pstruct".
The act of converting this big "pstruct" into smaller bits consumable by the
schema library is "widget deserialization". In this process, each widget owns
a part of the pstruct. When it deserializes a pstruct piece, each widget
generates a "cstruct" piece, which can either be the same as the pstruct piece
it's responsible for deserializing or a simpler value derived from the pstruct
piece.
When the widget system is done deserializing all the pstruct pieces, it
composes a big "cstruct" out of the result and passes it off to the schema
library. The schema library then further deserializes the cstruct to an
"appstruct" consumable by the application:
formdata -> widget -> schema -> application
^ ^ ^
| | |
pstruct cstruct appstruct
Peppercorn is responsible for generating a pstruct from raw form controls. The
widget system is responsible for generating a cstruct consumable by the schema
system during widget deserialization. The schema system is responsible for
generating an appstruct during schema deserialization.
Each serialized representation (pstruct/cstruct) can be as complex or as simple
as it needs to be, including being represented by more "raw" values than it
deserializes to downstream.
> I'll want to use something that uses peppercorn personally, whatever
> the web accessibility people might say, because it's clearly,
> absolutely, inarguably, by god the right thing. <wink>
>
>
> It's definitely the clever thing :-)
I'll take it you mean that as a bad thing. ;-)
> But for the nonbelievers, we *could* make this pluggable. There's
> exactly one place in the code that parses the form post data and
> turns it into a "pstruct" (which stands for "peppercorn structure",
> but it's just a name really).
>
> I'd be interested in knowing what you think the main advantage of using
> peppercorn is being as it 's not visible to the user of deforma and it
> makes the forms themselves less readable (I frequently use the post
> structures for debugging for instance). I realise it's clever but I'm
> not sure it's actually solving a problem (in fact in my mind it creates
> problems in some cases).
I'm not really sure how to respond without sounding like a tool or repeating
myself. So I'll just have to sound like one I guess.
I wrote a passable clone of the form library you currently maintain; the result
is about 1/4 of the size of the thing it clones even with 100% test coverage
code. The sequence-manipulating javascript is much simpler (although
admittedly incomplete). Widgets and templates are (IMO) simpler to write.
It's (IMO) more readable and maintainable.
But if you aren't swayed by that, I guess the main win really is the javascript
sequence stuff and the lack of numbering necessary there. There's also the
lack of any requirement that individual leaf level widgets have special HTML in
them to be a well-behaved member of a sequence.
<snip my exposition about form stream look and feel based on your response>
> - When serializing a form, the system would need to provide an API
> to generate a dotted name from the current field.
>
> - The variabledecode-style sequence widget will requires a bunch of
> hairy javascript for dealing with sequence manipulation.
>
> See my question in the previous email - I think you still need this for
> id's, however as discussed we can get rid of sequence numbers in the
> form names themselves which would simplify things.
Whether the sequence numbers are inserted into ids or names doesn't matter
much. If there need to *be* sequence ids attached to anything, we've lost.
That said, I'll investigate your use case. I'm willing to investigate things
like this, because you know a *lot* more about it than I do.
> On the other hand, if we can avoid it being a necessity, I'd like
> to. ;-) The peppercorn deserialization stuff is good exactly
> *because* it avoids the need to write code that deals with sequence
> numbering. This is, AFAICT, a major reason as to why the codebase
> is much smaller. And in my testing so far, it's easy enough to
> style via CSS. You didn't respond to my counterarguments about this
> in the second-prior email I sent, and I'd like to understand exactly
> why you think styling this stuff will be hard. I've had no problems
> at all so far.
>
> The code is smaller mainly because it doesn't do a lot of stuff that
> formish is doing ;-)
Hmm. I've tried to do a lot of what formish has done already. I don't think
I'm that far off, actually. I think I understand formish and know its
capabilities, I've been working with it for a reasonable while now. Even if
deform got twice as big as it is now to handle all the cases, it'd still be
half the size of formish. That's still a win to me personally.
The entire point of peppercorn is to turn the stuff in the form submission data
into something a bit more like what you can achieve via JSON, so peppercorn
just wouldn't be involved when JSON is used.
> I really would like to cooperate but only if the result is something
> that myself and other web development people would be happy using..
>
>
> I'm sure that most people who deal heavily with CSS styling of forms
> will of course hate it, because it means they need to learn
> something, and learning things is the enemy of most people. So I
> don't think we need to get a focus group together: we already know
> people naturally hate everything except what they're using right
> now. So I'm not willing to ditch peppercorn-style form renderings
> on that basis. I am willing to consider making it pluggable tho.
>
> Sounds like the goal isn't to make something targeting web developers
> and commercial environments (where the html isn't built by the developer).
All form generation has tradeoffs. If I didnt want the form generated, I would
just write it by hand, right? I can see places where, yes, I'd want to
generate a form by hand that would be handled by formish/deform on the
submission side. And that's just as possible in both systems. It's just
(hopefully) not the 90% case.
> OTOH, as far as I can tell, designers *already* hate generated
> forms. As long as sufficient documentation exists for styling, it
> seems pretty dubious that people will *actually* have much more of a
> problem styling stuff with peppercorn structure in it than stuff
> without it. Can you offer a concrete common case where some
> particular styling would be much harder to do in a form rendered
> with peppercorn structure in it than without?
>
> The hate generated forms because it's impossible to customise them,
> something that formish is the only library that achieves..
>
> I'll send you a sample form that we're using in an application and you
> can tell me how the html would look with deform and we'll see how easy
> it would be to get the right hooks for css/js. Nothing is impossible -
> some ways make the path easy, some make you fight..
That sounds fine.
> The granular form rendering part is really important also.
> Having worked
> in various environments, the death of a form library is judged
> by how
> flexible it is to the arrival of html from web department (which
> itself
> is dependant on how the UX/design department want to do things).
>
>
> This isn't really an architecture issue. For example, when I made
> Chameleon templates for Formish, I collapsed most of the little
> microtemplates for each widget from the Mako original into one
> larger Chameleon one because we didn't need the granularity of
> replacing the microtemplates individually. It's really just a
> matter of how you write the templates themselves, and the
> granularity can be dialed up and down as necessary.
>
> Ah - so because you don't use it means it's not useful ;-) Generally you
> don't need to have customised forms, however that 2% when you need to,
> you really need to.
What? I.. what? I'm not negating the need for smaller templates? I don't
know what I said wrong here.
> A lot of work went into getting formish so that it would satisfy
> these
> requirements (hence the callable 'templated' attributes on form
> elements
> which allow this).
>
>
> Deform allows for the same by making a "renderer" available to the
> main widget template. Of course it's all a matter of factoring too;
> you can write your own widgets which have methods that render
> templates themselves and have the main widget template call these
> methods. Not really architecture, though.
>
>
> Happy to carry on the conversation - hopefully I'll improve the formish
> library so people can choose between arcane convoluted form name
> encoding and elegant logical form name encoding (I'm sure we'll disagree
> on which is which though).
Hopefully this isn't a competition. I'm here asking you, (really begging) you
for collaboration on this thing, because I respect your software design skills.
Chris McDonough
> I'll write a date parts widget tonight to show it can be done. I
> probably should have done it before sending my preivous email, given you
> used it as an example previously.
Done at http://svn.repoze.org/deform/trunk/deform/widget.py
Lest you be completely confused, like most of the other widgets in deform, it
deals almost exclusively in strings because the schema type that it fronts for
serializes a native datetime.date to a string before it ever reaches the
widget. The widget then uses the schema-serialized string as input to the
template. Likewise, it converts the parts to a single dash-separated string
during deserialization so the schema type can consume it as input during its
own deserialization (where it will be converted to a datetime.date object if it
passes validation). The schema type is responsible for all validation.
This example maybe demonstrates the presumed division of labor between schema
types and widgets in deform better than the exposition I tried in the last
email. The division of labor between schema type and widget is largely by
convention; a widget is free to do more or less validation and conversion on
its own as long as the schema type it's a widget for can deal with its return
value during deserialization.
>> See my question in the previous email - I think you still need this for
>> id's, however as discussed we can get rid of sequence numbers in the
>> form names themselves which would simplify things.
>
> Whether the sequence numbers are inserted into ids or names doesn't
> matter much. If there need to *be* sequence ids attached to anything,
> we've lost. That said, I'll investigate your use case. I'm willing to
> investigate things like this, because you know a *lot* more about it
> than I do.
I did investigate this, and it is indeed a genuine problem that can't be solved
without assigning ids to form elements. That sucks. Thanks for the heads up.
This particular one might be solved with JavaScript that assigned ids to input
elements dynamically at document load time (and sequence add/remove/reorder)
which adjusted label for= attributes to point to the next element that has a
particular class. E.g.:
function reidentify(form) {
.. find all labels in the document ..
.. for each label, find the element which most directly follows them in
document order with an class attribute "labeltarget" ..
.. assign an id to the element with the labeltarget class ..
.. if the labeltarget is an input field ...
.. adjust the for= of the current label to this id ..
}
The rendered HTML would need to look like so:
<label>Blah Blah</label>
<div>
<input type="text" name="whatever" class="labeltarget"/>
</div>
It's kinda dumb that label attributes without a for= don't behave like this
already given that the behavior is for accessibility and it's not otherwise
defined.
Google tells me that these days most screen readers them can deal with DOM
reshuffling and don't rely on parsing the raw HTML text, so this should work
both for "normal" click-to-focus as well as for accessibility for the blind.
I think this might be the only problem of its type. No other HTML tag I can
find appears to reference the id of another tag as an attribute in the same
way. Any other examples you can think of here?
Chris McDonough
> I did investigate this, and it is indeed a genuine problem that can't be
> solved without assigning ids to form elements. That sucks. Thanks for
> the heads up.
>
> This particular one might be solved with JavaScript that assigned ids to
> input elements dynamically at document load time (and sequence
> add/remove/reorder) which adjusted label for= attributes to point to the
> next element that has a particular class. E.g.:
>
> function reidentify(form) {
> .. find all labels in the document ..
> .. for each label, find the element which most directly follows them in
> document order with an class attribute "labeltarget" ..
> .. assign an id to the element with the labeltarget class ..
> .. if the labeltarget is an input field ...
> .. adjust the for= of the current label to this id ..
> }
OK, that idea was insane. But this idea isn't so bad.
First, each field in the tree of fields on the server side gets a unique
"order" attribute for the duration of a rendering. The order attribute is used
within the template code to generate a unique id, e.g. it might look like:
<label tal:condition="not field.widget.hidden"
for="field${field.order}"
>${field.title}<span tal:condition="field.required" class="req"
>*</span>
</label>
<input type="text" name="${field.name}" value="${cstruct}"
id="field${field.order}"/>
The rendered HTML might look something like:
<label for="field1">Name<span class="req">*</span></label>
<input type="text" name="name" value="" id="field1" />
The add_new_items function to add a new item to an existing sequence now looks
like this (I started to use jQuery for brevity):
function add_new_item(source) {
var protonode = source.previousElementSibling;
var code = protonode.attributes['prototype'].value;
var html = decodeURIComponent(code);
var $node = $(html);
var $labels = $node.find('label');
for (var i = 0; i < $labels.length; i++) {
var label = $labels.get(i);
var labelid = label.htmlFor;
if (labelid) {
genid = randomString(10);
$node.find('#' + labelid).attr('id', genid);
label.htmlFor = genid;
};
};
insert_after($node[0], last_sibling(source));
}
In other words, when we add a new sequence item:
- Clone the prototype node
- Find all the labels in the prototype node.
- For each label referencing an id, find the node with that id.
- Change the label reference and the node id to a random string.
This appears to work ok in practice.
- C
Tim Parkin
Chris McDonough
> You'll want to namespace the id to prevent clashes with other stuff on
> the page..
Thanks.. like deform_fieldX you mean?
> p.s. Just had a quick glance at the date widget. Am I right in saying
> that you would only have one serialisation option for a date type (i.e.
> you would only ever serialise dates to dd-mm-yyyy?) What happens if I
> want to use an isoformat date serialisation?
I think that question needs clarification... what if you wanted to use an
isoformat date serialzation *where*? Can you be more specific about where
exists a serialized date format that needs to be deserialized specially? E.g.
do you mean that your application stores date serializations in a database
rather than using Python datetime objects? Or do you mean that someone might
want to type an ISO date format serialization into a text box in a form? (FTR,
yyyy-mm-dd actually is a valid ISO8601 date format serialization, I'll assume
you just mean another arbitrary date serialization).
Tim Parkin
Chris McDonough
> On 3/30/10 4:46 PM, Tim Parkin wrote:
>> p.s. Just had a quick glance at the date widget. Am I right in saying
>> that you would only have one serialisation option for a date type (i.e.
>> you would only ever serialise dates to dd-mm-yyyy?) What happens if I
>> want to use an isoformat date serialisation?
I think I understand your question.
Here are a widget's responsibilities:
- Serialize: receive a cstruct value and serialize it to HTML.
- Deserialize: receive a pstruct value and deserialize it to a cstruct value.
This is all a widget needs to know how to do. How the "cstruct" its asked to
serialize generated isn't its business and how the "pstruct" it is asked to
deserialize is generated is also not its business.
However, a widget can't serialize an arbitrary cstruct nor can it deserialize
arbitrary pstruct value. A widget, through its documentation, advertises its
capability to serialize a certain kind of cstruct and deserialize a certain
kind of pstruct.
In our case, the DatePartsWidget is willing to serialize cstructs that are in
ISO8601 date format ('YYYY-MM-DD' format) to an HTML input field. This is
spelled out in its documentation.
In the meantime, the deform.schema.Date type happens to be willing to serialize
Python "datetime.date" objects to 'YYYY-MM-DD' format. It says so in its
documentation too.
The serialization of a schema field is a cstruct. A cstruct is consumed by the
"serialize" method of a widget.
So given the following configuration:
from deform import schema
from deform import widget
from deform import form
class MySchema(schema.Schema):
date = SchemaNode(schema.Date())
schema = MySchema()
f = form.Form(schema)
f['date'].widget = widget.DatePartsWidget()
Rendering and validation will work because the "date" in the schema is a
deform.schema.Date object, which will generate a 'YYYY-MM-DD' cstruct
serialization based on a Python "datetime.date" object. These 'YYYY-MM-DD'
serializations are consumed by the "serialize" method of the DatePartsWidget
and turned into HTML input elements.
Likewise, the DatePartsWidget consumes input elements from peppercorn, then
creates a 'YYYY-MM-DD' format string in its deserialize method; this is handed
back to the deserialize method of the schema node representing a Date object
which converts it back into a Python "datetime.date" object.
This configuration would also work though:
from deform import schema
from deform import widget
from deform import form
from deform import exception
def isoformat(node, value):
try:
y, m, d = value.split('-', 2)
except Exception:
raise exception.Invalid(node, 'Value isnt in YYYY-MM-DD format')
class MySchema(schema.Schema):
date = SchemaNode(schema.String(), validator=isoformat)
schema = MySchema()
f = form.Form(schema)
f['date'].widget = widget.DatePartsWidget()
In this configuration:
The "date" in the schema is of type deform.schema.String object, which will be
provided with an arbitrary application string for serialization, presumably
'YYYY-MM-DD' strings. These 'YYYY-MM-DD' serializations are consumed by the
"serialize" method of the DatePartsWidget and turned into HTML input elements.
The DatePartsWidget then consumes input elements from peppercorn, then creates
a 'YYYY-MM-DD' format string in its deserialize method; this is handed back to
the deserialize method of the schema node representing a String, which
validates that it is indeed a 'YYYY-MM-DD' string. Unlike a schema node
representing a Date object, however, the 'YYYY-MM-DD' serialization is not
converted to a Python "datetime.date" object. It is just presented to the
application in the form 'YYYY-MM-DD'.
The relative responsibilities of the schema types and the widgets can be dialed
up and down. The default set of schema types tends to serialize things to data
structures composed entirely of lists, mappings, and strings (and file objects,
in the case of file uploads). Custom user-written schema types and widgets
that are meant to be used together can of course do whatever they like in their
serialize and deserialize methods.
The fact that a Date object happens to serialize a Python 'datetime.date'
object to a YYYY-MM-DD representation isn't really all that interesting to an
end user of deform, because the end user may never see that serialization.
What is more interesting to them will be that a Date schema type is compatible
with the DatePartsWidget. An end user might start to take more of an interest
in how schema types are serialized when he needs to write his own widgets, but
until then the intermediate string representation passed from the schema node
to the widget is mostly just a detail related to a separation of concerns they
don't need to understand.
The implementations of "Date" and "String" can be found mostly in
http://svn.repoze.org/colander/trunk/colander/__init__.py
Chris McDonough
> Yeah any prefix would help..
>
> re: serialising dates.. how about just dd-mm-yyyy or mm-dd-yyyy or maybe
> dd/mm/yyyy?
>
> i.e. is your text serialisation just a transport serialisation or is it
> meant to be used at some point? Does the widget need options to pass in
> datefirst and that takes the string apart and puts it back together again?
<hopefully answered in previous email>
> another question, how would you implement a csv style textarea that
> represented a list of date, integer tuples?
> In your confirm widget, you attach data to the field during
> roundtripping? Is this a permanent way of doing things or just a
> temporary fix?
Permanent. Field objects exist to be per-request scratchpads like this.
> I'd be concerned about what would happen if you wanted to
> handle a sequence of confirm boxes, would the cloned elements end up
> duplicating the confirm attribute?
No, because each control in a sequence will have its own field object copy.
Eg. from SequenceWidget:
subfields = [ (val, item_field.clone()) for val in cstruct ]
> By the way - Don't think I'm being competitive, I think we have
> different design goals and and are producing different libraries. I'll
> consider it a 'passable formish clone' when it can do what formish does
> though ;-)
OK, I guess that means I should stop trying to find a collaboration point. It
was worth a shot, I think, though. Sorry for the noise, and let me know if you
change your mind.
Tim Parkin
Chris McDonough
> I don't mind collaborating but I can't see us coming up with a single
> library that would satisfy both requirements without it becoming
> 'unwieldy' (i.e. two request processing modules, two widget templating
> modules, two deserialisation modules, etc). Do you think that makes any
> sense?
Well, there are a number of layers here I guess. I don't think it's an
all-or-nothing:
- We could share a "renderer" API* for template lookup.
- We could share a top-level form validation/rendering API*.
- A "Formish2" could use Colander schemas rather than Schemaish schemas.
* When I say "API" above, I don't mean we need to share implementation.
I just mean try to define an API that defines the behavior of "a renderer"
and "a form" for their primary purposes.
If all those things were true, the differences between deform and formish would
be smaller. We could share even more stuff if:
- You presented me with a showstopper reason why peppercorn can't work
in a specific common situation for developers or designers. If there
were no workaround, and I couldn't rectify the issue, I'd almost
certainly ditch peppercorn for formgen then.
.. or ..
- You came to the conclusion that there actually were no showstoppers
in using peppercorn.
But if you'd rather not work that hard, I can understand it.
Tim Parkin
Chris McDonough
> OK... I'm happy to work at it..
>
> If we are going to work at it though, perhaps it's better to talk about
> what a form library should do rather than trying to work out whether
> formish is better than deform or vice versa. If we can start from a
> clean board and design an API and feature set that people can use, we
> will not only help our own libraries (either separately or together) but
> create a resource that other people can use to review their own
> library/needs against?
Agreed. Excellent idea.
> To that end, I'll start to write up a spec of what I see is important in
> a form library (which will probably match some of formish but I'll try
> to keep an open mind). If you can do the same then we can compare
> notes.. An idea of API but not specifics might be a good idea..
That'd be awesome.
> Hopefully at some iteration we can then accomodate our agreements and
> try to create a core library with pluggables (if possible). If not, then
> we might at least be able to do things like have common schema,
> validationm, rendering or conversion modules?
Right, perfect. I'd definitely be up for collaborating this way. If you do
write down some ideas, if you put them in version control, I'd be eager to
contribute to them.
Tim Parkin
Wichert Akkerman
> Hi Chris,
>
> What you've done is undoubtedly ingenious and if we treat html as just
> an encoding stream, it makes a lot of sense. However, I'm not quite sure
> what the advantage is over using a dotted key approach? Personally I'd
> prefer to see the structure of my data easily in both html and request
> as well as in application code and dotted keys solve that problem in a
> way that is clear to the user of html, request data and application
> code.
FWIW I found the dotted approach to be cumbersome: it assumes that you
always want a list, so for the common case where you only have a single
field you still need to explicit do a .0 or [0]. I would love to see a
change where that indexing is only needed when a sequence is explicitly
asked for.
> 1) useful in javascript/css as hooks for styling/progressive enhancement
> (and are used by formish's js in reordering/deleting but see the bottom
> pps for how we're thinking about using form ordering for this).
I don't think this is true. As Chris explained the ordering itself is
enough and no dotted names in input names is needed. In fact I consider
doing deletion based on the dot-numbering to be dangerous: if you have a
generated form, and another user removes an item from the list while you
are busy making changes in your form things will get badly out of sync
if you only rely on the dot-numbering. You can fix that by adding a
simple hidden input field that contains a database-id or other unique
token to solve that.
> 2) used when assigning widgets/defaults/etc to form elements
Is there a use case for using a different widget for the same field when
it has a different position?
> I don't particularly like the hidden field markers that are used to
> denote start and end sections of sub-sections. It seems like unnecessary
> clutter in the html to me (personal opinion) and would mean many
> progressive enhancement JS would not be straightforward to write.
I don't quite buy the javascript reasoning, but I do feel that they are
unnecessary clutter. The Zope publisher, and by extension repoze.monty,
have shown that you do not need those magic markers.
Wichert.
--
Wichert Akkerman <wic...@wiggy.net> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.
Tim Parkin
On 2010-3-23 13:58, Tim Parkin wrote:FWIW I found the dotted approach to be cumbersome: it assumes that you always want a list, so for the common case where you only have a single field you still need to explicit do a .0 or [0]. I would love to see a change where that indexing is only needed when a sequence is explicitly asked for.
I don't think this is true. As Chris explained the ordering itself is enough and no dotted names in input names is needed. In fact I consider doing deletion based on the dot-numbering to be dangerous: if you have a generated form, and another user removes an item from the list while you are busy making changes in your form things will get badly out of sync if you only rely on the dot-numbering. You can fix that by adding a simple hidden input field that contains a database-id or other unique token to solve that.
1) useful in javascript/css as hooks for styling/progressive enhancement
(and are used by formish's js in reordering/deleting but see the bottom
pps for how we're thinking about using form ordering for this).
Is there a use case for using a different widget for the same field when it has a different position?
2) used when assigning widgets/defaults/etc to form elements
I don't quite buy the javascript reasoning, but I do feel that they are unnecessary clutter. The Zope publisher, and by extension repoze.monty, have shown that you do not need those magic markers.
I don't particularly like the hidden field markers that are used to
denote start and end sections of sub-sections. It seems like unnecessary
clutter in the html to me (personal opinion) and would mean many
progressive enhancement JS would not be straightforward to write.
Wichert Akkerman
> On Sun, Apr 4, 2010 at 11:38 AM, Wichert Akkerman <wic...@wiggy.net
> <mailto:wic...@wiggy.net>> wrote:
>
> On 2010-3-23 13:58, Tim Parkin wrote:
>
> FWIW I found the dotted approach to be cumbersome: it assumes that
> you always want a list, so for the common case where you only have a
> single field you still need to explicit do a .0 or [0]. I would love
> to see a change where that indexing is only needed when a sequence
> is explicitly asked for.
>
> Hmm.. not in the usual formish. If you want a single field, it will have
> a single key. E.g. If you want a single search box then the data would
> be data['search']. I can't think of a case where you would use a
> sequence if all you wanted was a single item.
>
> Can you give me an example of the sort of form you think would end up
> cumbersome?
You notice this on the html side when rendering a template. For example
here is how I render a text field:
<label tal:define="field form.get_field('title')">Title
<sup class="required">*</sup>
<input class="autofocus span-5" type="text" maxlength="128"
name="title" value="${field.value[0]}" />
<em class="message warning"
tal:condition="field.error">${field.error}</em>
</label>
This is quite elegant except for the field.value[0] part. The pattern
for sequences is harder and I've had to subclass Form to do that cleanly
from a template. The code looks like this:
class Form(formish.Form):
def get_field(self, *name):
name = ".".join(name)
return formish.Form.get_field(self, name)
def keys(self, *name):
field = self.get_field(*name)
groups = list(field.collection_fields())
return [ group.name for group in groups]
which allows this in templates:
<ul>
<li tal:repeat="prefix form.keys('contacts')">
<label tal:define="field form.get_field(prefix, 'name')">Name
... </label>
<label tal:define="field form.get_field(prefix, 'email')">Email
... </label>
</li>
</ul>
> 1) useful in javascript/css as hooks for styling/progressive
> enhancement
> (and are used by formish's js in reordering/deleting but see the
> bottom
> pps for how we're thinking about using form ordering for this).
>
>
> I don't think this is true. As Chris explained the ordering itself
> is enough and no dotted names in input names is needed. In fact I
> consider doing deletion based on the dot-numbering to be dangerous:
> if you have a generated form, and another user removes an item from
> the list while you are busy making changes in your form things will
> get badly out of sync if you only rely on the dot-numbering. You can
> fix that by adding a simple hidden input field that contains a
> database-id or other unique token to solve that.
>
> I agre that you can use a hidden input if you want one. However, I don't
> believe you should force a hidden field on people if they don't want
> one. If you don't care which items are deleted because they are just
> scalars (for instance) then it doesn't matter which number 7 has been
> removed.
It does when you start thinking about elements in SQL storage or other
situations where you have relations between objects. Suppose I have a
one-to-many relation for company to contacts, but other tables also have
relations to the contacts table. When I update the contacts for a
company it is very important that I update the right contact even if
they are reordered or a contact is removed when editing the company. It
would be bad if the data from contact 3 would be stored in contact 2 and
contact 3 removed when you really wanted to remove contact 2. The only
way to do that is to stick the database row id in a hidden input element
so you can keep track of that.
I do not think you ever need a numbering scheme inside the input name
itself, and both Zope input marshalling and peppercorn demonstrate that.
> 2) used when assigning widgets/defaults/etc to form elements
>
>
> Is there a use case for using a different widget for the same field
> when it has a different position?
>
> Yes, you could have two fields where you want to collect a date. The
> first could be a selection of default dates (a list of Sundays for
> workship maybe) where the second item in the form could be a birth date.
> Both of these quite possible by called 'date' within a data structure
> and you would end up with a name clash and probably an id clash if you
> were using the input as the form element id.
That is not what I mean. What I meant was: is there a use case where you
have a sequence of things, and you want to use a different widget for
the same field in the 3rd item in that sequence? That is the only reason
I see for needing form.field["contact.*.name"]=MyWidget() instead of
form.field["contact.name"]=MyWidget() .
> I don't particularly like the hidden field markers that are used to
> denote start and end sections of sub-sections. It seems like
> unnecessary
> clutter in the html to me (personal opinion) and would mean many
> progressive enhancement JS would not be straightforward to write.
>
>
> I don't quite buy the javascript reasoning, but I do feel that they
> are unnecessary clutter. The Zope publisher, and by extension
> repoze.monty, have shown that you do not need those magic markers.
>
>
> I can imagine a problem where you have two sequences next door to each
> other which both have the same name e.g. ['home']['emails'] and
> ['work']['emails'] The name of the input elements would all be 'emails'
> and without the markers you wouldn't know where the home emails ended
> and the work emails began.
>
> repoze.monty looks suspiciously like it uses dotted names?
It does. I have no problem at all with dotted names. I just don't see a
good reason to use a numbering system in the name like formish does or
require special markers like peppercorn.
Wichert.
Tim Parkin
Wichert Akkerman
So far it hasn't come up yet: I don't need a label/for since I put the
input elements inside the label, and so far I haven't had a need to put
an id on input elements.
Wichert.
Chris McDonough
http://lists.repoze.org/pipermail/repoze-dev/2010-May/003229.html
I credit formish in its docs:
http://docs.repoze.org/deform
Although I've gone off the Formish reservation, I'm still interested in
particpating in discussions about the role of form generation systems in
general.
> You received this message because you are subscribed to the Google
> Groups "ish.io" group.
> To post to this group, send an email to is...@googlegroups.com.
> To unsubscribe from this group, send email to ishio