Fwd: [Pylons] #405: [Patch] Split up @validate

[Mike Orr]

---------- Forwarded message ----------
#405: [Patch] Split up @validate

Comment(by Ben Bangert):

 I'm rather tempted to go with Marcus's idea:
 http://marcuscavanaugh.com/posts/pylons-django-forms/

 Though of course it kind of sucks to need Django as a dependency just to
 do forms. I do like the idiom it uses (I've used the same idiom in the
 past with FormEncode, but its messier), and FormEncode does seem to be
 getting a bit crufty. Course, I like that FormEncode sticks to one thing,
 and does it... mostly well (is there any docs anywhere on how the hell to
 write fancy validators vs partial validators, etc???).

 Maybe get the django.forms stuff forked off of Django? Would Marcus be up
 for maintaining just a django.forms fork? ;)

--
Ticket URL: <http://pylonshq.com/project/pylonshq/ticket/405#comment:9>
=== End forward

Interesting. So if the boss is questioning FormEncode, I guess we'd
better decide which way to go.

Writing a FormEncode manual is next on my plate after the WebHelpers
docs. I't'll cover the missing pieces: when to use schema, how to use
validators with a framework, how to write a FancyValidator with
advanced features (although I don't really understand ForEach), and
documenting the undocumented validators and which ones are worth
using. I wish Sphinx didn't take forever to make all the WebHelpers
docstrings consistent, and make the layout complementary between
projects. That's what's mainly bogged me down.

I wrote earlier about @validate vs WTForms vs django-forms.

http://groups.google.com/group/pylons-discuss/browse_thread/thread/2e88433546d97410/cf74daf7c3417c8f?lnk=gst&q=wtforms#cf74daf7c3417c8f

http://groups.google.com/group/pylons-discuss/browse_thread/thread/b2762d57ef33be0b/4050e4b1a9762e8b?lnk=gst&q=wtforms#4050e4b1a9762e8b

WTForms works similar to django-forms but is standalone. I initially
supported it, but there are some things FormEncode does that it
doesn't (e.g., validating the configuration dict), and other things
people may be using that we don't know about. Plus WTForms would make
the form helpers arguably redundant... we'd have two ways to do it.
So while I think it may have been good to start with that approach in
Pylons, I'm not sure changing the form libraries would be a good idea
at this point. The biggest needs seem to be:

1. An alternative to @validate. There are some patches in the ticket.
Going to a one-method approach (form display and processing in the
same controller action) would also simplify things.

2. Document the undocumented parts of FormEncode.

3. Add an alternate resource route to Routes, that uses the one-method approach.

--
Mike Orr <slugg...@gmail.com>

[Ben Bangert]

On Nov 19, 2009, at 7:14 AM, Mike Orr wrote:

> Writing a FormEncode manual is next on my plate after the WebHelpers
> docs. I't'll cover the missing pieces: when to use schema, how to use
> validators with a framework, how to write a FancyValidator with
> advanced features (although I don't really understand ForEach), and
> documenting the undocumented validators and which ones are worth
> using. I wish Sphinx didn't take forever to make all the WebHelpers
> docstrings consistent, and make the layout complementary between
> projects. That's what's mainly bogged me down.

It's not just the lack of documentation. It does things downright.... weird. It's also pretty much unmaintained. Sure, patches will be applied usually, but other than that, its 'done'. For better or worse.

It does tons of things in odd ways, writing my own validators never feels quite right, and I still have no clue why its so incredibly painful to write a FancyValidator, or even what a partial validation is. I'm also not a fan of having to catch exceptions just to find out a form is invalid. Why can't I just ask it if its valid or not? It makes my controller code more annoying to write than if I was using WTForms or django-forms as well.

Overall though, I really just don't understand how FormEncode works, and not having any docs that kindly explain it with examples, when/how to use partial validation, what class attributes do I need in my own custom validators for each, etc. makes it painful. Ideally, I'd like a simple widget extension system perhaps similar to django-forms that lets me provide basic blocks that spit out what I want, as I find the way TW handles 'resource injection' to be way too overbearing (I want to place the CSS and JS *exactly* where I want them damn it! Please not more middleware!!).

I like the django-forms and WTForms have the 'use what you want' approach of letting the entire form be generated, or you can generate just individual elements of it. With TW, its all or nothing it seems. I'm either using Webhelpers for parts of the form, or I try and generate it all. And of course, its a pain to override TW widgets, while its quite easy in django-forms or WTForms. But I also agree with what Ian wrote on the form threads, where things can become kind of sucky when validation and form generation get too tied up.

> I wrote earlier about @validate vs WTForms vs django-forms.
>
> http://groups.google.com/group/pylons-discuss/browse_thread/thread/2e88433546d97410/cf74daf7c3417c8f?lnk=gst&q=wtforms#cf74daf7c3417c8f
>
> http://groups.google.com/group/pylons-discuss/browse_thread/thread/b2762d57ef33be0b/4050e4b1a9762e8b?lnk=gst&q=wtforms#4050e4b1a9762e8b
>
> WTForms works similar to django-forms but is standalone. I initially
> supported it, but there are some things FormEncode does that it
> doesn't (e.g., validating the configuration dict), and other things

Doesn't any form lib have to validate a dict to handle the request params dict?

> people may be using that we don't know about. Plus WTForms would make
> the form helpers arguably redundant... we'd have two ways to do it.
> So while I think it may have been good to start with that approach in
> Pylons, I'm not sure changing the form libraries would be a good idea
> at this point. The biggest needs seem to be:
>
> 1. An alternative to @validate. There are some patches in the ticket.
> Going to a one-method approach (form display and processing in the
> same controller action) would also simplify things.
>
> 2. Document the undocumented parts of FormEncode.
>
> 3. Add an alternate resource route to Routes, that uses the one-method approach.

Now, at the same time, I need a lot of the power of FormEncode that the others don't seem to have. Like partial validation, which I think I've needed in the past, but could never figure out how to use as there's no docs on it. ;)

I think your goal of increasing the docs for FE will help a lot, I'm talking with Chris Perkins about ways TW2 can make it friendlier to make forms with FE schemas and validators as I'm not a fan of many things TW does at the moment.

Also on a side-note, I'm starting to really wonder if there's any point in trying to change @validate before 1.0 or if we should just release 1.0 now with the legacy removal stuff and the few bugfixes that have come up since 0.9.7. Changing @validate now would break a ton of code for 1.0. Maybe rather than fix @validate, we should consider deprecating it entirely post-1.0 and move to a more declarative style since specifying 10 keyword options to a validator looks god-aweful anyways.

Thoughts on that?

Cheers,
Ben

[Mike Orr]

On Thu, Nov 19, 2009 at 3:25 PM, Ben Bangert <b...@groovie.org> wrote:
> On Nov 19, 2009, at 7:14 AM, Mike Orr wrote:
>
> It's not just the lack of documentation. It does things downright.... weird. It's also pretty much unmaintained. Sure, patches will be applied usually, but other than that, its 'done'. For better or worse.

Ian has been looking for a maintainer since April. I thought I might
do it after I get the manual done. But perhaps a rewrite is in order,
starting from what we want it to do.

>> WTForms works similar to django-forms but is standalone. I initially
>> supported it, but there are some things FormEncode does that it
>> doesn't (e.g., validating the configuration dict), and other things
>
> Doesn't any form lib have to validate a dict to handle the request params dict?

Maybe. But it feels funny to validate a configuration with an object
with something that can also produce a totally irrelevant form for it.
And I think Pylons' lack of config validation/type conversion is not
good because it leads to obscure exceptions during requests rather
than all at once when the program starts.

> Now, at the same time, I need a lot of the power of FormEncode that the others don't seem to have. Like partial validation, which I think I've needed in the past, but could never figure out how to use as there's no docs on it. ;)

I'll try to explain partial validation but I don't understand it much either.

> Also on a side-note, I'm starting to really wonder if there's any point in trying to change @validate before 1.0 or if we should just release 1.0 now with the legacy removal stuff and the few bugfixes that have come up since 0.9.7. Changing @validate now would break a ton of code for 1.0. Maybe rather than fix @validate, we should consider deprecating it entirely post-1.0 and move to a more declarative style since specifying 10 keyword options to a validator looks god-aweful anyways.

That's not a bad idea. Pylons 2 is on the horizon, and that would give
us a chance to make significant changes then. It would also hasten
Pylons 2's adoption if it has a better form-handling library.

We can clone django-forms, but we definitely can't depend on Django.
That would make poor Noah throw himself off one of the Northern
California cliffs.

--
Mike Orr <slugg...@gmail.com>

[Marcus Cavanaugh]

On Nov 19, 7:32 pm, Mike Orr <sluggos...@gmail.com> wrote:
> We can clone django-forms, but we definitely can't depend on Django.
> That would make poor Noah throw himself off one of the Northern
> California cliffs.

Ha!

It'd be a good exercise to pull out django forms, if for no other
reason than so that I don't have to require django in its entirety in
my own projects. I was actually contacted a couple of days ago by
someone who wanted to use that django-pylons forms adapter with
AppEngine, but he couldn't because AppEngine has a file limit and he
didn't have enough room to include Django outright.

If such a fork turns out to be useful beyond that, that's a bonus. If
not, nothing's lost. Maybe tomorrow I'll try to yank out django.forms
and see how much Django code gets pulled along with it. (I haven't
seen anyone do this yet, but I'm surprised if no one has. I don't see
anything on github or mercurial at first glance though.)

I think Ben's thoughts on depreciating @validate would be the best for
pylons in the long run. I think it'd be fruitless to try to fix it at
this point.

Marcus

[Damjan]

One options would be to use http://dev.pocoo.org/hg/bureaucracy-main/
it's a new library, very alpha at this time, but it's been pulled from
some active projects like zine/solace and made by the Werkzeug/Pocoo
people.

If the idea is to write/rewrite something from scratch, this could be
a chance to join forces.

[whit]

having recently run this gauntlet, what was the decision about this issue?

deform is looking nice... formencode is still kinda an ugly wierd kid
and begging for rewriting. did anyone ever fork django-forms?

-w

--
You received this message because you are subscribed to the Google Groups "pylons-devel" group.
To post to this group, send email to pylons...@googlegroups.com.
To unsubscribe from this group, send email to pylons-devel...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-devel?hl=en.

[Mike Orr]

On Fri, May 21, 2010 at 12:02 PM, whit <wh...@surveymonkey.com> wrote:
> Marcus Cavanaugh wrote:

[Discussion about replacing @validate with Django forms.]

>> I think Ben's thoughts on depreciating @validate would be the best for
>> pylons in the long run. I think it'd be fruitless to try to fix it at
>> this point.
>
> having recently run this gauntlet, what was the decision about this issue?
>
> deform is looking nice... formencode is still kinda an ugly wierd kid and
> begging for rewriting.  did anyone ever fork django-forms?

Nothing will happen in 1.0. An alternative to @validate might be
provided in 1.1. Ben said he'll outline a new roadmap for Pylons soon
after 1.0 is released.

We've generally decided to keep FormEncode because of its wide
flexibility (it can validate anything, and doesn't have HTML-centric
vocabulary), so whatever we choose would have to work with it. Some
people have written standalone pieces of @validate which could be used
inside actions or combined in a decorator, so we'd have to evaluate
those.

If you have any specific ideas, you can add them to the "Split up
@validate" ticket.

--
Mike Orr <slugg...@gmail.com>

[whit]

Mike Orr wrote:
> On Fri, May 21, 2010 at 12:02 PM, whit <wh...@surveymonkey.com> wrote:
>> Marcus Cavanaugh wrote:
>
> [Discussion about replacing @validate with Django forms.]
>
>>> I think Ben's thoughts on depreciating @validate would be the best for
>>> pylons in the long run. I think it'd be fruitless to try to fix it at
>>> this point.
>> having recently run this gauntlet, what was the decision about this issue?
>>
>> deform is looking nice... formencode is still kinda an ugly wierd kid and
>> begging for rewriting. did anyone ever fork django-forms?
>
> Nothing will happen in 1.0. An alternative to @validate might be
> provided in 1.1. Ben said he'll outline a new roadmap for Pylons soon
> after 1.0 is released.
>
> We've generally decided to keep FormEncode because of its wide
> flexibility (it can validate anything, and doesn't have HTML-centric
> vocabulary), so whatever we choose would have to work with it. Some
> people have written standalone pieces of @validate which could be used
> inside actions or combined in a decorator, so we'd have to evaluate
> those.
>
> If you have any specific ideas, you can add them to the "Split up
> @validate" ticket.
>

works for me.

I hacked validate to allow it to work with peppercorn (does kinda the
same thing as variable_decode, but respects repeated var names). This
also help htmlfill deal with the repeated variable names that might
occur using peppercorn.

http://gist.github.com/396568 (will append link on the ticket)

-w