Validation Aware Models and django.forms on steroids
Joseph Kocherhans
get more powerful, models grow validation. This is an attempt to
clarify and add to Adrian's previous proposal. I hope it takes care of
people's concerns. Here are some details:
Forms and FormFields are intended to be used for web applications
only. Models do validation, so using forms isn't necessary when
manipulating data directly in python. Also, I think something similar
to this would allow for recursive forms (edit_inline behavior), but I
haven't worked out all the details.
Models would grow a validate method and validate=True is added as an
argument to Model's save methods. Models would not do any type
coercion. (I guess they could, but I think most type coercion should
happen in the form fields, not the model fields.)
I'm ignoring a bunch of metaclass magic that needs to go on here, but
I hope the intentions are clear. Bring on the pseudocode...
class Form(object):
def __init__(self, request, prefix=''):
self.request = request
# prefix corresponds to a prefix to http variable names. The prefix
# should be used to strip off the first part of the var names.
# Hopefully this will allow for recursively defined forms... in other
# words, edit_inline is available outside of the admin system.
self.prefix = prefix
# this would actually be more complicated... use prefix to get a dict
# of strings for the form fields to coerce.
self.data = request.POST.copy()
def _get_model_obj(self):
"""
return the cached model object, or init/cache/return one.
"""
model = property(_get_model_obj)
def set_defaults(self):
"""
Call get_default() (hopefully passing in self.request) on each
FormField, and set the appropriate attributes on self.model.
"""
def validate(self):
"""
Basically just return self.model.validate(). Call validate for child
forms as well.
"""
def save(self, validate=True):
"""
Basically just call self.model.save(validate=validate) Call save for
child forms as well.
"""
def html(self):
"""
This is really just convenience for people who are too lazy to write
real templates. Returns a very generic form redered as html. It
should probably have enough css hooks to make it workable though. It
is meant to be called in a template like {{ form.html }}
"""
class ArticleAddForm(Form):
# author will not be editable or displayed
exclude_fields = ('author',)
extend_fields = (
formfields.EmailField(field_name="from", is_required=True),
)
class ArticleChangeForm(Form):
# author will be displayed, but not editable
readonly_fields = ('author',)
class Article(models.Models):
name = models.CharField(maxlength=100)
body = models.TextField()
author = models.AuthorField()
class Meta:
# this gets used in generic create views
add_form = ArticleAddForm
class Admin:
# this gets used in the admin system
change_form = ArticleChangeForm
# Usage:
def myview(request):
add_form = Article.AddForm(request)
errors = add_form.validate()
if not errors:
add_form.save(validate=False)
return HttpResponseRedirect('the_next_page')
ctx = RequestContext({'form': add_form})
return render_to_response('template', ctx)
I hope something like this will come out of Adrian's validation aware
model proposal. This would solve those issues, and add a ton of
flexibility to Django. There are many details that need to be pinned
down, but I think something like this should be workable.
Also, there are things that I've probably overlooked. What are they?
Joseph
Christopher Lenz
> The short version of this is really, forms and manipulators merge and
> get more powerful, models grow validation. This is an attempt to
> clarify and add to Adrian's previous proposal. I hope it takes care of
> people's concerns. Here are some details:
>
> Forms and FormFields are intended to be used for web applications
> only. Models do validation, so using forms isn't necessary when
> manipulating data directly in python. Also, I think something similar
> to this would allow for recursive forms (edit_inline behavior), but I
> haven't worked out all the details.
>
> Models would grow a validate method and validate=True is added as an
> argument to Model's save methods. Models would not do any type
> coercion. (I guess they could, but I think most type coercion should
> happen in the form fields, not the model fields.)
Agreed. Models should expect the correct Python types, coercion from
strings/whatever should be in the web layer.
> I'm ignoring a bunch of metaclass magic that needs to go on here, but
> I hope the intentions are clear. Bring on the pseudocode...
>
>
> class Form(object):
> def __init__(self, request, prefix=''):
> self.request = request
> # prefix corresponds to a prefix to http variable names.
> The prefix
> # should be used to strip off the first part of the var names.
> # Hopefully this will allow for recursively defined
> forms... in other
> # words, edit_inline is available outside of the admin system.
> self.prefix = prefix
> # this would actually be more complicated... use prefix to
> get a dict
> # of strings for the form fields to coerce.
> self.data = request.POST.copy()
>
> def _get_model_obj(self):
> """
> return the cached model object, or init/cache/return one.
> """
>
> model = property(_get_model_obj)
So how is a Form connected to a Model?
> def set_defaults(self):
> """
> Call get_default() (hopefully passing in self.request) on each
> FormField, and set the appropriate attributes on self.model.
> """
>
> def validate(self):
> """
> Basically just return self.model.validate(). Call validate
> for child
> forms as well.
> """
>
> def save(self, validate=True):
> """
> Basically just call self.model.save(validate=validate) Call
> save for
> child forms as well.
> """
>
> def html(self):
> """
> This is really just convenience for people who are too lazy
> to write
> real templates. Returns a very generic form redered as
> html. It
> should probably have enough css hooks to make it workable
> though. It
> is meant to be called in a template like {{ form.html }}
> """
Maybe this would be better implemented as a template tag?
I've probably missed some of the previous discussion, but does all
this mean that Manipulators would go away completely?
Cheers,
Chris
--
Christopher Lenz
cmlenz at gmx.de
http://www.cmlenz.net/
Joseph Kocherhans
>
> Am 08.03.2006 um 16:20 schrieb Joseph Kocherhans:
>
> So how is a Form connected to a Model?
This is the coolest part I think. Unfortunately I buried it with a
bunch of other stuff. The AddForm and ChangeForm would be created
automatically, just like the default Manager. You can override them in
the same fashion. See
http://code.djangoproject.com/wiki/RemovingTheMagic#YoucanoverridedefaultQuerySets
for how Manager overrides work.
class ArticleChangeForm(Form):
# author will be displayed, but not editable
readonly_fields = ('author',)
class Article(models.Models):
name = models.CharField(maxlength=100)
body = models.TextField()
author = models.AuthorField()
class Meta:
# this gets used in generic create views. It overrides the
default AddForm.
add_form = ArticleAddForm
class Admin:
# this gets used in the admin system. It overrides the default
ChangeForm.
change_form = ArticleChangeForm
> > Also, there are things that I've probably overlooked. What are they?
>
> I've probably missed some of the previous discussion, but does all
> this mean that Manipulators would go away completely?
There has been confusion regarding that. Adrian has said that
*automatic* maipulators would go away, which I interpret to mean
"don't use manipulators unless you are doing a custom form, i.e.
changing the manipulator's fields attribute" I may be misinterpreting
though.
What I am proposing here is that manipulators go away entirely. Their
functionality would be replaced by forms.
Joseph
Adrian Holovaty
Let's make this happen!
Joseph (in the e-mail below) has spelled out a pretty decent plan for
the new manipulator scheme. I see that we've got another proposal in
Trac by Brant Harris -- http://code.djangoproject.com/ticket/2586.
Let's get something decided and implemented for Django's next version;
this will be one of the last big(ish) changes before 1.0.
Models already have a validate() method, but it's undocumented and not
all validation (for all field types) is implemented.
Put succinctly, we're looking to replace the automatic AddManipulators
and ChangeManipulators that Django creates for each model. We're
looking for something more natural, more flexible and more sexy.
Definitely more sexy.
How To Be Sexy, Rule 1: The word "manipulator" has really got to go.
Thoughts/comments/suggestions on Joseph's plan below, and on Brant's
plan in Trac?
Adrian
--
Adrian Holovaty
holovaty.com | djangoproject.com
James Bennett
> Thoughts/comments/suggestions on Joseph's plan below, and on Brant's
> plan in Trac?
I think Brant's rocking the sexiness; the concept of validation
behaving as a try/except block feels nice to me. And bidding good-bye
to 'if errors', FormWrapper and friends will be a huge win.
The exact details could use a little fine-tuning, though; a couple
things in particular jump out at me:
1. I'm not sure I like the idea of manipulators having a 'process'
method which does everything; it would feel more natural to just try
'manipulator.save()', have that save if all is well, and catch any
validation errors.
2. Since we're already striving hard to get rid of 'get_absolute_url'
I'm not sure how I feel about introducing a 'get_update_url'.
3. Doing 'manipulator.process(request)' is probably unacceptable,
since there are tons of common use cases where you'll want to fiddle
with the soon-to-be-validated data before you let the manipulator get
involved. I can live with 'new_data' in this case.
--
"May the forces of evil become confused on the way to your house."
-- George Carlin
Brantley Harris
On 8/23/06, Adrian Holovaty <holo...@gmail.com> wrote:
>
> How To Be Sexy, Rule 1: The word "manipulator" has really got to go.
>
> Thoughts/comments/suggestions on Joseph's plan below, and on Brant's
> plan in Trac?
>
I know you want to get rid of the concept of "Manipulator". So two things:
1. I will think very hard of alternate ways of doing this.
2. Currently I think the idea of the "Manipulator" is really quite
great. It allows you to define, in a simple way, a Form process. But
also, the whole idea of the manipulator means that the "manipulation"
of an object is in the "model" space, essentially, and not in the
"view" space. Maybe it's a philosophic question, but I see it best
defined in the "model" space because then it provides a modular
process for views to leverage.
Maybe a constructive thing would be to decide what we DON'T like in
the current Manipulators system?
Personally most of the problems I see, have been sorted out in my proposal.
Brantley Harris
> 1. I'm not sure I like the idea of manipulators having a 'process'
> method which does everything; it would feel more natural to just try
> 'manipulator.save()', have that save if all is well, and catch any
> validation errors.
The problem is that to make it usefull to the user (read: api-user /
developer), you have to put the model save in a try / except block so
that if there is a validation error, it can raise the form.
Otherwise, the user will have to provide their own try / except block.
> 2. Since we're already striving hard to get rid of 'get_absolute_url'
> I'm not sure how I feel about introducing a 'get_update_url'.
I totally agree. Maybe get_url() and get_edit_url()? I think most
objects should have a .url property. And come to think of it, this
might be an entry for some sort of routes implimentation, that's not
routes (man I hate the routes syntax).
> 3. Doing 'manipulator.process(request)' is probably unacceptable,
> since there are tons of common use cases where you'll want to fiddle
> with the soon-to-be-validated data before you let the manipulator get
> involved. I can live with 'new_data' in this case.
Yeah, you know I agree. In an earlier rendition I was doing
manipulator.process(request.POST), which actually makes a lot more
sense to me. But that disallows easy access to the request.user,
which makes it harder to make a manipulator that, for instance,
automatically updates an author field with the current user.
James Bennett
> The problem is that to make it usefull to the user (read: api-user /
> developer), you have to put the model save in a try / except block so
> that if there is a validation error, it can raise the form.
> Otherwise, the user will have to provide their own try / except block.
If I'm reading the example on the wiki correctly, you'd already need
to do this in your view -- the difference is whether the try/except
wraps around 'manipulator.process' or 'manipulator.save', and though
it's mainly semantics, I think the latter is preferable.
Brantley Harris
>
> On 8/23/06, Brantley Harris <deadw...@gmail.com> wrote:
> > The problem is that to make it usefull to the user (read: api-user /
> > developer), you have to put the model save in a try / except block so
> > that if there is a validation error, it can raise the form.
> > Otherwise, the user will have to provide their own try / except block.
>
> If I'm reading the example on the wiki correctly, you'd already need
> to do this in your view -- the difference is whether the try/except
> wraps around 'manipulator.process' or 'manipulator.save', and though
> it's mainly semantics, I think the latter is preferable.
No, watch for the difference between a ValidationError being raised
and a Form exception being raised. In the ValidationError case, it
must be saved and returned with the other validation errors in the
given step (1. conversion; 2. form validation; 3. model validation),
and then finally a Form exception must be raised.
Check out the ._save() function in the Manipulator. It wraps the user
provided .save(), and catches the ValidationError and raises a Form.
Come to think about it, ValidationError really should be
ValidationErrors, with an s. If at all possible we should get as many
errors out as we can to give to the form-user. That's why I pick out
those three steps above; no later step can go forward, or even check
for errors, before the previous steps are complete.
I did play with:
m = Manipulator(request.POST)
m.save()
That has some merit, I think. But again, the user would have to
provide a try / except block. Something like this:
try:
m = Manipulator(request.POST)
m.save()
except ValidationErrors, errors:
render_to_response('polls/create.html', {'form': m.get_form(errors)})
Not very pretty...
James Bennett
> No, watch for the difference between a ValidationError being raised
> and a Form exception being raised. In the ValidationError case, it
> must be saved and returned with the other validation errors in the
> given step (1. conversion; 2. form validation; 3. model validation),
> and then finally a Form exception must be raised.
Huh?
Here's one of the example views ripped straight out of the wiki page:
def create_poll(request):
try:
m = Poll.CreateManipulator()
poll = m.process(request)
return HttpResponseRedirect('/polls/%d/' % poll.id)
except Form, form:
return render_to_response('polls/create.html', {'form': form})
Where is this bit about catching "ValidationError" coming from?
Ivan Sagalaev
> Maybe it's a philosophic question, but I see it best
> defined in the "model" space because then it provides a modular
> process for views to leverage.
Manipulators can (and do) serve not only as model savers. They can
authorize, register, send mail etc. Manipulator is just a very good
pattern for dealing with form POSTs.
I think that manipulators that save models should still look like other
manipulators and this logic is best suitable to live in a view. If we
relay an entire model manipulation into the model itself then we'll get
two kinds of manipulators: those living in views and those living in models.
JP
I like James Bennetts's example a lot, but I'd like it more if the form
class were ModelForm, descending from a more general Form class that
would look much like the base Manipulator class of today.
I'm afraid that I find the idea in Brantley Harris's proposal of
raising a Form as an exception as a form of flow control really
counter-intuitive. Apologies and it's just my opinion of course, but it
has a sort of "too clever" feel to me. Also, exceptions are very
expensive, and I don't see the practical benefits of that usage pattern
over something like:
form = Poll.CreateForm(request)
errors = form.validate()
if errors:
...
else:
...
Last up, a question for James. In your example you have exclude_fields
and extra_fields. What if I want to bind a different form field type to
a model field? What would the synax be for that? Something like:
class CrazyArticleForm(ArticleAddForm):
override_fields = ( myapp.CrazyField(field_name='title',
model_field='title', )
Maybe? (I don't like 'override' there but I'm not sure what else to
call it.) And in service of that sort of thing, I think at the input
conversion phase, the form should delegate to each field to pull its
data out of the request, since some fields may be singular in concept
but spawn multiple html elements.
That brings up a related question: can one form field bind to multiple
model fields? (I'm thinking of file upload, where you get can back a
filename and a mime type and the file contents).
JP
Dan Watson
> raising a Form as an exception as a form of flow control really
> counter-intuitive. Apologies and it's just my opinion of course, but it
> has a sort of "too clever" feel to me. Also, exceptions are very
> expensive, and I don't see the practical benefits of that usage pattern
> over something like:
Actually that was one of my favorite pieces. I think it captures what's
going on in a very intuitive way: try to create/update, if that fails,
redisplay with errors. Maybe a good solution to the "too clever"
feeling would be to, instead of raising a Form as an exception, raising
a real error, with a property/accessor to get at a partially-completed
form?
try:
m = Poll.CreateManipulator()
poll = m.save( req.POST ) #, req.user?
return HttpResponseRedirect( '/poll/%d/' % poll.id )
except ManipulatorError, e:
# print e.errors
return render_to_response( 'poll/create.html', {'form': e.form} )
Ivan Sagalaev
> Actually that was one of my favorite pieces. I think it captures what's
> going on in a very intuitive way: try to create/update, if that fails,
> redisplay with errors.
I think the piece on which I agree with JP is that a _form_ serving as
an exception is counter-intuitive. I would prefer an exception to be an
error dict that is then used to display them.
Consider a form working over Ajax. In most cases you don't want to
redisplay the whole form with errors but give out just errors serialized
as JSON or XML.
James Bennett
> I like James Bennetts's example a lot, but I'd like it more if the form
> class were ModelForm, descending from a more general Form class that
> would look much like the base Manipulator class of today.
I think you're confusing me with someone else...
> I'm afraid that I find the idea in Brantley Harris's proposal of
> raising a Form as an exception as a form of flow control really
> counter-intuitive. Apologies and it's just my opinion of course, but it
> has a sort of "too clever" feel to me. Also, exceptions are very
> expensive, and I don't see the practical benefits of that usage pattern
> over something like:
The benefit, as I see it, is that it's much simpler and much more
descriptive of what's actually going on; you've got some data, you're
trying to save an object from it. Which, logically, translates into a
try/except block around the form's 'save' method; that makes the code
crystal-clear on what you're actually trying to do.
The expense of an exception is something to talk about, but I think it
does need to be talked about in the context of how often most
applications actually write to the DB, and how many of those writes
are mediated by a manipulator.
James Bennett
> I think the piece on which I agree with JP is that a _form_ serving as
> an exception is counter-intuitive. I would prefer an exception to be an
> error dict that is then used to display them.
I'm not so sure about that. The form itself was the thing that was
"bad" and needs to be dealt with, so why *shouldn't* it be the
exception?
> Consider a form working over Ajax. In most cases you don't want to
> redisplay the whole form with errors but give out just errors serialized
> as JSON or XML.
try:
p = poll_form.save(poll_data)
return HttpResponse(simplejson.dumps(p), mimetype="application/javascript")
except Form, form:
return HttpResponse(simplejson.dumps(form),
mimetype="application/javascript")
Brantley Harris
I think this is the crux of the problem, honestly. We have to decide
if manipulation should be in the "model" space or the "view" space (or
is that controller? whatev.). I rather see it in it's own space...
MMVC anyone? But you're right, they can certainly be used for other
uses, which is why the Manipulator class I made allows you to define
whatever save() you might want, including just returning the data so
that you can do something with it in the view.
Ivan Sagalaev
> I'm not so sure about that. The form itself was the thing that was
> "bad" and needs to be dealt with, so why *shouldn't* it be the
> exception?
It just feels right to me... When you're opening a file and you, say,
don't have permission you get OSError, not this "bad" file as an exception.
> try:
> p = poll_form.save(poll_data)
> return HttpResponse(simplejson.dumps(p), mimetype="application/javascript")
> except Form, form:
> return HttpResponse(simplejson.dumps(form),
> mimetype="application/javascript")
Well, of course it's doable. I'm talking about convenience for people's
mind. I'm not ready to say that my vision is the only right thing but
may be others feel like this also.
JP
> On 8/24/06, JP <jpel...@gmail.com> wrote:
> > I like James Bennetts's example a lot, but I'd like it more if the form
> > class were ModelForm, descending from a more general Form class that
> > would look much like the base Manipulator class of today.
>
> I think you're confusing me with someone else...
Sorry! I mixed up Joseph, who was quoted in Adrian's email, with your
first reply to the message with the quote. Apologies to James and
Joseph for the misattribution. I still like the ideas. :)
> > I'm afraid that I find the idea in Brantley Harris's proposal of
> > raising a Form as an exception as a form of flow control really
> > counter-intuitive. Apologies and it's just my opinion of course, but it
> > has a sort of "too clever" feel to me. Also, exceptions are very
> > expensive, and I don't see the practical benefits of that usage pattern
> > over something like:
>
> The benefit, as I see it, is that it's much simpler and much more
> descriptive of what's actually going on; you've got some data, you're
> trying to save an object from it. Which, logically, translates into a
> try/except block around the form's 'save' method; that makes the code
> crystal-clear on what you're actually trying to do.
All I can say is that I don't see it that way. For me, raising the form
as an exception (or an exception referencing the form) seems unnatural
and unintuitive, and much less explicit, readable, etc, than just
dealing with the flow of form validation and saving in the same way
that Manipulators do now. And it ignores the many uses of forms that
don't involve binding a form to single model that validates and saves
the form input. I don't think that the view-manipulator interaction is
broken, and I think it can be easily adapted to handle model-driven
validation and saving, as in Joseph's sample code.
So count me as -1 on using exceptions for form processing control, for
whatever that's worth.
JP
Jacob Kaplan-Moss
> Let's make this happen!
Indeed!
I've got some thoughts and preferences given all the ideas I've seen,
so here goes... Essentially, I think there's two tasks here:
1. Validation-aware models.
I very much want to be able to validate and save models without the
use of manipulators (or forms or whatever you call 'em). I'd like to
be able to do something like::
p = Person(**data)
try:
p.save()
except ValidationError:
handle_the_error()
The ``Model.validate()`` method that Adrian's started on should
handle this, and also nicely allows models to define custom
validation behavior outside of the manipulator framework (by
extending that ``validate()`` method).
One issue that's been raised is that ``p.save()`` could potentially
raise many validation errors, and ideally those would be communicated
up the extension stack. So probably we'd need a ``ValidationErrors``
exception that encapsulates a bunch of validation errors in a single
exception. ``Model.save()`` could raise one of these, and I'd
imagine that ``Model.validate()`` would do the same.
I think we're well on our way towards making this work. We'd need to
complete the various field-level validation, and then make it active.
2. Simplify forms, and make the word "Manipulator" go away.
Like Adrian, I can't stand the word :) It's just so... Java-ish.
The other thing I can't stand is the whole FormWrapper layer over
Manipulators; ideally the replacement would merge the two.
I quite like Joseph's proposal. I especially like being able to
control which form is used both for generic views and in the admin.
I'm also a fan of the {{ form.html }} shortcut. With some careful
use of nice semantic HTML we should be able to make that shortcut
work nearly all the time.
I'm less hot on the view syntax from Joseph's proposal; it still
seems like more steps than I'd like to use.
Brant's proposal also has some high points; in particular, the
separation of converting data from its POSTed representation is
something that we need to get right; making it an explicit step is
the right way to go.
However, this::
except Form, form:
*really* rubs me the wrong way. To me, this is a pretty counter-
intuitive use of exceptions -- exceptions should be used to indicate
an exceptional situation; not an expected one. Furthermore, PEP 352
(http://www.python.org/dev/peps/pep-0352/) -- which will be complete
in Python 3.0 - requires that exceptions actually extend from
``BaseException``, which introduces weird semantics all its own. I'm
a big -1 to using exceptions for Forms.
So, given all that, I think we ought to use Joseph's proposal as a
starting point, but integrate a few ideas from what Brant came up
with. In particular, here's how I'd add to/change Joseph's outline:
- Add an explicit ``convert()`` step to the ``Form``. It would be
called automatically, but making the step separate would allow
subclasses to override it if necessary, and would also allow code
just convert data (a need I've seen in a few places).
- Figure out a way to make the common-case view code as simple and
streightforward as possible. I'm not sure where to go here, but I
think we can improve on Joseph's idea a bit.
Al-rightly then... thoughts?
Jacob
gabor
> Also, exceptions are very
> expensive,
while this is true in java or c++, it is not necessarily true in python.
even python itself uses exceptions for example when doing a for-loop
over an iterable (the iterator's next() method raises StopIteration when
it's at the end).
gabor
Brantley Harris
>
> Al-rightly then... thoughts?
>
+1 ValidationErrors with an s, as I've said before: the more errors
returned at once, the better.
My problem with Joseph's proposal is that I believe it to be too tied
to the concept of the Model. Idealy, the Form/Manipulator should not
really have anything to do with a Model, except perhaps in the
finalization, like in .save().
Also it doesn't end up actually leveraging model-validation. You'd
have to define the validation twice, once in the Model, and then once
in Field. But leveraging on the Model validation turns out to be
rather tricky anyway, even in my proposal. Lo:
Let's say I have a dict, named "poll_data" with a data for a new Poll
object. And an dict, named "choice_data", which has info for a new
choice added in the inline.
poll = Poll(**poll_data)
errors = poll.save() # We don't get any errors.
poll.choice_set.create(**choice_data)
errors = choice.save() # But we get some here.
So now we return the form with the errors. But the user decides that
he can't add his clever poll with a choice that is empty, so he
cancels and goes home for the day, expecting that his poll was never
created. But alas, it IS created because we did "poll.save()". We
could rework it to do poll.validate() instead of saving, but since the
Poll then hasn't been created yet, you have to make the Choice without
a poll, and then assign it after. Seems like a transaction.
Bill de hÓra
Some comments:
1: hardwired to HTML
[[[
def process(self, request):
"Perform the manipulation process."
if (not request.POST):
raise self.form
self.request = request
self.data = copy_dict(request.POST)
]]]
I gather this proposal means validation will be highly optimised for web
forms and available only on POST? I understand that is the mainline use
case, but I have a preference something that wasn't baked into HTML 'cos
I'll have to hack around it eventually or use something else.
Can I suggest one of the following. Set up a Manipulator family and
extract the prep work to a method in process() so other cases can be
supported (even if it's just unit testing the validations). Or, leave
the validation framework as specced but tie it right into form handling
so there's room for non-form validations work on models.
2: fast fail
Throwing exceptions means the first error will exit the chain. A lot of
work I do involves gathering as many errors as possible.
3: exception based flow
This will be my inner Java guy no doubt, but using exceptions to manage
mainline execution seems wrong to me. I guess I see validation failure
as not being exceptional behaviour. But as it goes, I think what's there
is about as elegant as you make it with exceptions. Partially related
to 2 because when you walk away from exceptions as a mechanism you are
left with a chaining/pipelining approach, which will support error
collation naturally.
cheers
Bill
Brantley Harris
>
> I gather this proposal means validation will be highly optimised for web
> forms and available only on POST? I understand that is the mainline use
> case, but I have a preference something that wasn't baked into HTML 'cos
> I'll have to hack around it eventually or use something else.
>
and a GETManipulator, or a JSONManipulator. It would be easy to make
as many as you'd like, just change that process function.
> 2: fast fail
>
> Throwing exceptions means the first error will exit the chain. A lot of
> work I do involves gathering as many errors as possible.
>
No, the Form is only thrown after each one of the three steps
(converting data, validating field data, validating model data), if
needed. I agree, gather as many errors as possible.
> 3: exception based flow
>
> This will be my inner Java guy no doubt, but using exceptions to manage
> mainline execution seems wrong to me. I guess I see validation failure
> as not being exceptional behaviour. But as it goes, I think what's there
> is about as elegant as you make it with exceptions. Partially related
> to 2 because when you walk away from exceptions as a mechanism you are
> left with a chaining/pipelining approach, which will support error
> collation naturally.
>
The whole raising a Form thing is just a shocking idea. Like gabor
mentioned, it's actually done often in Python. Exceptions are just a
different and handy control structure to me; I understand that it's a
weird idea, I just don't understand why. Anyhow, ValidationErrors are
exceptions now anway, so this, in general, would actually be lessoning
the ammount of Exceptions being thrown.
DavidA
in the past, I've typically returned two levels of validations
messages: warnings and errors. An error indicates that the attempted
save will fail (i.e. it would either cause a an object to be saved in
an invalid state or it would violate a DB constraint and throw an
exception in the backend). A warning would not result in a failure but
is still worthy of notifying the user, but there are cases where its OK
so its not identified as an error.
An example with a user registration form:
user id:
first name:
last name:
password:
confirm password:
Errors:
- user id cannot be blank
- user id already exists
- password and confirmation do not match
Warnings
- first/last name not set
- password blank
- password shorter than suggested length
And the logic I'll typically have is
- if errors exist then redisplay form with errors and ask user to fix
and resubmit
- if no errors but some warnings, redisplay form with warnings and ask
user to fix _or_confirm_save_with_warnings_
- if no errors and no warnings, just save as usual
The reason I point this out is that I like to centralize the logic
where the validation rules live - I don't want errors checked for in
one place and warnings in another (as others have pointed out, I want
ALL validation messages generated at once and then displayed in the
form).
So I'd suggest a ValidationException that has an errors dict, a
warnings dict and possibly a form (or a very easy way to create a form
from the exception).
-Dave
Jacob Kaplan-Moss
On Aug 24, 2006, at 2:21 PM, Brantley Harris wrote:
> My problem with Joseph's proposal is that I believe it to be too tied
> to the concept of the Model. Idealy, the Form/Manipulator should not
> really have anything to do with a Model, except perhaps in the
> finalization, like in .save().
The way I read it, it doesn't -- a Form can delegate to a model for
validation and saving, but doesn't have to.
> Also it doesn't end up actually leveraging model-validation. You'd
> have to define the validation twice, once in the Model, and then once
> in Field. But leveraging on the Model validation turns out to be
> rather tricky anyway, even in my proposal.
Again, the way I see it would be that Form.validate() would delegate
to Model.validate(), iff that Form represents a Model. For custom
forms, validate() would delegate differently. Or am I missing something?
> The whole raising a Form thing is just a shocking idea.
Which, right there, is what I'm opposed to it. Often there's a reason
why things are usually done one way, and I don't want to break from
the standard way of using exceptions. Having a function raise a Form
object as an exception is just strange (to me, at least), and
strangeness is *not* what I'm looking for in form processing.
On Aug 25, 2006, at 7:04 AM, DavidA wrote:
> One comment on ValidationErrors: When I've done these types of things
> in the past, I've typically returned two levels of validations
> messages: warnings and errors. An error indicates that the attempted
> save will fail (i.e. it would either cause a an object to be saved in
> an invalid state or it would violate a DB constraint and throw an
> exception in the backend). A warning would not result in a failure but
> is still worthy of notifying the user, but there are cases where
> its OK
> so its not identified as an error.
Yes! This is something that I forgot about -- we've wanted to add
validation warnings to Django as long as I can remember.
> So I'd suggest a ValidationException that has an errors dict, a
> warnings dict and possibly a form (or a very easy way to create a form
> from the exception).
This sounds like a reasonable idea to me...
Jacob
Ahmad Alhashemi
> > The whole raising a Form thing is just a shocking idea.
+1
At first, I really digged this raising a Form, but then I realized
that it is just returning a value, but too cleverly...
1. It feels like a goto is happening (execution jumping in an unusal path)
2. It abuses raise as a replacement for a return and a try...except as
a replacement for an if-statement
3. It makes the return value strongly typed (you have to raise a Form object)
4. Form becomes a sub-class of Exception (yuch)
Brantley Harris
> > The whole raising a Form thing is just a shocking idea.
>
> Which, right there, is what I'm opposed to it. Often there's a reason
> why things are usually done one way, and I don't want to break from
> the standard way of using exceptions. Having a function raise a Form
> object as an exception is just strange (to me, at least), and
> strangeness is *not* what I'm looking for in form processing.
Yeah, and that's fine. Personally, I'm weary of any objection that
can cite no other reason but that it is "strange"; many good ideas
have been destroyed that way. But whatever, really I'm not pushing
the form-exception at all. What I am pushing for is clean and simple
api usage for the developer. But what I'm seeing so far is pretty
good. If what you say follows.
However, many of the details here are still missing. Like how exactly
are objects assigned to this Form so that it knows to use the correct
validate(), and how to save() the object. And what about Forms that
are complex and deal with more than one object, not just one that is
inlined? Also, I'm still hazy on how the set_defaults / get_defaults
works exactly.
James Bennett
think it's un-Pythonic or too expensive to use an exception for it,
then I can get behind that.
I like a lot of the stuff in Joseph's proposal, especially the method
of handling read-only fields (that would solve a lot of common
problems people run into currently).
There are two things that I'd like to see hammered out as we do this, though:
1. When, exactly, the "manipulator" (whatever name we end up choosing;
I like Form) fills in default values.
2. A sexier syntax for using manipulators in views. Stripped of the
form-as-exception, both these proposals boil down to almost the same
basic pattern we see now. There's a lot to be said for not breaking
compatibility with existing views, but it's always felt slightly
clunky to me; I'd love to see something better.
Gary Wilson
> On Aug 25, 2006, at 7:04 AM, DavidA wrote:
> > One comment on ValidationErrors: When I've done these types of things
> > in the past, I've typically returned two levels of validations
> > messages: warnings and errors. An error indicates that the attempted
> > save will fail (i.e. it would either cause a an object to be saved in
> > an invalid state or it would violate a DB constraint and throw an
> > exception in the backend). A warning would not result in a failure but
> > is still worthy of notifying the user, but there are cases where
> > its OK
> > so its not identified as an error.
>
> Yes! This is something that I forgot about -- we've wanted to add
> validation warnings to Django as long as I can remember.
>
> > So I'd suggest a ValidationException that has an errors dict, a
> > warnings dict and possibly a form (or a very easy way to create a form
> > from the exception).
I would also love to see a way to throw a validation error at the
form/manipulator level that would short circuit further validation at
the field level. For example, I have a login form and want to attach a
hasCookiesEnabled validator at the manipulator level so that if the
user didn't have cookies enabled, further validation would be skipped
(saving hits to my authentication server and database server) and a
manipulator level error would be raised.
Brantley Harris
>
Thinger = "Manipulator" or "Form" # The thing that holds the fields
Bah, I try and try, but I can't figure out how to seperate the
manipulation process from the Thinger. It really belongs in there.
If you can define it in one spot, all of the different apps (including
the Admin) can use that same process. This means that the Thinger is
an actor on the object, and therefore a "Manipulator". Er, we could
change the name to "Negotiator", or "Handler"
Perhaps The Thinger doesn't have a home in the MVC paradigm-- It is
defining a Form structure (not model structure) and controller code to
"manipulate" a model instance.
Also, I've realized now that a lot of what is defined in the Django
Model should really be placed into the Thinger, as it's not Model
specific (like blank=True), but is rather specific to the Admin /
Thinger. I guess it's in there for simplicity (DRY and all that), but
I wonder if that is correct.