Validation-aware models: First stab
Adrian Holovaty
into the Django groove. Earlier today I committed the first stab at
validation-aware models on the magic-removal branch:
http://code.djangoproject.com/changeset/2518
Here's how it works:
* Model instances have a validate() method. It returns a dictionary of
field name -> list of error messages. If there are no errors, it
returns an empty dictionary.
* Behind the scenes, validate() actually does a two-pass validation
over all the model instance's fields:
* First, it runs the field's to_python() method, which is
field-type-specific and converts the data to the appropriate Python
type. For example, if a date field is set to '2005-01-03', to_python()
converts it to a datetime object.
* Second, for all fields that didn't have validation errors in
to_python(), it executes the field's validate_full() method. That
method can safely assume the data is in the correct data type.
The separation of "type-specific" validation and "logical" validation
makes things a lot cleaner, both in concept and implementation. It's
sort of like the CriticalValidationError framework in
django/core/formfields.py in trunk.
* Model instances' save() method will call validate() behind the
scenes, and the object won't save if there are any validation errors.
I haven't coded this part yet, but it's a simple change.
* I've only implemented to_python() and validate_full() for a handful
of field types in django/db/models/fields/__init__.py. The rest still
need to be implemented.
* Every one of the to_python() and validate_full() functions that
*are* implemented (to my knowledge) have unit tests in
tests/modeltests/validation/models.py. As we implement to_python() and
validate_full() for other field types, they should get unit tests
immediately.
EXAMPLE CODE:
class Person(models.Model):
name = models.CharField(maxlength=50)
birthday = models.DateField()
favorite_number = models.IntegerField()
>>> p = Person(name='Joe', birthday='never', favorite_number='foo')
>>> p.validate()
{'birthday': 'Enter a valid date in YYYY-MM-DD format.',
'favorite_number': 'This value must be an integer.'}
>>> p = Person(name='Joe', birthday='2005-1-3', favorite_number='34')
>>> p.validate()
{}
>>> p.birthday
datetime.date(2005, 1, 3)
>>> p.favorite_number # note this is an integer now, no longer a string
34
Thoughts, criticisms? The only part I'm sketchy on is the side effect
of converting the values to their Python data types. The
cleanliness/consistency is quite nice, but it's slightly not something
a developer might expect. However, we can solve the latter through
clear documentation.
Adrian
--
Adrian Holovaty
holovaty.com | djangoproject.com
Gábor Farkas
>
> EXAMPLE CODE:
>
> class Person(models.Model):
> name = models.CharField(maxlength=50)
> birthday = models.DateField()
> favorite_number = models.IntegerField()
>
>>>> p = Person(name='Joe', birthday='never', favorite_number='foo')
>>>> p.validate()
> {'birthday': 'Enter a valid date in YYYY-MM-DD format.',
> 'favorite_number': 'This value must be an integer.'}
>>>> p = Person(name='Joe', birthday='2005-1-3', favorite_number='34')
>>>> p.validate()
> {}
>>>> p.birthday
> datetime.date(2005, 1, 3)
>>>> p.favorite_number # note this is an integer now, no longer a string
> 34
>
>
> Thoughts, criticisms? The only part I'm sketchy on is the side effect
> of converting the values to their Python data types. The
> cleanliness/consistency is quite nice, but it's slightly not something
> a developer might expect. However, we can solve the latter through
> clear documentation.
>>> p1 = Person(name='Joe', birthday='2005-1-3', favorite_number='foo')
>>> p2 = Person(name='Jack', birthday = datetime.date(2005,1,3),
favorite_number='foo')
>>> p1.birthday == p2.birthday
False
i don't know. i understand that this would make life easier, but on the
other hand, it does not feel good for me :) some points:
- explicit is better than implicit
- we're removing the magic there or adding it back? :-)
- does this not tie too much the web-layer to the db-layer?
basically.. what does the developer win with the auto-conversion?
perharps some use-case scenarios would help to explain it to us...
summary:
it's a very cool change. i'm just not sure about the auto-conversion part.
gabor
Amit Upadhyay
Does it mean AddManipulator/UpdateManipulator are going away?
This way of doing things is inconsitant with handling custom forms.
If it is, a good step for django ORM, a bad step for django web framework.
Let me elaborate, this is my typical view:
def confirm_email(request):
if not request.user.is_anonymous(): return HttpResponseRedirect('../home/')
manipulator = EmailConfirmationManipulator(request)
if request.POST:
# If data was POSTed
new_data = request.POST.copy()
# Check for errors.
errors = manipulator.get_validation_errors (new_data)
if not errors:
# No errors. This means we can save the data!
manipulator.do_html2python(new_data)
manipulator.save(new_data)
return HttpResponseRedirect("../home/")
else:
errors = new_data = {}
form = formfields.FormWrapper(manipulator, new_data, errors)
return render_to_response('blogger/confirm_email',
{'form': form},
context_instance=DjangoContext(request))
Line in red used to be the only line which would depend on if I am using a django AddManipulator or my custom form. This perticular view initially was using AddManipulator with an extra validator, then I realised I needed a few more fields in the form, and I had to write my custom form, I derived it from the AddManipulator, added a few fields, overwrote .save() and nothing else changed. Virtues of object oriented programming.
Please let me know if I am missing something.
--
A Sad Djangonaut
Amit Upadhyay
Blog: http://www.rootshell.be/~upadhyay
+91-9867-359-701
Adrian Holovaty
> Does it mean AddManipulator/UpdateManipulator are going
> away?
Yes, the automatic manipulators are going away, but there will of
course still be away to generate forms/manipulators automatically from
a model.
Linicks
Validation aware models have seen allot of attention so far with a
number of proposals out there. This proposal, like the others, has
it's strengths and weaknesses. This method of validating models is a
good start, and will surely evolve or be replaced over time. I believe
that we need to come to a consensus on this proposal and move forward
with it as soon as possible.
--Nick
Christopher Lenz
[snip]
> Thoughts, criticisms? The only part I'm sketchy on is the side effect
> of converting the values to their Python data types. The
> cleanliness/consistency is quite nice, but it's slightly not something
> a developer might expect. However, we can solve the latter through
> clear documentation.
IMHO automatic conversion from strings to python types shouldn't be
in the model layer. Model fields should expect to get a compatible
type (at least directly castable to the target type, such as int
(floatnum) or str(intnum)). I could even imagine a DateTimeField
accepting a timestamp or time tuple... but parsing out the date from
a string seems like a step too far. The form/manipulator should be
doing that kind of conversion.
Also, the names validate() and validate_full() had me confused for a
bit until I read the code. IIUC, validate() is the internal interface
for subclasses, while validate_full() is the external interface
invoked by the Model class. But that's rather non-obvious from the
names ;-)
Cheers,
Chris
--
Christopher Lenz
cmlenz at gmx.de
http://www.cmlenz.net/
limodou
-1 I agree with gabor. Many orm models don't process validation, and
these things should be implemented by manipulator. Why do so much
things in model?
--
I like python!
My Blog: http://www.donews.net/limodou
NewEdit Maillist: http://groups.google.com/group/NewEdit
James Bennett
> -1 I agree with gabor. Many orm models don't process validation, and
> these things should be implemented by manipulator. Why do so much
> things in model?
Among other things, the model is the place where the types of data are
specified for a particular class of objects; given that, why have a
separate layer for the purpose of comparing incoming data to ensure it
conforms to those? The model is the thing that's supposed to "know"
this, so why not let it?
--
"May the forces of evil become confused on the way to your house."
-- George Carlin
Malcolm Tredinnick
[...]
>
> -1 I agree with gabor. Many orm models don't process validation, and
> these things should be implemented by manipulator. Why do so much
> things in model?
This isn't really an argument, though. Other systems working differently
is not a reason why Django should not work in the way Adrian described
("other systems are not blue like this, they are red. So blue is no
good.") Why do you see it as bad that it should work this way? What are
the arguments against doing this in the model (which is a representation
of the data structure and so not a completely illogical place to work
with the data)?
Malcolm
limodou
>
> On 3/13/06, limodou <lim...@gmail.com> wrote:
> > -1 I agree with gabor. Many orm models don't process validation, and
> > these things should be implemented by manipulator. Why do so much
> > things in model?
>
> Among other things, the model is the place where the types of data are
> specified for a particular class of objects; given that, why have a
> separate layer for the purpose of comparing incoming data to ensure it
> conforms to those? The model is the thing that's supposed to "know"
> this, so why not let it?
>
In most cases we do not do anything alone use model, but always there
is a ui layer, such as gui or web. They have to do any such thing.
And the model can only handle field themself. In some cases, we also
need more time to deal with increased logic, particularly the
relationship between the model, such things as users and conditions,
and These things can not be accomplished only by models. Therefore, we
still need to do a validation independent of the model validation. Had
that been the case, why must it do in the model. So I think
manipulator is enought and clearly.
limodou
because I think manipulator can be standalone separated from the model
and maybe someday the orm can be designed as an interface, so that we
can use other orm module to replace django orm.
If you don't want, and the model becames more and more rich, so these
things maybe not be separated any more.
limodou
I like django, and I'm learning hard on it. But someone don't like
django, and they don't like django orm, they like sqlobject,
sqlalchemy. So if the db api is really loose and replable, we can drag
them back to django.
I don't know whether there is a such plan in django team.
James Bennett
> In most cases we do not do anything alone use model, but always there
> is a ui layer, such as gui or web. They have to do any such thing.
There should be the ability to generate an HTML form for
creating/changing an instance of a model, yes, but that code doesn't
necessarily have to be what's doing the validation.
> And the model can only handle field themself. In some cases, we also
> need more time to deal with increased logic, particularly the
> relationship between the model, such things as users and conditions,
> and These things can not be accomplished only by models.
Sure they can. Imagine explaining Django to someone who's never seen
an MVC model before; if you told them "the model is where you provide
all the information about a class of objects, and describe how it
works", they'd be awfully confused if you then said, "oh, except for
this and this and this, which have to be specified somewhere else."
Better to have all the information about a class of objects, including
whatever information is needed to validate it, all in one place.
Jacob Kaplan-Moss
> Sure they can. Imagine explaining Django to someone who's never seen
> an MVC model before; if you told them "the model is where you provide
> all the information about a class of objects, and describe how it
> works", they'd be awfully confused if you then said, "oh, except for
> this and this and this, which have to be specified somewhere else."
>
> Better to have all the information about a class of objects, including
> whatever information is needed to validate it, all in one place.
That's a great explanation, James.
To explore it a bit further: I'm often asked why Django doesn't use
SQLObject or some other Pythonic ORM. My standard answer is that
Django's ORM does a lot more than just describe how to map relational
tables to Python objects: it explains how to construct an interface
for editing the object, it describes validation information about an
object's fields that the database often can't encode, and so on.
Given that, I think validation is something that very much SHOULD be
defined and happen on the model. Yes, this runs counter to MVC, but
we've already established that Django isn't exactly MVC anyway. As
I see it:
- Model: describe the data
- View: describe what the user sees
- Template: describe how the user sees it
Remember, kids, we're the MTV generation.
Jacob
limodou
> > In most cases we do not do anything alone use model, but always there
> > is a ui layer, such as gui or web. They have to do any such thing.
>
> There should be the ability to generate an HTML form for
> creating/changing an instance of a model, yes, but that code doesn't
> necessarily have to be what's doing the validation.
Maybe or maybe not. What django has implemented is exactly as what I
said, I think the ways is good. And I remember there was a discussion
about js invalidation, why we need this. Because we want to make
mistakes can be found early, but not until the data will be saved.
Validation can be implemented at different layer, but why the last layer?
>
> > And the model can only handle field themself. In some cases, we also
> > need more time to deal with increased logic, particularly the
> > relationship between the model, such things as users and conditions,
> > and These things can not be accomplished only by models.
>
> Sure they can. Imagine explaining Django to someone who's never seen
> an MVC model before; if you told them "the model is where you provide
> all the information about a class of objects, and describe how it
> works", they'd be awfully confused if you then said, "oh, except for
> this and this and this, which have to be specified somewhere else."
I think this is not a reason. Can we make this design is to promote a
model to others can do many things? This is a good design?
>
> Better to have all the information about a class of objects, including
> whatever information is needed to validate it, all in one place.
>
As what I said, Validation can be implemented at different layer, but
why the last layer?
limodou
So this is a different design, all decisions in one place, or tiered
to decide. Perhaps I am more accustomed tiered achieved. Design more
concentrated, the more impossible to separate. If django team decided
not to use other orm module to replace current one, then to realize.
But I said the logic of validation and custom validation issues
remain, there is still an independent check processing, and this can
not be changed.
James Bennett
> Maybe or maybe not. What django has implemented is exactly as what I
> said, I think the ways is good. And I remember there was a discussion
> about js invalidation, why we need this. Because we want to make
> mistakes can be found early, but not until the data will be saved.
As I see it, the question here is which of two options we want to take:
1. Validation is performed when you try to save an object; if
validation fails, you send back some error messages and try again with
corrected data.
2. One or more extra layers are placed between data entry and saving
the object; different layers do different types of validation, and any
one of them can kick you back out to the data-entry layer.
Option 1 is what we seem to be heading toward with validation-aware
models, and I think it's good for a couple of reasons:
* It's conceptually simpler; instead of worrying about which
validation layer you got down to before an error occurred, you only
have to worry about two states: success and failure.
* It's more logical; I still don't see any good reason why information
about how to validate an instance of a model shouldn't be part of the
model.
* It's better from an end-user standpoint; to see why, imagine that
instead we have a system with multiple layers which do validation. In
this case, an end user creating or editing an object may make a
mistake in entering the data, and be presented with a set of errors.
He corrects them and tries again to submit the data, but this time it
comes back with a different set of errors that have been present all
along, but were never reported because the data never made it to the
layer that would have caught them before. This poor user would become,
understandably, frustrated that he wasn't told about these errors the
first time he tried to submit the data. When validation is all in the
same place, this will never be a problem because all errors will be
caught and reported each time the data is submitted.
limodou
Reasons sound very reasonable. But I think current implement in django
is enough. Maybe model with validation is better but unnecessary. We
have many ways to validate data before it saved in db.
Max Battcher
> Reasons sound very reasonable. But I think current implement in django
> is enough. Maybe model with validation is better but unnecessary. We
> have many ways to validate data before it saved in db.
Any Security Foil-Hat will tell: Validate early and validate often. It
makes no sense for a good ORM to allow someone to save data that breaks
database constraints, without jumping through a few hoops
(validate=False) if you expect a secure ORM.
It makes perfect sense that at least basic data validation should be
implemented in models, and it *is* a necessity to anyone security-minded.
--
--Max Battcher--
http://www.worldmaker.net/
Max Battcher
The current proposal leaves Manipulators (custom forms rely on them,
they are not going away), and validation at model level is simply yet
another defense in a long line of defenses.
Gábor Farkas
i'm fine with this approach, i have no problems with the model
validating itself.
my problem is with the model auto-converting data to a suitable format.
i think it should be an explicit thing, not an implicit one.
it's ok if the model checks if the DateField is a datetime.date (or
whatever) object. but i don't like it that it will not fail when there's
a string, and it will convert it to a datetime.date object.
to specify it better:
- in my opinion, validation should not convert date. it should validate.
- data conversion should be a separate task.
please, could you describe some situations, where it's so good to be
able to play the everything-is-a-string game?
i see, that with this auto-convert approach, you will be able to do
something like:
def myview(request):
try:
m = MyModel(**request.POSTDATA)
m.save()
except SomethingFailedError:
//handle it
or something like that.
but for me it feels more pythonic to do it like:
def myview(request):
try:
m = MyModel( MyModel.hmtml2python(**request.POSTDATA))
m.save()
except SomethingFailedError:
//handle it
or have a new constructor, that takes a dictionary of strings:
def myview(request):
try:
m = MyModel( request.POSTDATA)
m.save()
except SomethingFailedError:
//handle it
anyway,
1. if there are other use-cases where it makes sense to have everything
in strings, please tell me.
2. it's a matter of how much pythonic you want your objects to be.
python does not auto-convert an int to a string when needed, so if
models will do so, they will be slightly less pythonic. i'm not saying
you should not do it, i'm only saying it will be less pythonic.
3. even if this auto-converting approach will become the way to do it in
Django, i will still use Django for all my web-app needs :-)
gabor
Max Battcher
>> Thoughts, criticisms? The only part I'm sketchy on is the side effect
>> of converting the values to their Python data types. The
>> cleanliness/consistency is quite nice, but it's slightly not something
>> a developer might expect. However, we can solve the latter through
>> clear documentation.
>
> >>> p1 = Person(name='Joe', birthday='2005-1-3', favorite_number='foo')
> >>> p2 = Person(name='Jack', birthday = datetime.date(2005,1,3),
> favorite_number='foo')
> >>> p1.birthday == p2.birthday
> False
>
> i don't know. i understand that this would make life easier, but on the
> other hand, it does not feel good for me :) some points:
>
> - explicit is better than implicit
> - we're removing the magic there or adding it back? :-)
> - does this not tie too much the web-layer to the db-layer?
>
> basically.. what does the developer win with the auto-conversion?
> perharps some use-case scenarios would help to explain it to us...
>
> summary:
> it's a very cool change. i'm just not sure about the auto-conversion part.
I'm going to agree with this. At the very least I think a developer
should *always* expect a field on field __get__ to return the Pythonic
type. Under the proposal this would probably mean an implicit
``to_python`` call and half-verification on field __get__, which sounds
un-Pythonic.
Here's a suggestion: abstract the string->python into a field function.
Consistently require Python types, but allow the string unmarshalling
through something like:
>>> p1.birthday.from_string('2005-1-3')
This is clear and explicit, albeit potentially wordy. It would not
allow complete stringified Constructors, but you could easily enough
create a "shortcut" tool. I think this may actually be more suitable
for the Manager class:
>>> p1 = Person(name='Jack', birthday=datetime.date(2005,1,3),
favorite_number='foo')
>>> p2 = Person.objects.html_unmarshal(name='Joe', birthday='2005-1-3',
favorite_number='foo') # convert strings to Python types
>>> p1.birthday == p2.birthday
True
This then makes it possible to change things like locale-based
unmarshalling (or other unmarshal situations, such as importing from
object-db systems or what have you) by overriding the system-wide
default Manager.
Ivan Sagalaev
>or have a new constructor, that takes a dictionary of strings:
>
>def myview(request):
> try:
> m = MyModel( request.POSTDATA)
> m.save()
> except SomethingFailedError:
> //handle it
>
>
I also proposed something like this in first or second reincarnation of
validation-aware models thread. But I think that it should be some named
method like:
m = MyModel.from_post(request.POST)
Amit Upadhyay
* It's conceptually simpler; instead of worrying about which
validation layer you got down to before an error occurred, you only
have to worry about two states: success and failure.
* It's more logical; I still don't see any good reason why information
about how to validate an instance of a model shouldn't be part of the
model.
This is based on a simplistic assumption that everytime we are talking about validation in django context we are talking about model based validation. views are handling forms, and we should always add form in front of validation whenever we are talking about validation. Model can have a simplistic type checking like validation, which will raise ValidationError while saving, but trying to fit all/any form validation into model validation is just twisted. Django is a web framework. In web apps, the most common thing to validate is form. We can not ignore these two things.
Bottom line, can you gaurantee all validations can go on model? Will you make model dependent on request, and any other potentially django external check a web developer might have to use in their application? We can not do validation in models, we can try, we can do a little bit, but there is no limit to what application specific requirement can ask. What is the point of making people learn about the model based validation framework when we are going to tell them that not all validation can be done, and you will have to invent your own validation framework?
Joseph Kocherhans
>
> Am 13.03.2006 um 08:00 schrieb Adrian Holovaty:
> [snip]
> > Thoughts, criticisms? The only part I'm sketchy on is the side effect
> > of converting the values to their Python data types. The
> > cleanliness/consistency is quite nice, but it's slightly not something
> > a developer might expect. However, we can solve the latter through
> > clear documentation.
>
> IMHO automatic conversion from strings to python types shouldn't be
> in the model layer. Model fields should expect to get a compatible
> type (at least directly castable to the target type, such as int
> (floatnum) or str(intnum)). I could even imagine a DateTimeField
> accepting a timestamp or time tuple... but parsing out the date from
> a string seems like a step too far. The form/manipulator should be
> doing that kind of conversion.
My gut reaction is that type conversion is controller level behavior,
it has nothing to do with "business logic" and everything to do with
the fact that we're getting this data from the web. It belongs in a
web controller layer, FormField seems like the obvious candidate.
However, when I try to dream up a use case where someone would
*really* set a date field to a sting, or an int field to a sting, and
expect a string back... well... Can anyone come up with an actual
practical use case? I suppose this is more about expected behavior
than actual usage as a feature though.
Joseph
James Bennett
> This is based on a simplistic assumption that everytime we are talking about
> validation in django context we are talking about model based validation.
No, it's based on an assumption that the facilities for doing custom
validation that currently exist in manipulators could be relocated.
> views are handling forms, and we should always add form in front of
> validation whenever we are talking about validation. Model can have a
> simplistic type checking like validation, which will raise ValidationError
> while saving, but trying to fit all/any form validation into model
> validation is just twisted. Django is a web framework. In web apps, the most
> common thing to validate is form. We can not ignore these two things.
Setting aside your premise about the most common thing in web apps
(which is actually retrieving data from a DB and displaying it), I
don't see that your conclusion follows; could you please demonstrate
why it would be impossible to allow custom validation code to be
placed in a model?
> Bottom line, can you gaurantee all validations can go on model? Will you
> make model dependent on request, and any other potentially django external
> check a web developer might have to use in their application?
Why would validation in a model be dependent on a request? If
anything, Adrian's proposal makes Django *more* flexible because it
removes the hard-wired assumption in the manipulator system that the
data is coming from an HTML form; under Adrian's proposal, the model
doesn't care where the data's coming from -- you just build it up and
try to call save(), and if there are errors it tells you about them.
> We can not do
> validation in models, we can try, we can do a little bit, but there is no
> limit to what application specific requirement can ask. What is the point of
> making people learn about the model based validation framework when we are
> going to tell them that not all validation can be done, and you will have to
> invent your own validation framework?
Again, please demonstrate to me why you feel it's impossible for
custom validation to be done in models. And if, as you feel, it's
impossible for the model layer to provide adequate flexibility for
validation, then I don't see how any other layer of the framework
could do so in its place.
Russell Keith-Magee
Thoughts, criticisms? The only part I'm sketchy on is the side effect
of converting the values to their Python data types. The
cleanliness/consistency is quite nice, but it's slightly not something
a developer might expect. However, we can solve the latter through
clear documentation.
I'm late to this discussion, but I have to say I'm +1 as implemented.
My only suggestion relates to validation conditions that aren't bound to a single field - e.g., field1 + field 2 > 10, exactly one of these three date fields must be in the future. How about adding a model-wide validation step (using a list of model wide validators specified on the Meta class)? I imagine the model wide validators would be checked after all the individual field validators have been applied. Invalid model-level validators would return in the validation dictionary under the key 'None', with a list of validation error strings. Comments?
Addressing the arguments against::
1) Validation shouldn't be in the model
If validation isn't in the model, then the model is little more than a light data container; if all you have is a light data container, then the onus is on the user to ensure that validation conditions are checked and rechecked whenever they are used (which smacks of code duplication to me). Putting validation right in the model means that it is never missed.
2) Instantiation shouldn't be from string
Adding string instantiation just seems straight up powerful. Push data in in any meaningful format, and it will get converted (if possible) to the internally preferred representation. Some have called this 'magic' - to me, it's just being liberal in what you accept, but conservative in what you produce.
There have been some interesting proposals for separating 'instantation from data' from 'instantiation from string'; While I understand the sentiment, I'm just not sure they're worth the effort. The only use case I can see here is if the way a string is interpreted will vary depending on the data source, but I don't see this as very likely (or even advisble).
Russ Magee %-)
Christopher Lenz
> Addressing the arguments against::
> 1) Validation shouldn't be in the model
>
> If validation isn't in the model, then the model is little more
> than a light data container; if all you have is a light data
> container, then the onus is on the user to ensure that validation
> conditions are checked and rechecked whenever they are used (which
> smacks of code duplication to me). Putting validation right in the
> model means that it is never missed.
I definitely agree that the model layer should do validation, as long
as that doesn't mean it's the *only* layer that does validation
(that's not how I understood the proposal, but some some of the
comments seemed to imply that). I may have forms that don't map 1:1
to models; for those, parts of the validation must be done by the
formwrapper/manipulator, and other parts by the model classes.
> 2) Instantiation shouldn't be from string
>
> Adding string instantiation just seems straight up powerful. Push
> data in in any meaningful format, and it will get converted (if
> possible) to the internally preferred representation. Some have
> called this 'magic' - to me, it's just being liberal in what you
> accept, but conservative in what you produce.
That phrase is about network protocol implementations, I don't think
it applies equally to programming interfaces ;-)
There's a thin line here between convenient auto-conversion (e.g.
strings to numbers) and overloading the model class with logic that
simply doesn't belong there. In your web layer code, you have
something akin to a manipulator sitting between your model classes
and the view. The manipulator would be responsible for unmarshalling
POST parameters into a data structure that can be handled by the
model, including parsing stuff like dates and exploding "dotted"
parameters into nested dicts or whatever.
> There have been some interesting proposals for separating
> 'instantation from data' from 'instantiation from string'; While I
> understand the sentiment, I'm just not sure they're worth the
> effort. The only use case I can see here is if the way a string is
> interpreted will vary depending on the data source, but I don't see
> this as very likely (or even advisble).
Example: the string representation of a date in may depend on the
locale of the remote user, which the model knows nothing about. The
conversion from string to datetime will need to happen in a layer
that is aware of the HTTP request context. (This is a somewhat
contrived example due to the abysmal state of international datetime
formatting/parsing in the Python standard library, but I'm sure there
are other examples).
Adrian Holovaty
> I definitely agree that the model layer should do validation, as long
> as that doesn't mean it's the *only* layer that does validation
> (that's not how I understood the proposal, but some some of the
> comments seemed to imply that). I may have forms that don't map 1:1
> to models; for those, parts of the validation must be done by the
> formwrapper/manipulator, and other parts by the model classes.
The model layer certainly isn't the *only* layer that does validation.
Of course, Django will continue to support custom forms and validation
logic on a level that's decoupled from models.
> There's a thin line here between convenient auto-conversion (e.g.
> strings to numbers) and overloading the model class with logic that
> simply doesn't belong there. In your web layer code, you have
> something akin to a manipulator sitting between your model classes
> and the view. The manipulator would be responsible for unmarshalling
> POST parameters into a data structure that can be handled by the
> model, including parsing stuff like dates and exploding "dotted"
> parameters into nested dicts or whatever.
Right -- that's exactly what will happen.
> Example: the string representation of a date in may depend on the
> locale of the remote user, which the model knows nothing about. The
> conversion from string to datetime will need to happen in a layer
> that is aware of the HTTP request context. (This is a somewhat
> contrived example due to the abysmal state of international datetime
> formatting/parsing in the Python standard library, but I'm sure there
> are other examples).
In this example, the view code would perform that logic -- not the
model layer. The model layer is only responsible for *critical*
validation logic that applies to *every instance of the model, ever*
-- not special-cases. For special cases, you'll use a custom
form/manipulator, just like before.
gabor
yes, i agree.
the only problem for me is:
is the string-to-date auto-conversion a *critical* validation-part?
gabor
Christopher Lenz
Thanks for the clarification... matches my understanding of the
proposal, I just had the feeling there was some confusion about these
points in this thread (e.g. Model.html2python, which is what
manipulators are for).