@validate revisited, JSON support, content negotiation
Mike Burrows
No, not another whinge - maybe there's life in @validate yet!
Locally to my project, I have a refactored @validate decorator sitting
in my lib/base.py, with most of the work of what was previously a 100+
line function extracted to methods on BaseController. What's left of
the function lis shown at the end of this message - the extracted
methods are obvious enough if you have seen the original function.
The JSON twist: The new _get_decoded() method adds the ability to
handle request bodies sent in JSON, and the new
_handle_validation_errors() method will render errors in JSON if
that's the format the client wants to accept. Just a couple of lines
in each case.
In my controllers, "show" and "list" actions needed a small change to
render JSON if requested, but the create and update actions needed no
change at all. Model objects all gained a to_dict() method, used by
the controllers to provide input to the json rendering. In the
objects I most care about, the dict representation was tweaked a
little by hand. Very modest effort for a basic JSON API I think.
This new @validate is clearly more extensible than the old one but I
still wonder about the extension interface. For example, does the
request parsing bit (especially the JSON part) belong here, in (say)
an extensible or otherwise format-aware request object, or somewhere
else - a new validation or form object, say?
While we're here, I'm thinking of a Routes enhancement (in the form of
a personal extension if no-one else likes the idea) that allows a
{.format} syntax - already proposed to the URI Template people - as a
parameter style for optional format extensions. The removal of large
numbers of "formatted routes" might even yield a small performance
benefit for some people, but even if it doesn't, I for one will be
pleased to help see some duplication disappear. Meanwhile I have only
header-based conneg, which gets a bit tedious at times!
Finally, I would like a more sophisticated conneg function than my
current accepts_json(request), but before I start to write one, surely
such things exist already?
Regards,
Mike
m...@asplake.co.uk
http://positiveincline.com
http://twitter.com/asplake
def validate(schema=None, validators=None, form=None,
variable_decode=False,
dict_char='.', list_char='-', post_only=True, state=None,
on_get=False, **htmlfill_kwargs):
"""The Pylons @validate decorator refactored, with most of the
work done
by controller methods defined on BaseController. Enhanced to
accept JSON.
"""
if state is None:
state = PylonsFormEncodeState
def wrapper(func, self, *args, **kwargs):
"""Decorator Wrapper function"""
request = self._py_object.request
# Skip the validation if on_get is False and its a GET
if not on_get and request.environ['REQUEST_METHOD'] == 'GET':
return func(self, *args, **kwargs)
decoded = self._get_decoded(
variable_decode, dict_char, list_char,
post_only)
self.form_result, self.form_errors = self._convert(
decoded, schema, validators, state,
variable_decode, dict_char, list_char)
if self.form_errors:
return self._handle_validation_errors(func, decoded,
self.form_errors,
form,
htmlfill_kwargs,
args, kwargs)
else:
return func(self, *args, **kwargs)
return decorator(wrapper)
Mike Burrows
I've done quick write-up on the now-completed {.format} thing here
(with a clarifying comment or two) :
* http://positiveincline.com/?p=617
This gist has the refactored and json-capable @validate plus other
small goodies referred to in previous discussions:
* http://gist.github.com/301613
Usage:
A typical json-capable GET action ends with something like this:
if accepts_json():
return render_json(thing.to_dict()) # to_dict is
implemented in my model
else:
return render(template)
[When rendering forms, render() is replaced by fill_render() - see
previous discussions on repeating groups. Maybe I should consolidate
them into one.]
Create and update actions end with
return redirect_to(formatted_url(...))
which remembers any format extension on the request. I'm tempted to
import formatted_url as url but I haven't yet.
As I'm already relying on the latest SubMapper helpers (I never use
resource() and hardly use connect()), the {.format} extensions came by
default and my confg/routing.py needed very little change. To be on
the safe side I added requirements=dict(format='json') at SubMapper/
collection level (a handful of places in my case). To minimise this,
maybe there needs to be a single place where a default pattern (or
even requirements in general?) can be registered. Thinking about it,
an outer SubMapper might do the trick...
It's not quite as generic as it might be, since we have helpers like
accepts_json() rather than accepts('json'), but it's easy to use and
good enough for my purposes. To generalise it, mappings between
format names and MIME types would be needed.
Clients can request json by adding either a .json path extension or an
accept:application/json header (the extension takes precedence).
Successful PUT/POST actions (client payload type having been
determined by extension or content-type header) return a redirect to
the created or updated resource, carrying forward the extension if
there was one. Validation failures are returned in a 200 OK response
with the expected content-type; if json, the body contains '{"errors":
form_errors}'.
So... anyone else interested?
Regards,
Mike
> m...@asplake.co.ukhttp://positiveincline.comhttp://twitter.com/asplake
Mike Orr
> So... anyone else interested?
I'm not sure, it's kind of complex to make a quick decision on. Could
you make a link to your proposal in the @validate reorganization
ticket? Then we can consider it for Pylons 1.1. My inclination at
this point is to get rid of @validate as a decorator, to be replaced
by something in the action.
http://pylonshq.com/project/pylonshq/ticket/405
--
Mike Orr <slugg...@gmail.com>
Mike Burrows
Sure - as soon as I receive my registration email - it has been a few
minutes now...
Personally, I would be happy to see a decorator-free Pylons but I say
that without knowing what our action methods will look like without
them!
Regards,
Mike
> Mike Orr <sluggos...@gmail.com>
Mike Orr
>
> Personally, I would be happy to see a decorator-free Pylons but I say
> that without knowing what our action methods will look like without
> them!
You can do it now; the trouble is you either have to work up from a
minimal try/except or work down from the @validate code. The problem
with the @validate code is it's hard to separate the universal pattern
from the specific argument slinging the decorator does.
The pattern is something like this (untested):
'''
try:
val = MyValidator()
data = val.to_python(request.params, None)
except formencode.Invalid, e:
html = self.form_method()
html = htmlfill.render(html, request.params,
e.unpack_errors(), force_defaults=False)
return html
# Success, perform action using ``data``.
'''
The render args are the source HTML, input values, and errors.
``e.unpack_errors()`` converts the error dict from a nested structure
(if applicable) to the flat HTML format. ``force_defaults=False``
prevents htmlfill from unsetting radio buttons and selects.
I would read the source code for render (formencode/htmlfill.py) and
Invalid (formencode/api.py) to get a feel for what exactly it's doing
and the possible arguments.
--
Mike Orr <slugg...@gmail.com>
Mike Burrows
Yes that's a much better pattern. You could even move the "except" to
the end and push some validation to the model.
One thing my version shows however is that it's very easy to cover
more than just the html case. So direct references to htmlfill should
be avoided, and either request.params needs somehow to work for json
(and potentially other formats) or the validator gets sent the full
request object. And it's going to get repetitive, so even if it's
using a new validator object under the covers I would be tempted to
make it look something like
try:
data = self.parse(request)
# maybe do something with data here, like save to the model
except formencode.Invalid as e:
return self.render_invalid(e)
# do more with data here
Push the exception handling up to the dispatcher and you get
data = self.parse(request)
# do stuff with data
Rename "data" to something more meaningful ("form_data" say) and make
it a memoized property and you're left with
# do stuff with self.form_data
Fun though this is (and I will try tomorrow to implement at least some
of the above patterns before commenting on the ticket) I'd like to
express a strong hope that Pylons 1.1 will be more format-aware. Some
content negotiation will be needed; mine's incomplete but it's a
start, and I'd surprised if there weren't better solutions out there
already. The {.format} enhancement for Routes isn't strictly required
but it does work and it does give tidier config, so I hope that it (or
something very much like it) gets adopted too.
Regards,
Mike
> Mike Orr <sluggos...@gmail.com>
Mike Burrows
Oops - it's getting late here but I forgot the form parameter required
by the error handler and this makes a nonsense of some of what
followed (ok I got carried away):
try:
data = self.parse(request)
# maybe do something with data here, like save to the model
except formencode.Invalid as e:
return self.render_invalid(e, form='edit')
# do more with data here
Mike
Mike Burrows
Done, new gist http://gist.github.com/302617, ticket updated.
@validate now uses the same _parse() and _render_invalid() methods
used by the example below.
def update(self):
try:
self._parse(request, schema=MyForm())
# Here self.form_result is populated as normal
# as is self.params, the decoded but unconverted form
params or
# json request. You could do more stuff here that raises
# formencode.Invalid, validated model updates for example.
except Invalid as e:
return self._render_invalid(e, form='edit')
# Do more stuff with self.form_result, e.g.
if accepts_json():
return
render_json(json_serialisable_thing_made_from_form_result)
else:
return render('template')
Regards,
Mike
Jonathan Vanasco
controller like my stab ( http://groups.google.com/group/pylons-discuss/browse_thread/thread/4269ca745e31793
) ?
because that's the only thing i care about.
example:
from OpenSocialNetwork.lib.decorators import osn_validate ,
osn_form_error
from OpenSocialNetwork.lib.errors import SubmissionError
@osn_validate( schema=Forms_Checkout.Shipping_1() ,
form='_checkout_shipping__print' , post_only=True )
def _checkout_shipping__submit(self):
try:
raise SubmissionError('Whatever')
except SubmissionError , e :
h.formerrors_set( message='!!!' )
return osn_form_reprint( self , '_checkout_shipping__print' )
except :
raise
Mike Burrows
I'm 100% sure that this answers your question, but (following my
outline above)
self._render_invalid(e, form='_checkout_shipping__print')
will call the _checkout_shipping__print action and apply the errors to
it, just as @validate would. In the example, the errors are obtained
from the Invalid exception e, but they can be supplied explicitly and
they will default to self.form_errors if neither input is truthy. If
the required format is JSON, errors are rendered directly, i.e.
without reference to the form argument.
While we're here, I should mention that I noticed this week that some
legacy cruft has been removed from @validate in the 1.0b version (work
done by Philip Jenvey I think). I have backported the changes
(deletions mostly) to my app but I haven't got round to posting a new
gist yet. If anyone is using the old one, please shout.
On Feb 24, 5:25 am, Jonathan Vanasco <jonat...@findmeon.com> wrote:
> does this new approach allow for form errors to be re-triggered in the
> controller like my stab (http://groups.google.com/group/pylons-discuss/browse_thread/thread/42...
Jonathan Vanasco
controller.
i'd like to make a suggestion to your then.
i have an arg to validate called gatekeeper , which is enabled as True
by default ( along with post_only )
In conjunction with one another, gatekeeper just makes sure that if
you GET a form that is post_only ( or: not on_get ) , you
automatically raise an error.
in the original pylons distro, it was possible to GET a post_only form
and have things not work out the way you would want them to.
personally i thought it opened the door to security issues, others
disagreed.
``gatekeeper``
Default True. Boolean to raise an error on form submission if
not complete.
the code in mine/patch to distribution is :
if request.environ['REQUEST_METHOD'] == 'GET' and not on_get:
if gatekeeper:
ControllerInstance.osn_form.is_error= True
raise ValidationStop()
I'm not sure how it would work on yours.
Mike Burrows
@validate becomes self._parse() and self._render_invalid(), both of
these defined on BaseController. The on_get bit is near the top of
_parse() - you could easily patch/override that. I have just noticed
that I didn't remove that fragment from what's left of validate() -
you would need to do that too if you still wanted to use the
decorator.
Regards,
Mike