Colander schema for dict with arbitrary key names

[Jesaja Everling]

Hi all,

I'm trying to validate JSON data with Colander, and have a little
trouble finding the right way to support arbitrary key names.

Suppose I have phone numbers like in the example schema:

http://docs.pylonsproject.org/projects/colander/en/latest/basics.html#defining-a-colander-schema
{
'phones':[{'location':'home', 'number':'555-1212'},
{'location':'work', 'number':'555-8989'},],
}

How would I best go about validating custom fields, like for example:
{
'phones':[{'location':'home', 'number':'555-1212', 'type':'landline'},
{'provider':'telekom', 'number':'555-8989'},],
}

I want the user to be able to add arbitrary fields to the phone numbers.

At the moment, I solved it like this (using a sequence of "fieldname",
"value" pairs):
}
'phones':[{'number':'555-1212', fields=[{'fieldname':'provider',
'value':'telekom'}, {'fieldname':'type', 'value':'landline'],}],
}


Which works, but I think it would be much nicer to be able to just use
{"provider":"telekom"}.

Is this possible, and how would you validate data like this with
Colander? I could just choose to not validate the fields, but I would
prefer if I could still make sure that the value for a dynamic field
is a string etc.


Thanks in advance,

Jesaja Everling

[kusut]

On Monday, 17 December 2012 19:21:16 UTC+7, Jesaja Everling wrote:

Hi all,

I'm trying to validate JSON data with Colander, and have a little
trouble finding the right way to support arbitrary key names.

Hello. I posted an answer yesterday via email but it did not reach here (stuck somewhere).

I also had the same problem. My DIY solution is to do this before deserializing

if arbitrary:
    node.children = [
        child for child in self.children
        if child.name in data
    ]

# where data is some struct like request.POST

It's ugly. If anyone has a better idea, or colander actually supports
this, please let us know.

[kusut]

On 17 December 2012 19:21, Jesaja Everling <jeve...@gmail.com> wrote:
> Hi all,
>
> I'm trying to validate JSON data with Colander, and have a little
> trouble finding the right way to support arbitrary key names.
>

Hi. I also had the same problem. My DIY solution:

# do this somewhere before deserializing

if arbitrary:
node.children = [
child for child in self.children
if child.name in data
]

# data is some struct like request.POST

It's ugly. If anyone has a better idea, or colander actually supports
this, please let us know.

--
http://kusut.web.id

[Andi Balke]

hi jesaja,

Suppose I have phone numbers like in the example schema:

http://docs.pylonsproject.org/projects/colander/en/latest/basics.html#defining-a-colander-schema

...

How would I best go about validating custom fields, like for example:
{
'phones':[{'location':'home', 'number':'555-1212', 'type':'landline'},
          {'provider':'telekom', 'number':'555-8989'},],
}

from my experience you should define all fields that are possible and set them to be nullable::

  class Phone(colander.MappingSchema):

    # ...

    type = colander.SchemaNode(

        colander.String(),

        widget = deform.widget.HiddenWidget(),

        missing = colander.null,

    )

    provider = colander.SchemaNode(

        colander.String(),

        missing = colander.null,

    )

this seems cumbersome in some cases but from my experience its worth doing that cause otherwise you will have a lot of unexpected workarounds. having an - even huge - schema is clear and bulletproof. if you work in this pattern the first obvious advantage (beside security) is having little view-code for forms.

I want the user to be able to add arbitrary fields to the phone numbers.

At the moment, I solved it like this (using a sequence of "fieldname",
"value" pairs):

also possible, but the structure is not as nice (like you say below) and you cannot distinguish the type of each ``fieldname``. 

unless you know this is all the same kind of validation and you do not care about the payload structure i would not do that.

Is this possible, and how would you validate data like this with

Colander?

you can validate each field on its own as usual::

    type = colander.SchemaNode(

        colander.String(),

        widget = deform.widget.HiddenWidget(),

        validator = colander.Length(min=1, max=255),

        missing = colander.null,

    )

if you need a cross-validation, like "i need B validated in relation to A" use a ``Validator`` on the ``Schema``

https://github.com/Pylons/colander/issues/74

If you need a dynamic value during validation see this example at ``deferred_date_validator``:

http://docs.pylonsproject.org/projects/colander/en/latest/binding.html

I could just choose to not validate the fields, but I would
prefer if I could still make sure that the value for a dynamic field
is a string etc.

NO!

please: do not think about not validating them!!!11! ;)

cheers, andi