Can't represent certain types of structures? (e.g. empty Dict)
Steven Kryskalla
I am struggling coming up with schemas for arbitrary structures that
don't look like form data (i.e. lists of named values).
For example, I tried coming up with a schema for this YAML
configuration data:
http://stackoverflow.com/questions/1061482/why-isnt-this-a-valid-schema-for-rx
The main problem I have is that for a nested structure everything has
to be "named". To validate the outermost structure, the schema needs
to specify a Dict of Dicts, where the names of the Dicts aren't known
ahead of time. Here's my attempt:
CmdSchema = Dict.of(
String.named('exec'),
List.named('aliases').using(optional=True).of(
String.using(validators=[LengthBetween(1,10)]),
),
Dict.named('filter').using(optional=True).of(
String.named('sms').using(optional=True),
String.named('email').using(optional=True),
String.named('all').using(optional=True),
)
)
ConfigSchema = Dict.of(CmdSchema)
When I go to instantiate it:
>>> ConfigSchema(yaml.load(testyml))
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "flatland-dev-py2.6.egg\flatland\schema\containers.py", line
745, in __init__
self.set(value)
File "flatland-dev-py2.6.egg\flatland\schema\containers.py", line
992, in set
self.name, key))
KeyError: "Dict None schema does not allow key 'load'"
Is there a way to get this kind of schema to work?
Also, can flatland represent a kind of "Any" type, which matches any
type of element? I tried using Element (e.g. List.of(Element)), but it
seems you can't include it in a schema on its own.
Finally, can flatland validate that a container is of a certain type
(without any member schema)? For example this works for the scalar
types:
>>> String()
<String None; value=None>
>>> String("hi")
<String None; value=u'hi'>
But not for the container types:
>>> List()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "flatland-dev-py2.6.egg\flatland\schema\containers.py", line
179, in __init__
type(self))
TypeError: Invalid schema: <class 'flatland.schema.containers.List'>
has no member_schema
>>> List([1,2])
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "flatland-dev-py2.6.egg\flatland\schema\containers.py", line
179, in __init__
type(self))
TypeError: Invalid schema: <class 'flatland.schema.containers.List'>
has no member_schema
>>> Dict()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "flatland-dev-py2.6.egg\flatland\schema\containers.py", line
742, in __init__
type(self).__name__))
TypeError: 'Dict' dictionary type has no fields defined
>>> Dict({'a': 1})
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "flatland-dev-py2.6.egg\flatland\schema\containers.py", line
742, in __init__
type(self).__name__))
TypeError: 'Dict' dictionary type has no fields defined
I looked at some other schema validation tools, mainly Rx, and it
seems they can handle most of these cases. For example to solve the
the Dict of Dicts problem, you can use this schema:
{
"type": "//map",
"values": {
"type": "//map",
"values": "//any"
}
}
thanks!
Steve
jason kirtland
Yes, though to my knowledge no one has yet published a Flatland type
that implements it. In theory a fairly light subclass of the Mapping
base type could allow for arbitrary keys. The simplest implementation
would likely involve applying a uniform value Element type to all the
values, for example a Mapping of string keys to Strings.
Given an implementation this lightly structured Mapping type, call it
SimpleMapping for lack of anything better coming to mind, you'd just
use it in place of Dict.
ConfigSchema = SimpleMapping.of(CmdSchema)
So that would get you the dictionary of arbitrarily named,
well-defined dictionary values.
If you want to give that a try, I don't think it will be too tough.
In the short term I'm swamped in the final days before a big project
release, otherwise I'd be happy to jump in to implement this request
myself. I expect to pop out of this hole after the first week of
December.
> Also, can flatland represent a kind of "Any" type, which matches any
> type of element? I tried using Element (e.g. List.of(Element)), but it
> seems you can't include it in a schema on its own.
Ali has suggested on IRC that this implementation be included in the
distribution. You might give it a try:
class Anything(Scalar):
def adapt(self, value):
return value
Yes, but not with that spelling*. The container types in flatland are
always defined as Lists *of* something, or Dicts *limited to fields
of* something. I haven't personally found it useful at the schema
validation level to do a generic test: Python's "isinstance" usually
does a fine job of that. It's been more useful to me to make sure the
structure is what I expect, down to the types of the list members and
dict values.
Given a little more structure, say a List.of(Integer)([1, 2]), you can
answer that question, and if you want to you can go further and
'assert isinstance(List.of(Integer)([1, 2]).raw, list)' to see that
the input value was of an exact type.
( *Given the Anything and SimpleMapping above, you could write your
queries as List.of(Anything)([1,2,'a','b']) or
SimpleMapping.of(Anything)({'a', 'b'}) )
> I looked at some other schema validation tools, mainly Rx, and it
> seems they can handle most of these cases. For example to solve the
> the Dict of Dicts problem, you can use this schema:
>
> {
> "type": "//map",
> "values": {
> "type": "//map",
> "values": "//any"
> }
> }
Pretty nice. I don't think flatland will ever get that compact out of
the box, but I'm confident that it can be equally expressive.
Thanks for bringing the new use case and for the interest in flatland!
Cheers,
Jason
> thanks!
> Steve
>
> --
> You received this message because you are subscribed to the Google Groups "flatland-users" group.
> To post to this group, send email to flatlan...@googlegroups.com.
> To unsubscribe from this group, send email to flatland-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/flatland-users?hl=en.
>
>