XML Mimer

[cootetom]

Hi,

I see in the piston code that the xml mimer that is registered is:

Mimer.register(lambda *a: None, ('text/xml',))

Do we have to register our own XML mimer or is there something going
on somewhere else in the code that I haven't found? If we need to
register our own where is the best place to do that? I typically use
beautiful soup for parsing XML, does anyone have any advice on what
they use?

Thanks

[patrickk]

we use a custom version of XML2Dict.

in piston/emitters.py:

def xml_deserializer(data):
from api.xml2dict import XML2Dict

xml = XML2Dict()
r = xml.fromstring(data)
return r

Emitter.register('xml', XMLEmitter, 'text/xml; charset=utf-8')
Mimer.register(xml_deserializer, ('text/xml',))

––––––––––

then you need a file xml2dict.py (we usually have an app called
"api"):

"""xml2dict: Convert bewteen xml file and python dict """

from api.oodict import OODict
import re

try:
import xml.etree.ElementTree as ET
except:
import cElementTree as ET # for 2.4

class XML2Dict:

def __init__(self):
pass

def _parse_node(self, node):
tree = OODict()
# Save value
value = node.text
if isinstance(value, str):
value = value.strip() # Only strip strings
tree.value = value
# Save attributes
for k,v in node.attrib.items():
tree.update(self._make_dict(k, v))
#Save childrens
for child in node.getchildren():
ctag = child.tag
ctree = self._parse_node(child)
cdict = self._make_dict(ctag, ctree)

if ctag not in tree: # First time found
tree.update(cdict)
continue

old = tree[ctag]
if not isinstance(old, list):
tree[ctag] = [old] # Multi entries, change to list
tree[ctag].append(ctree) # Add new entry

return tree

def _make_dict(self, tag, value):
"""Generate a new dict with tag and value
If tag is like '{http://cs.sfsu.edu/csc867/myscheduler}
patients',
split it first to: http://cs.sfsu.edu/csc867/myscheduler,
patients
"""

tmp = value
result = re.compile("\{(.*)\}(.*)").search(tag)
if result:
tmp = OODict()
tmp.xmlns, tag = result.groups() # We have a namespace!
tmp.value = value
return OODict({tag: tmp})

def parse(self, file):
"""Parse xml file to dict"""
f = open(file, 'r')
return self.fromstring(f.read())

def fromstring(self, s):
"""Parse xml string to dict"""
tmp = ET.fromstring(s)
return self._make_dict(tmp.tag, self._parse_node(tmp))

––––––––––

and a second file oodict.py:

"""OODict: object view of dict

Copyright (C) 2008-2009 Chen Zheng <nkc...@gmail.com>
Distributed under terms of GPL v2
"""

class OODict(dict):
"""
OODict
OO style dict

Examples:
>>> a = OODict({'a': 1, 'c': {'d': 2}, 'b': 2})
>>> a
{'a': 1, 'c': {'d': 2}, 'b': 2}
>>> a.a=0
>>> a
{'a': 0, 'c': {'d': 2}, 'b': 2}
>>> a.e=0
>>> a
{'a': 0, 'c': {'d': 2}, 'b': 2, 'e': 0}
>>> a.c = 5
>>> a
{'a': 0, 'c': 5, 'b': 2, 'e': 0}
>>> a.f = OODict({'f':'f'})
>>> a
{'a': 0, 'c': 5, 'b': 2, 'e': 0, 'f': {'f': 'f'}}
>>> a.f.f
'f'
>>> a.c = {'d': 2}
>>> a
{'a': 0, 'c': {'d': 2}, 'b': 2, 'e': 0, 'f': {'f': 'f'}}
>>> a.c
{'d': 2}
>>> a.c.d
2
>>> a.c.e = {'e': 'e'}
>>> a
{'a': 0, 'c': {'e': {'e': 'e'}, 'd': 2}, 'b': 2, 'e': 0, 'f':
{'f': 'f'}}
>>> a.c.e.e
'e'
>>> a['c']['e'].e
'e'

Problems:
* can't use del a.c, must use a['c']
*If a.k is a dict, v is returned still as a dict in the following
code:
for k, v in a.items():
pass # v is still a dict

You can use like this instead:
for k in a.keys():
v = a[k] # v is a OODict now

Perhaps we should define our own 'items'.
"""

def __init__(self, data = {}):
dict.__init__(self, data)

def __getitem__(self, key):
value = dict.__getitem__(self, key)

if isinstance(value, dict) and value.keys() == ['value']:
value = value['value']

if isinstance(value, dict) and not isinstance(value, OODict):
# Fixme! There maybe a problem here when value is a
subclass of dict
value = OODict(value)
self[key] = value # Auto covert children dict to OODict
return value

def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError

def __setattr__(self, key, value):
self[key] = value

if __name__ == "__main__":
import doctest
doctest.testmod()

––––––––––

for us, this works fine. hope it helps.

regards,
patrick

[cootetom]

Patrick, thanks for this code. It works very well and seems a neat
solution. Currently my handlers all work with the dictionaries built
from django's JSON parser. That being the case I think I'm going to
take your code and see if I can modify it so that the XML is parsed
into the same dictionary structure that my handlers already work with.
This will hopefully mean that I can use the same logic for handling
both XML and JSON. For the API I'm building I'm only interested in
JSON and XML but I think if I can get an XML parser working so that
handlers don't need to worry about the difference between these two
formats then it would be something really useful.

Perhaps when finished, if it works well enough, I'll contact jespern
and see if it could be added to the piston source as the XML mimer.
Would be nice to have something out of the box in the piston source.

Thanks again Patrick, very useful.

> Copyright (C) 2008-2009 Chen Zheng <nkch...@gmail.com>

[patrickk]

please let me know how it goes ...

here´s the ticket about the xml-mimer missing (just for the records):
http://bitbucket.org/jespern/django-piston/issue/76/posting-xml-doesn-t-work

regards,
patrick

[cootetom]

I've updated the ticket with my comment. I created a patch queue with
a much simpler xml mimer in it, it's in bitbucket
http://bitbucket.org/cootetom/xml-mimer/src/tip/XMLtoDict.patch

The mimer creates a dictionary like the one you get from the JSON
mimer. It's what I'm using now as it means I don't have to change any
handler code that already works with JSON requests.

Hopefully it'll be of use to others.

On Dec 21, 7:50 am, patrickk <sehmasch...@gmail.com> wrote:
> please let me know how it goes ...
>

> here´s the ticket about the xml-mimer missing (just for the records):http://bitbucket.org/jespern/django-piston/issue/76/posting-xml-doesn...