xml approach

[lars van gemerden]

Hi,

I am looking for a way to convert SQLalchemy objects to XML and back, in order to support a webapplication. I made a mixin class that does a decent job on 'normal' python objects (basically works for in the same cases as pickle, though more testing is required). I would prefer to have a simple mixin interface with something like:

def toXML(): 

    'convert attributes to xml'

    return xmlstring

def fromXML(xmlstring)

    'update attributes from xml'

The most simple case would be that the web user request indicates the primary key of the object, gets the corrsponding xml string in return, changes fields in the xml string and posts it back, which leads to an update of the object. 

I have formulated a couple of initial questions:

-  what attributes should be converted to and from xml?

-  how do i handle foreign keys on the 'other' side?

Note that the object remains in memory during the user interaction, so it does not have to be reinitialized from scratch.

Cheers, Lars

 

[Julien Cigar]

On 03/22/2012 11:36, lars van gemerden wrote:
> Hi,
>

Hello,

> I am looking for a way to convert SQLalchemy objects to XML and back,
> in order to support a webapplication. I made a mixin class that does a
> decent job on 'normal' python objects (basically works for in the same
> cases as pickle, though more testing is required). I would prefer to
> have a simple mixin interface with something like:
>
> def toXML():
> 'convert attributes to xml'
> return xmlstring
>
> def fromXML(xmlstring)
> 'update attributes from xml'
>
> The most simple case would be that the web user request indicates the
> primary key of the object, gets the corrsponding xml string in return,
> changes fields in the xml string and posts it back, which leads to an
> update of the object.
>
> I have formulated a couple of initial questions:
>
> - what attributes should be converted to and from xml?
> - how do i handle foreign keys on the 'other' side?
>

If it can help, I made a serializer tool some time ago, it's not yet
complete but it can already serialize columns, relations, etc to XML,
JSON, and dict
Code is on http://www.pastie.org/3647038

> Note that the object remains in memory during the user interaction, so
> it does not have to be reinitialized from scratch.
>
> Cheers, Lars
>

best regards,
Julien

> --
> You received this message because you are subscribed to the Google
> Groups "sqlalchemy" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/sqlalchemy/-/QaC2CW65FVgJ.
> To post to this group, send email to sqlal...@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.

--
No trees were killed in the creation of this message.
However, many electrons were terribly inconvenienced.

[lars van gemerden]

Hi Julian,

Thanks, I am looking into it and it looks interesting. Have you done much testing yet? How do you (plan to; haven't looked at too much detail yet) check for circular references (like backrefs)?

Cheers, Lars

[Julien Cigar]

On 03/23/2012 10:37, lars van gemerden wrote:
> Hi Julian,
>
> Thanks, I am looking into it and it looks interesting. Have you done
> much testing yet? How do you (plan to; haven't looked at too much
> detail yet) check for circular references (like backrefs)?
>

circular references aren't handled yet .. at the moment all what you can
do is something like:

>>> a = Page.query.get(454)

>>> SAMappedSerializer(a).json(exclude_columns=('body', ),
include_relations=('type', 'tags'))
'{"updated": "2012-03-21T16:52:43Z", "added": "2009-10-16T14:23:24Z",
"description": "The Belgian Biodiversity Platform", "effective": null,
"tags": [{"description": null, "id": 26, "name": "Belspo"},
{"description": null, "id": 29, "name": "Belgian Biodiversity
Platform"}], "icon_content_id": null, "count_children": 0, "customized":
false, "weight": 23, "owner_id": 1, "content_id": 454, "expiration":
null, "polymorphic_loading": null, "state_id": 3, "title": "About us",
"exclude_nav": false, "type": {"description": null, "icon": "page.png",
"id": 2, "name": "page"}, "id": 454, "container_id": 1,
"content_type_id": 2}'

>>> SAMappedSerializer(a).xml(exclude_columns=('body', ),
include_relations=('type', 'tags'))
<content><updated>2012-03-21T16:52:43Z</updated><added>2009-10-16T14:23:24Z</added><description>The
Belgian Biodiversity Platform</description><effective
/><tags><tags_0><description
/><id>26</id><name>Belspo</name></tags_0><tags_1><description
/><id>29</id><name>Belgian Biodiversity
Platform</name></tags_1></tags><icon_content_id
/><count_children>0</count_children><customized>False</customized><weight>23</weight><owner_id>1</owner_id><content_id>454</content_id><expiration
/><polymorphic_loading /><state_id>3</state_id><title>About
us</title><exclude_nav>False</exclude_nav><type><description
/><icon>page.png</icon><id>2</id><name>page</name></type><id>454</id><container_id>1</container_id><content_type_id>2</content_type_id></content>

etc

> Cheers, Lars
>
> On Thursday, March 22, 2012 11:36:41 AM UTC+1, lars van gemerden wrote:
>
> Hi,
>
> I am looking for a way to convert SQLalchemy objects to XML and
> back, in order to support a webapplication. I made a mixin class
> that does a decent job on 'normal' python objects (basically works
> for in the same cases as pickle, though more testing is required).
> I would prefer to have a simple mixin interface with something like:
>
> def toXML():
> 'convert attributes to xml'
> return xmlstring
>
> def fromXML(xmlstring)
> 'update attributes from xml'
>
> The most simple case would be that the web user request indicates
> the primary key of the object, gets the corrsponding xml string in
> return, changes fields in the xml string and posts it back, which
> leads to an update of the object.
>
> I have formulated a couple of initial questions:
>
> - what attributes should be converted to and from xml?
> - how do i handle foreign keys on the 'other' side?
>
> Note that the object remains in memory during the user
> interaction, so it does not have to be reinitialized from scratch.
>
> Cheers, Lars
>

> --
> You received this message because you are subscribed to the Google
> Groups "sqlalchemy" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/sqlalchemy/-/QNBLxpPxDBgJ.

[lars van gemerden]

Using some of your example, together with code i wrote to XMLify
simple python classes (like pickle, but xml out.input) I have got
something working that handles circular references and updates the
database after XML has been changed(e.g. in a client) and parsed
back.

It creates a reference dict with python function id(obj) as key
(registry[id(obj)] = obj) to later see whether an object has already
been written to or read from the XML string. This id is stored in the
XML tag (<TAGNAME ID = 1245125> something </TAGNAME>). If the id is
already in the registry, it will write new occurences of obj as
<TAGNAME ID = "1245125"/>. When the xml is read, basically the same
registry is created and used to look up objects that have been created
already.

I also stores the primary key(s) as XML attribute (<TAGNAME PK =
"(2325525,)"> something </TAGNAME>) to be able to look up the correct
database entry to be updated when the XML is read back
(session.query(cls).get(eval(element.get("PK")))..

The result looks something like this:

<somemodule.Address PK = "(1,)" ID = "46720168">
<int name = "id"> 1 </int>
<unicode name = "data"> bla bla </unicode>
<sqlalchemy.orm.collections.InstrumentedList name = "persons" ID =
"46689160">
<somemodule.Engineer PK = "(1,)" ID = "46695480">
<somemodule.Address PK = "(1,)" name = "address" ID = "46720168"/>
<int name = "id"> 1 </int>
<int name = "address_id"> 1 </int>
<unicode name = "firstname"> john </unicode>
</somemodule.Engineer>
</sqlalchemy.orm.collections.InstrumentedList>
</somemodule.Address>

(xml attribute "name" is the name of the attribute)

If there is any interest i will post a sqlalchemy specific version of
the code, when it is somewhat ready.

Cheers, Lars

> /><count_children>0</count_children><customized>False</customized><weight>2 3</weight><owner_id>1</owner_id><content_id>454</content_id><expiration

> /><polymorphic_loading /><state_id>3</state_id><title>About
> us</title><exclude_nav>False</exclude_nav><type><description

> /><icon>page.png</icon><id>2</id><name>page</name></type><id>454</id><conta iner_id>1</container_id><content_type_id>2</content_type_id></content>

>  jcigar.vcf
> < 1KViewDownload