Inserting large XML files efficiently

[Nyenyec N]

Hi,

I asked this question on the kid mailing list but didn't get any useful answer.

In my application I have relatively large (20k-100k) XHTML files in
the DB. I'd like to show a number of these to the user based based on
some filtering criteria using a single page.

I didn't find any efficient way in kid to include large XML documents.
Using the XML() function makes it very slow and the page takes more
than 2 seconds to generate.

In the context of TurboGears are there any efficient solutions to do
this? Are the alternative templating systems well tested? Can I still
use widgets on my page if I switch from kid to something else?

Thanks,
nyenyec

[Martina Oefelein]

Hi Nyenyec,

> In my application I have relatively large (20k-100k) XHTML files in
> the DB. I'd like to show a number of these to the user based based on
> some filtering criteria using a single page.
>
> I didn't find any efficient way in kid to include large XML documents.
> Using the XML() function makes it very slow and the page takes more
> than 2 seconds to generate.
>
> In the context of TurboGears are there any efficient solutions to do
> this?

One option would be to serve the XHTML files separately from the
templated page. In the templated page, use either iframes to include
the xhtml files, or use a bit of javascript to load them via
XmlHttpRequest.

Another option would be to put placeholders for the xhtml files in
the template, and postprocess the kid-generated html to replace these
placeholders with the real xhtml data.

ciao
Martina

[Nyenyec N]

This kind of sucks. I hoped there was an easy way to include large
pre-formatted portions in a kid template.

I have another: are there any examples on implementing caching of
XHTML fragments with Kid?

Thanks,
nyenyec

[David Bernard]

Hi,

Nyenyec N a écrit :

> In the context of TurboGears are there any efficient solutions to do
> this? Are the alternative templating systems well tested? Can I still
> use widgets on my page if I switch from kid to something else?

TurboHtmlPy (alternative templating engine) could may be help you.
It supports TG's Widgets.
About insertion of XML fragment (name xmldata in the dict return by the
controller), several choices :
* fragment is a string, and you want the template engine check it and convert
char before insertion: use the "hpy__xml(vars.xmldata)"
* fragment is a string, and you *don't* want the template engine check it and
convert char before insertion (already done...): use the "hpy__raw(vars.xmldata)"
* fragment is a string and you want to display the xml into the xhtml page with
> and < : use "vars.xmldata" directly
* fragment is an elementtree.Element : use "vars.xmldata" directly

To test the solution you could simply :
* install TurboHtmlPy
$> easy_install TurboHtmlPy

* convert/create a template file to check time need to display XML:

#!/usr/bin/python
# -*- coding: utf-8 -*-

html(
body(
hpy__raw(vars.xmldata),
),
)

* modify the controllors method to use the alternate template
@expose(template="htmlpy:prj.template.displayxml")
def displayxml():
xmldata="<greet>Hello</greet>"
return dict(xmldata=xmldata)

If you have any questions, I'll be please to answer/help.
I hope TurboHtmlPy could help you.

--
--------------------------------------------------------------
David "Dwayne" Bernard TurboHtmlPy creator

[David Bernard]

I forgot : documentation about TurboHtmlPy is http://dwayneb.free.fr/turbohtmlpy

David Bernard a écrit :

--------------------------------------------------------------
David "Dwayne" Bernard Freelance Developer
mailto:dwa...@java-fan.com
\|/ http://dwayne.java-fan.com
--o0O @.@ O0o-------------------------------------------------