[Setup] huge performance issues using Diazo

[Héctor Velarde]

during the deployment of a new site using Diazo we have found huge
performance problems causing delays of more than 25 seconds on each page
loaded.

these problems are more evident, and intolerable, for logged in users
accessing the site to add content.

a quick review using some profiling tool showed us where the problem is:
64 calls to transformIterable() [1] causing delays of 0.374 seconds each
one.

we obviously need to rethink the theme to reduce the number of
transformations, but I would like to listen to other people's advice:

have you ever found such performance issues when using Diazo on your
deployments?

what's the best approach for using Diazo with complex themes? WSGI or
nginx? [2]

best regards

[1]
https://github.com/plone/plone.app.theming/blob/master/src/plone/app/theming/transform.py#L200

[2] http://docs.diazo.org/en/latest/deployment.html
--
Héctor Velarde

[Laurence Rowe]

hvelarde wrote

I've certainly never observed anything like this. Martin's performance tests
from a few years ago showed that performing the transform in process was
faster than doing so in a proxy. For Plone, plone.app.theming is the
recommended deployment mechanism.

There's not really enough information here to know why this might be
happening. If you can isolate it and turn it into something I can easily
reproduce in a standard plone.app.theming build I'll take a look.

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7372162.html
Sent from the Installation, Setup, Upgrades mailing list archive at Nabble.com.
_______________________________________________
Setup mailing list
Se...@lists.plone.org
https://lists.plone.org/mailman/listinfo/plone-setup

[Héctor Velarde]

On 14/03/12 12:34, Laurence Rowe wrote:
> I've certainly never observed anything like this. Martin's performance tests
> from a few years ago showed that performing the transform in process was
> faster than doing so in a proxy. For Plone, plone.app.theming is the
> recommended deployment mechanism.
>
> There's not really enough information here to know why this might be
> happening. If you can isolate it and turn it into something I can easily
> reproduce in a standard plone.app.theming build I'll take a look.

I made some additional profiling using a development buildout in my
machine and now I'm more confused:

ncalls tottime percall cumtime percall filename:lineno(function)
53 9.218 0.174 12.062 0.228
transform.py:200(transformIterable)

so, time spent on a laptop running with low memory in development mode
is half than time on a dedicated server running plenty of memory in
production mode; this doesn't make sense.

could it be some issue with the lxml installed on the server? some issue
with it's compilation?

BTW, I'm running Plone 4.1.3, Dexterity 1.1, plone.app.theming 1.0b9 and
lxml 2.3 on my machine.

I will confirm current versions on the server later.

best regards

[Laurence Rowe]

hvelarde wrote

plone.app.theming.transform.transformIterable should be called only once per
request, I would suggest working out why is being called so many times.

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7374959.html

[Héctor Velarde]

On 15/03/12 06:11, Laurence Rowe wrote:
> plone.app.theming.transform.transformIterable should be called only once per
> request, I would suggest working out why is being called so many times.

no, plone.app.theming.transform.transformIterable is called on _every_
request; it just exits early if the content isn't Diazo's business, but
that's another story.

after more profiling I have found that all the delay happens inside this
method when the rules are applied and it's very different depending on
what kind of page is being processed; it's becoming critical on content
creation (30s vs 1s with no Diazo theme applied).

we're right now working on a complete refactoring of the rules, but I
think it's important to give more information to theme developers about
how complex sites performance could be affected by Diazo and what are
the best practices do deal with this.

as I mentioned before, this is extremely important on the content
creation side because (and correct me if I'm wrong) no caching can be
applied there and the delay occurs all the time.

best regards

[Nathan Van Gheem]

Hector,

Are you doing a lot of href(sub requests) in your rules to pull in
different content?

I can imagine that can really slow down performance.

[Mikko Ohtamaa]

we're right now working on a complete refactoring of the rules, but I think it's important to give more information to theme developers about how complex sites performance could be affected by Diazo and what are the best practices do deal with this.

From XSLT work it did many many many years ago there was one golden rule:

Never user //tag selector

It traverses the whole document, every tag and is slow as hell. Always try to find id based selectors, or very accurate selectors.

Also make sure whether there is difference if XPath selectors are evaluated from left to right or right to left. CSS is right to left, but I am not sure about XPath.

[emb03]

Hi,

I would be interested in how to optimize a rules file. I am using tag
selectors (I can't fathom not using them) and we are having a weird problem
with css from plone sunburst theme replicating themselves 6 times on the
front page (the same css file shows up six times). I based my theme on
Sunburst and wondering if that is causing this issue.

Elaine

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7379383.html

Sent from the Installation, Setup, Upgrades mailing list archive at Nabble.com.

[Ken Wasetis [Contextual Corp.]]

Just curious - do you have a main rules file that includes other rules
files (main one that conditionally includes one or more include rules
files, such as one for home page, one for interior landing pages, one
for other interior pages, etc.)? If so, perhaps you've copied pasted
your <head> area rules into those include rules files and so you're
adding them more than once into the resulting theme.

-Ken

[emb03]

Hi Ken,

Thanks for responding. No, I really have just the basics in the rules file--
nothing fancy at all.

<?xml version="1.0" encoding="UTF-8"?>
<rules
xmlns="http://namespaces.plone.org/diazo"
xmlns:css="http://namespaces.plone.org/diazo/css"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<rules css:if-content="#visual-portal-wrapper">
<theme href="theme.html"/>
</rules>

<prepend content='//*[@id="portal-personaltools"]'
theme='//*[@id="header"]'/>
<copy css:content="div#portlets-footer" css:theme="div#footer" />
/>

<copy attributes="href title" css:content='#portal-logo' css:theme="div.logo
a"/>
<before css:content="head link" css:theme="head link" />
<replace css:content="head title" css:theme="head title" />
<copy css:content="head base" css:theme="head base" />

<copy css:content="#portal-column-two" css:theme="#sidebar" />
<copy css:content="#portal-column-content" css:theme=".main_content" />

<copy attributes="id dir class" css:content="body" css:theme="body"
/>
<before
css:content="div.manageInHeaderPortletsLink a.managePortletsFallback"
css:theme-children="div.tencol"
/>
<replace content='children:#parent-fieldname-text'
theme='children:#body'
if-content='#parent-fieldname-text'/>
<copy attributes="class" css:content="body" css:theme="body"/>

<copy attributes="href title" css:content="#portal-logo" css:theme="#logo
a"/>

<replace css:content="#portal-searchbox" css:theme=".search"/>
<drop
css:content="div.searchSection"
/>

<replace
css:content-children="ul#portal-globalnav"
css:theme-children="ul#nav"
/>
<merge attributes="class" css:theme=".dir" css:content="ul" />

<after css:content="head script" css:theme="#wrapper" />
<after css:content="#visual-portal-wrapper .row:last-child script"
css:theme="#wrapper" />

</rules>

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7379603.html

[Ken Wasetis [Contextual Corp.]]

Perhaps browse to the 'unthemed' version of your site (okay, actually is
applying the Sunburst theme by default, but not your Diazo theme).

By default, your Diazo settings in Site Setup are probably setup in the
Advanced tab to allow requests to '127.0.0.1' to bypass the Diazo theme
and just let Plone serve up the default Plone theme (Sunburst, if you
haven't changed it.) You can add some other domain (e.g.,
test.yourdomain.org) if you aren't running Plone locally on 127.0.0.1
and aren't able to create an ssh tunnel to it, so that you can make
requests to your server-based Plone site.

Once you are able to browse to your Plone site directly without any
Diazo theme involved, do a view source and search for all of the <link>
and <style> tags in the <head> or search for '.css' include references
in the source for the list of all CSS files being requested by the page
(or use a browser add-on, such as Web Developer Tool or Firebug for
Firefox to 'View CSS'.)

Confirm that you aren't seeing multiple copies of the same CSS in the
non-Diazo pages of your site. If you do see 6 or however many copies
(more than 1) of the same .css being referred to, then at least you can
eliminate Diazo as being the problem. Perhaps you have multiple entries
in your portal_css registry or have hardcoded references in your
main_template, header, or other viewlets to look at instead.

But what you describe does sound more like a Diazo rules issue. I was
looking at what you sent below, trying to find some 'for-each' loop that
was perhaps the case, but I don't see one.

One other thing to verify: Be sure that you don't have multiple CSS
references in your static theme HTML file in its <head> section *in
addition to* the CSS references (<link> tags) that you're copying into
the Diazo-themed page in your rules below. Maybe someone did some
copy/paste work in your static html theme page that's causing this.

Cheers,
Ken

--
Ken Wasetis

President& CMS Solution Architect
Contextual Corp.
office: 847-356-3027
ken.w...@contextualcorp.com

[Laurence Rowe]

hvelarde wrote

>
> On 15/03/12 06:11, Laurence Rowe wrote:
>> plone.app.theming.transform.transformIterable should be called only once
>> per
>> request, I would suggest working out why is being called so many times.
>
> no, plone.app.theming.transform.transformIterable is called on _every_
> request; it just exits early if the content isn't Diazo's business, but
> that's another story.
>
> after more profiling I have found that all the delay happens inside this
> method when the rules are applied and it's very different depending on
> what kind of page is being processed; it's becoming critical on content
> creation (30s vs 1s with no Diazo theme applied).
>

Are you are running in production mode? bin/instance console rather than
bin/instance fg if you want to run non-daemonised. If you run in debug mode
then the whole theme is compiled down to XSLT every request (so changes in
your theme appear immediately.)

What does your profiling data say for that single request? How much time is
spent in the transformIterable and then how much applying the XSLT itself?
libxslt is generally pretty fast and you generally need a very long document
before you need to worry about optimising your xpaths.

> we're right now working on a complete refactoring of the rules, but I
> think it's important to give more information to theme developers about
> how complex sites performance could be affected by Diazo and what are
> the best practices do deal with this.
>
> as I mentioned before, this is extremely important on the content
> creation side because (and correct me if I'm wrong) no caching can be
> applied there and the delay occurs all the time.
>

Running the XSLT to apply a Diazo theme is usually orders of magnitude
faster than rendering the Plone page to begin with. If you are seeing really
slow rendering and you are sure that the time is being spent applying the
XSLT then you have a couple of options:

1. Simple binary search. Remove half your theme, rerun it and see how long
it takes. If it's taking 30s then you can do that in development mode, even
when compiling the theme every request Diazo should take < 1s. (I would
conside 10-50ms to be normal, but obviously it depends on the complexity of
your theme.)

2. Compile your theme with the standalone Diazo and use xsltproc --timing to
see how long it takes. This is a trivial theme and content document:

$ bin/diazocompiler -r rules.xml -o compiled.xsl
$ xsltproc --timing --repeat --html --noout compiled.xsl content.html
Parsing stylesheet compiled.xsl took 0 ms
Parsing document content.html took 0 ms
Applying stylesheet 20 times took 5 ms

Using the plone.org theme (compiled with an old version of xdv which
produces slightly slower XSLT) against a 563KB page (the source to
http://plone.org/documentation/manual/theme-reference/referencemanual-all-pages)
is

$ xsltproc --html --timing --noout default.xsl
theme-reference-all-pages.html
Parsing stylesheet default.xsl took 1 ms
# lots of warnings about invalid html, they are ignored so don't worry.
Parsing document theme-reference-all-pages.html took 35 ms
Applying stylesheet took 217 ms

(Unfortunately the xsltproc --profile only gives per template statistics
which is not very useful for us.)

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7380644.html

[Laurence Rowe]

emb03 wrote

>
> Hi,
>
> I would be interested in how to optimize a rules file. I am using tag
> selectors (I can't fathom not using them) and we are having a weird
> problem with css from plone sunburst theme replicating themselves 6 times
> on the front page (the same css file shows up six times). I based my theme
> on Sunburst and wondering if that is causing this issue.
>
> Elaine
>

The rule giving you the problem is this one:

Diazo rules may match multiple nodes in each document. This rule places a
copy of all of the link tags from the content before each link tag in the
theme. (I'm guessing your theme has 6 link tags in the head.) You probably
want to say something like this instead:

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7380649.html

[Holden Hao]

On Sat, Mar 17, 2012 at 1:15 AM, emb03 <ela...@ssc.ucla.edu> wrote:

Hi Ken,

Thanks for responding. No, I really have just the basics in the rules file--
nothing fancy at all.

Check your theme if it also calls the Sunburst CSS.  Your rules xml did not show that you dropped them but you have rules that included Plone's CSS.  That could cause the loop.  Also, check your CSS for @import directives.



Holden

[Laurence Rowe]

Laurence Rowe wrote

>
> Using the plone.org theme (compiled with an old version of xdv which
> produces slightly slower XSLT) against a 563KB page (the source to
> http://plone.org/documentation/manual/theme-reference/referencemanual-all-pages)
> is
>
> $ xsltproc --html --timing --noout default.xsl
> theme-reference-all-pages.html
> Parsing stylesheet default.xsl took 1 ms
> # lots of warnings about invalid html, they are ignored so don't worry.
> Parsing document theme-reference-all-pages.html took 35 ms
> Applying stylesheet took 217 ms
>

For comparison, the same rules.xml compiled with Diazo instead of XDV runs
in approximately half the time:

$ xsltproc --html --timing --noout diazo-default.xsl
theme-reference-all-pages.html
Parsing stylesheet diazo-default.xsl took 0 ms
# Same invalid html parsing warnings

Parsing document theme-reference-all-pages.html took 35 ms

Applying stylesheet took 113 ms

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7380708.html

[emb03]

Thanks, Laurence, for pointing me in the right direction. This ended up working and getting rid of the duplicate style sheets:

    <before css:content="head style, head link, head meta" css:theme-children="head" />

Thanks for your hard work, we appreciate it!

Elaine

Elaine Blakeman

UCLA Social Sciences Computing

PAB 2041 (Public Affairs Building)

Web and Multimedia Development

[hidden email]

310-206-1524

On Mar 16, 2012, at 6:17 PM, Laurence Rowe [via Plone] wrote:

emb03 wrote

Hi,

I would be interested in how to optimize a rules file. I am using tag selectors (I can't fathom not using them) and we are having a weird  problem with css from plone sunburst theme replicating themselves 6 times on the front page (the same css file shows up six times). I based my theme on Sunburst and wondering if that is causing this issue.

Elaine

The rule giving you the problem is this one:

 <before css:content="head link" css:theme="head link" />

Diazo rules may match multiple nodes in each document. This rule places a copy of all of the link tags from the content before each link tag in the theme. (I'm guessing your theme has 6 link tags in the head.) You probably want to say something like this instead:

 <before css:content="head link" theme="/head/link[1]" />

Laurence

---

If you reply to this email, your message will be added to the discussion below:

http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7380649.html

To unsubscribe from huge performance issues using Diazo, click here.
NAML

---

View this message in context: Re: huge performance issues using Diazo

[Suresh V.]

Accessing /sitemap also seems to do this. Not sure how this can be disabled.

[Héctor Velarde]

just to let you know how we reduced the process time from 25 to 2.5
seconds: we changed most conditional rules from CSS3 to XPath selectors.

here is an example:

- <rules css:if-content=".template-section-view,
- .template-opinion-view,
- .template-opinion-interview-view,
- .template-opinion-context-view">
+ <rules if-content="//*[contains(@class, 'template-section-view')] or
+ //*[contains(@class, 'template-opinion-view')] or
+ //*[contains(@class,
'template-opinion-interview-view')] or
+ //*[contains(@class,
'template-opinion-context-view')]">

Diazo documentation states that: "CSS selectors are replaced by the
equivalent XPath selector during the pre-processing step of the
compiler. Thus, they have no performance impact."

this seems to be false according to what we have found.

I would love to give you more information on this, but the guy who
actually wrote the rules is a little bit busy right now.

it would be great to have some sort of tool to test performance for
rules files before deployment.

Héctor Velarde

[Ken Wasetis [Contextual Corp.]]

Héctor,

I'm just curious - while the example below is for a condition that is likely only checked once per page render, did you have some other 'if-content' conditions that are checked more frequently?  Perhaps in iterations, even, such as across all links in portal-globalnav, all <li> tags within a navigation portlet, etc.?  I would think that if there is some performance gain to be had from converting CSS3 to XPath selectors at the outset (in the rules.xml) that if you're seeing such significant performance gains it's more likely to be in rules that apply to many links/portlets/items on a page, so that it's more of a compounded problem/remedy.

I'm seeing some very, very large/long content pages in one of our Diazo-themed sites producing some poor performance too, so thanks for the tip below.  I'll be trying out XPath vs. CSS3 syntax to see whether that helps in our case, or if it's just a matter of too much content being pushed through the XSLT layer on a given page.  We get reasonable (2 sec) page load times when viewing through an 'unthemed' URL (just has Plone Sunburst theme applied w/o Diazo involvement), but getting 30-60 second page load times when viewing through the Diazo-applied-theme URL.

I'll report back if I can confirm that we see similar improvements from converting the syntax to XPath.

Thanks,
Ken

_______________________________________________
Setup mailing list
Se...@lists.plone.org
https://lists.plone.org/mailman/listinfo/plone-setup

-- 
Ken Wasetis

President & CMS Solution Architect
Contextual Corp.
office: 847-356-3027
ken.w...@contextualcorp.com

[Laurence Rowe]

hvelarde wrote

>
> just to let you know how we reduced the process time from 25 to 2.5
> seconds: we changed most conditional rules from CSS3 to XPath selectors.
>
> here is an example:
>
> - <rules css:if-content=".template-section-view,
> - .template-opinion-view,
> - .template-opinion-interview-view,
> - .template-opinion-context-view">
> + <rules if-content="//*[contains(@class, 'template-section-view')] or
> + //*[contains(@class, 'template-opinion-view')] or
> + //*[contains(@class,
> 'template-opinion-interview-view')] or
> + //*[contains(@class,
> 'template-opinion-context-view')]">
>
> Diazo documentation states that: "CSS selectors are replaced by the
> equivalent XPath selector during the pre-processing step of the
> compiler. Thus, they have no performance impact."
>
> this seems to be false according to what we have found.
>

No, your selectors are not equivalent. The xpath "//*[contains(@class,
'template-section-view')]" matches any element with 'template-section-view'
in the class. That could be or equally

The reason it is so slow is that every single rule you have is testing that
there is any element in the page with that class. Those are body classes and
you should be restrict them to the body element where they appear, i.e.:

This should reduce your XSLT execution time by two orders of magnitude.

> I would love to give you more information on this, but the guy who
> actually wrote the rules is a little bit busy right now.
>
> it would be great to have some sort of tool to test performance for
> rules files before deployment.
>

It would be great to have a diazo debug toolbar with this sort of
information. We'll probably get around to it eventually, but if anyone is
interested in sponsoring such work do email me.

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7417171.html

[Laurence Rowe]

ctxlken wrote

>
> Héctor,
>
> I'm just curious - while the example below is for a condition that is
> likely only checked once per page render, did you have some other
> 'if-content' conditions that are checked more frequently? Perhaps in
> iterations, even, such as across all links in portal-globalnav, all <li>
> tags within a navigation portlet, etc.? I would think that if there is
> some performance gain to be had from converting CSS3 to XPath selectors
> at the outset (in the rules.xml) that if you're seeing such significant
> performance gains it's more likely to be in rules that apply to many
> links/portlets/items on a page, so that it's more of a compounded
> problem/remedy.
>
> I'm seeing some very, very large/long content pages in one of our
> Diazo-themed sites producing some poor performance too, so thanks for
> the tip below. I'll be trying out XPath vs. CSS3 syntax to see whether
> that helps in our case, or if it's just a matter of too much content
> being pushed through the XSLT layer on a given page. We get reasonable
> (2 sec) page load times when viewing through an 'unthemed' URL (just has
> Plone Sunburst theme applied w/o Diazo involvement), but getting 30-60
> second page load times when viewing through the Diazo-applied-theme URL.
>
> I'll report back if I can confirm that we see similar improvements from
> converting the syntax to XPath.
>

Conditions on a rules tag are just logically ANDed to the conditions on all
descendant rules, so they are executed multiple times per render (often
multiple times for every element on the page). The first thing to do is to
make your conditions more specific (testing a class currently requires
testing every otherwise matched element.)

To execute a condition once per render you must set it as a variable, e.g:

But this will not always result in a speedup. Unfortunately XSLT1 is a
little limited and can't directly refer to variables in which are used for
drop/replace content and we have to get around this by dynamically
evaluating those expressions which can be slower than just executing the
condition multiple times.

There are a number of potential optimizations that could be applied
automatically in a future version of Diazo. As ever, contributing (with code
or sponsorship) is the way to make this happen sooner.

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7417289.html

Sent from the Installation, Setup, Upgrades mailing list archive at Nabble.com.

[Kees Hink]

> No, your selectors are not equivalent. The xpath "//*[contains(@class,
> 'template-section-view')]" matches any element with 'template-section-view'
> in the class. That could be or equally
>
> The reason it is so slow is that every single rule you have is testing that
> there is any element in the page with that class. Those are body classes and
> you should be restrict them to the body element where they appear, i.e.:

Thanks Hector and Laurence for this thread.

We had the same problem, and re-writing our if-content selectors brought
page load time down from 42 to 7 seconds.

Laurence, transformIterable is called many times, as in Hector's case.
Do you still think this is wrong?

Kees

[Kees Hink]

> No, your selectors are not equivalent. The xpath "//*[contains(@class,
> 'template-section-view')]" matches any element with 'template-section-view'
> in the class. That could be or equally
>
> The reason it is so slow is that every single rule you have is testing that
> there is any element in the page with that class. Those are body classes and
> you should be restrict them to the body element where they appear, i.e.:
>
> This should reduce your XSLT execution time by two orders of magnitude.

Thanks Hector and Laurence for this thread.

We had the same problem, and re-writing our if-content selectors brought
page load time down from 42 to 7 seconds.

Laurence, transformIterable is called many times, as in Hector's case.

You mentioned before it should be called only onze on each request, do
you still think having many calls is wrong?

Kees

[Laurence Rowe]

khink wrote

>
>> No, your selectors are not equivalent. The xpath "//*[contains(@class,
>> 'template-section-view')]" matches any element with
>> 'template-section-view'
>> in the class. That could be or equally
>>
>> The reason it is so slow is that every single rule you have is testing
>> that
>> there is any element in the page with that class. Those are body classes
>> and
>> you should be restrict them to the body element where they appear, i.e.:
>
> Thanks Hector and Laurence for this thread.
>
> We had the same problem, and re-writing our if-content selectors brought
> page load time down from 42 to 7 seconds.
>
> Laurence, transformIterable is called many times, as in Hector's case.
> Do you still think this is wrong?
>

Is this the time taken for the single request to fetch the HTML, or
including all CSS/JS/images etc? How long does an unthemed page load take?

transformIterable does need to be called every request as it makes the
decision whether to theme or not. Looking at it again, it seems to be doing
a bit too much work in that case (attempting to parse the response before
looking at the content type header). I'll fix that.

Laurence

transformIterable has to be called every request because it makes the
decision whether a response needs theming or not. Most of the time it just
passes

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7421023.html

Sent from the Installation, Setup, Upgrades mailing list archive at Nabble.com.

[Laurence Rowe]

Laurence Rowe wrote

I responded too soon, parseTree doesn't attempt to parse anything that is
not text/html so it's only unthemed html that gets parsed unnecessarily.

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7421037.html

[Héctor Velarde]

On 28/03/12 16:44, Ken Wasetis [Contextual Corp.] wrote:
> I'm just curious - while the example below is for a condition that is
> likely only checked once per page render, did you have some other
> 'if-content' conditions that are checked more frequently? Perhaps in
> iterations, even, such as across all links in portal-globalnav, all <li>
> tags within a navigation portlet, etc.? I would think that if there is
> some performance gain to be had from converting CSS3 to XPath selectors
> at the outset (in the rules.xml) that if you're seeing such significant
> performance gains it's more likely to be in rules that apply to many
> links/portlets/items on a page, so that it's more of a compounded
> problem/remedy.

the main rules file includes 5 or 6 additional rules files and the whole
thing is plenty of conditions... you know, the site was full of portlets
and things like that, and rules grew insanely during development and
deployment: now nobody knows where is what :-p

I'm not the guy who wrote it, but I'm thinking on changing the strategy
next time: we have now separate templates and rules files for frontpage,
sections, content types, etc.

I guess that it could be wiser to create rules for anonymous and logged
in users also, because problems are only evident for the latest.

best regards

[Héctor Velarde]

On 30/03/12 08:01, Laurence Rowe wrote:
> transformIterable does need to be called every request as it makes the
> decision whether to theme or not. Looking at it again, it seems to be doing
> a bit too much work in that case (attempting to parse the response before
> looking at the content type header). I'll fix that.

I think the problem is not there; we need to understand better how to
write good rules that create the fastest XSLT transformations possible.

that could be included in the documentation with some examples of good
and bad practices.

best regards

[Alan Runyan]

if you check collective.stats I added example of using stats on real world customer to find plone.app.theming cause 300% performance drag.

hector it would be great to complete this email thread with the before and after of your rules.xml files.

also always good to disable theme and hit views without theme to see delta. collective.stats is only way I know to get numbers efficiently. disable theme look at stats in firebug enable theme recheck firebug.

Alan Runyan
Enfold Systems
Sent from my iPhone

[Kees Hink]

> Is this the time taken for the single request to fetch the HTML, or
> including all CSS/JS/images etc?

Single page: in the instance Z2-log, i only see page hits.

> How long does an unthemed page load take?

Half of that - about 3.5s.

[Laurence Rowe]

Laurence Rowe wrote

Further to this, there's a bug in our fork of experiemental.cssselect (fixed
in newer versions of lxml after our fork) where spaces in CSS are not parsed
correctly causing the comma+space separated selectors to be compiled to
exceptionally slow XPaths. This should account for the ~30s timings you've
seen. I'll make another release in the next few days which will also
incorporate some of the simpler class selector optimizations.

There are a couple of options to fix the ~3s timings though these are rather
more involved and I can't promise when I'll get to those yet. I'd like to
make it difficult to write a theme that takes more than a few hundred ms to
execute on reasonably long pages.

It would be really helpful if people could publish their themes (rules.xml,
theme html and example content html, no need for CSS/images) in a way that
would let me incorporate them into a benchmark, particularly if they are
very slow.

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7426744.html

[Elaine Blakeman]

I have published my theme here:

https://github.com/emb03/plone.newbie.theming/downloads

Elaine

Elaine Blakeman
SSC Web and Multimedia Development
Ela...@ssc.ucla.edu
310-206-1524

sent from the Mars Rover

________________________________________
From: plone-set...@lists.plone.org [plone-set...@lists.plone.org] on behalf of Laurence Rowe [l...@lrowe.co.uk]
Sent: Sunday, April 01, 2012 6:07 AM
To: se...@lists.plone.org
Subject: Re: [Setup] huge performance issues using Diazo

[Laurence Rowe]

emb03 wrote

>
> I have published my theme here:
>
> https://github.com/emb03/plone.newbie.theming/downloads
>

This might just a Github problem, but I get a "File was not found" error
when I click to download the rules.xml.

Laurence

--
View this message in context: http://plone.293351.n2.nabble.com/huge-performance-issues-using-Diazo-tp7372056p7429388.html