mod_pagespeed killing entire server during "index"? It doesn't like Mondays...

[Carl Whalley]

This has taken ages to nail down, but I finally have the reason my server slows to a crawl on Mondays! The server hosts about a dozen sites, one of which runs Joomla 2.5 and has about 10 years worth of content over about 3000 pages (10k articles). The pattern was the whole site would crawl on Monday mornings, and inspection showed there were dozens of open MySQL connections to this big Joomla site, which meant the rest weren't getting a look in since they were also reliant on MySQL. Inspection of the apache log showed little traffic.So I disabled mod_pagespeed for just that site in it's .htaccess file and now all is well, although of course the site isn't optimized any more. 

So, does this pattern sound familiar and if so is there something smarter I can do with this site, such as ensure only 1 MySQL connection is made during "indexing" (if that the problem) or even stop caching but continue minifying? Thanks.

mod_pagespeed is 1.9.32.3-4448

[Jeff Kaufman]

mod_pagespeed doesn't have a scheduled "index" step, it optimizes
pages as they pass through. Do your resources (css / js / images)
rely on Joomla and/or MySQL, or is that only needed by your HTML? Do
you see any errors in the error log when this happens?

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mod-pagespeed-discuss/8a608f8e-a383-4bc3-873b-11f600a2540f%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Carl Whalley]

For this site, everything is served by Joomla which itself uses Mysql. There is no html which is not served by Joomla, and there's nothing unusual in the logs. The joomla pages all have pagination links, so its possible some crawler kept hitting "next", "next" etc through the site, but Apache seemed very quiet. There were no errors in the logs, but in the past when this happened there were MySQL timeout errors as if it had exhausted its connections limit. This was across all MySql, so visits to other sites which were normally unaffected, but powered by joomla, would fail.

> email to mod-pagespeed-discuss+unsub...@googlegroups.com.

[Jeff Kaufman]

Are your css, js, and images also served through Joomla, or are those
served statically?

>> > email to mod-pagespeed-di...@googlegroups.com.

>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/d/msgid/mod-pagespeed-discuss/8a608f8e-a383-4bc3-873b-11f600a2540f%40googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to mod-pagespeed-di...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/7ac6faa8-1a42-4fb9-b7de-a6ce6e56856d%40googlegroups.com.

[Carl Whalley]

If you like Jeff pm me and I'll give you the site url.

>> > email to mod-pagespeed-discuss+unsub...@googlegroups.com.

>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/d/msgid/mod-pagespeed-discuss/8a608f8e-a383-4bc3-873b-11f600a2540f%40googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to mod-pagespeed-discuss+unsub...@googlegroups.com.

[Carl Whalley]

I thought I'd better post an update to this since it is all working great now, and since it doesn't appear to be mod_pagespeed's fault there could be tips for anyone else experiencing it.

I can't definitively say what nailed it - you can see from what I wrote it was intermittent but fairly frequent. Since these changes, there hasn't even been a hiccup :-)

1. We were running Ubuntu 14 LTS and upgraded to Ubuntu 15 LTS

2. We had a version of the mod_pagespeed conf file which hadn't changed for years so brought that up to the current version (not many changes needed actually)

3. We *think* the ulimit on open system files was 1024. It's now much bigger.

4. There was a MySql max open file limit of 4096, this is now much higher.

If these file limits were the culprit we'd have hoped to have seen immediate problems manifest in other areas.

So we like Mondays again!

[Jeff Kaufman]

Thanks for the update!

>>> >> > email to mod-pagespeed-di...@googlegroups.com.

>>> >> > To view this discussion on the web visit
>>> >> >
>>> >> >
>>> >> > https://groups.google.com/d/msgid/mod-pagespeed-discuss/8a608f8e-a383-4bc3-873b-11f600a2540f%40googlegroups.com.
>>> >> > For more options, visit https://groups.google.com/d/optout.
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> > Groups
>>> > "mod-pagespeed-discuss" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> > an

>>> > email to mod-pagespeed-di...@googlegroups.com.

>>> > To view this discussion on the web visit
>>> >
>>> > https://groups.google.com/d/msgid/mod-pagespeed-discuss/7ac6faa8-1a42-4fb9-b7de-a6ce6e56856d%40googlegroups.com.
>>> >
>>> > For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to mod-pagespeed-di...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/1bcbfae0-2462-4128-acc3-3715668c8b92%40googlegroups.com.