Had to uninstall as too many httpd's processes were being forked

[IBSgroup]

Running Centos 5.5 with Apache 2.2.16, we found that when
mod_pagespeed was ON that the server went from the usual 20 httpd
processes to upwards of 225. This caused tremendous slowness in the
site. We installed the vanilla mod_pagespeed setup with only local
(domain) changes to the conf file.

Errors reported in logs/errors were likely the cause for the slowness:

[Thu Nov 04 14:18:09 2010] [error] [1104/141809:ERROR:net/instaweb/
util/google_message_handler.cc(35)] Could not create directories for
file /var/mod_pagespeed/cache/http,3A/,/www.ibsgroup.org/forums/public/
style_images/ipb23/feed.png,
[Thu Nov 04 14:18:09 2010] [error] [1104/141809:ERROR:net/instaweb/
util/google_message_handler.cc(54)] /var/mod_pagespeed/cache/http,3A/,/
www.ibsgroup.org/forums/public/style_images/ipb23/feed.png,DuCtuX:0:
open temp file (code=2 No such file or directory)
[Thu Nov 04 14:18:09 2010] [error] [1104/141809:ERROR:net/instaweb/
util/google_message_handler.cc(54)] /var/mod_pagespeed/cache/http,3A:
0: creating dir (code=13 Permission denied)
[Thu Nov 04 14:18:09 2010] [error] [1104/141809:ERROR:net/instaweb/
util/google_message_handler.cc(35)] Could not create directories for
file /var/mod_pagespeed/cache/http,3A/,/www.ibsgroup.org/forums/public/
style_images/ipb23/key.png,

[Joshua Marantz]

Please try changing the ModPagespeedFileCachePath setting in pagespeed.conf to a directory to which your Apache server has write access.

Thanks!

-Josh

[Bryan McQuade]

Can you specifically try /var/www/mod_pagespeed/cache and see if those
log messages go away? I encountered a similar issue on one machine and
switching to /var/www/mod_pagespeed/cache seemed to fix it. Would be
good to get confirmation from another user that this is a valid path.

[IBSgroup]

Changing ModPagespeedFileCachePath to a location that is writeable (/
var/www/mod_pagespeed) as well as changing the ownership of the
directory to be the same owner as the Apache server corrected the
error messages; however, although we start with 20 forked httpd
processes, the number of processes seems to grow very quickly from 20 -
> 50 -> 200+ and then back down again. Is this type of behavior
expected?

[Joshua Marantz]

Short answer:  Yes, this is not surprising.  

Longer answer, for the curious:

The first time mod_pagespeed rewrites an HTML page, it must fetch the resources from the origin, put them in the cache, and optimize them.  Image compression, in particular, is compute intensive.  But the optimized images are cached and so after an initial spike of CPU-bound activity, the server load should return to normal.

The origin Cache TTL (Time To Live) of the resources (CSS JS Images) determines the interval when the server must re-analyze the resources to see if they've changed and need to be re-optimized.

One of our tasks in the upcoming weeks is to minimize the CPU overhead for the HTML processing path and the latency through that path.  As part of this effort we will try to look at ways to reduce the load spike through the system as it starts up.  However, it's possible that this might be a one-time hit that you might not suffer again unless you flush your cache by deleting the directory.

[IBSgroup]

Thanks for the explanation. We are now seeing loads and process
levels for httpd that seem more acceptable.

Is there anyway to turn off the [warn] and [notice] entries that are
accumulating in the error_log?

I changed LogLevel to Error in httpd.conf, but this doesn't seem to
stop those entries from piling up. This might account for some of the
additional load.

[Joshua Marantz]

Yes; that's on our short list of issues to clean up.  Thanks for your perseverance and patience!

-Josh

[Shawn Ligocki]

BTW this is listed at http://code.google.com/p/modpagespeed/issues/detail?q=LogLevel&id=18

Thanks again for your patience,
-Shawn

[IBSgroup]

Thank you! I would have given up if you guys weren't so on top of
this.

J*

[Shawn Ligocki]

We believe we've fixed the problem, if you can gclient sync --force and build from source, let us know if we fixed this for you.

We will have a binary on Monday.

-Shawn