Extreme high disk IO and CPU when enabled on a dedicated server running 3 magento stores

[Steve Blackburn]

Hi,

 I tried using mod_pagespeed on our dedicated server running 3 Magento stores and a few wordpress sites.

 When enabled the default settings end up causing a very high harddrive IO and eventually our server locks up and becomes inaccessible.

 It seems that it could possibly be whatever is constantly caching to the disk would be the cause of the extreme high IO.

 Our server has 8GB RAM, quad core intel processor, and a 7200RPM drive for applications and SSD for MySQL databases.

 I tried disabling several of the filters but that did not help.  I have currently disabled the plugin but would really like someone to help us take a look at our server and see if it has the capability to support your software. Maybe it's a matter of using RAM for cache instead of the drive.

 What is the next step to have someone help us out? I am so lost!

Thanks

Steve

 

[Joshua Marantz]

I think you've already diagnosed the problem.  Your query-load is too much to support on a hard disk cache.  You should use either tmpfs or memcached

https://developers.google.com/speed/pagespeed/module/system#memcached

https://developers.google.com/speed/pagespeed/module/faq#tmpfs

Hope this helps!

Also can you tell us what version number of mod_pagespeed you are using?

-Josh

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/7cfd058e-cc18-49e6-95b5-21d9e2910ec1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[magewes]

Hi Josh

I believe we had a very similar problem. 

However there was very low traffic to our site, (0-5 concurrent site visitors during the testing period). Could such low traffic and site activity still cause a problematic query load? 

Here are some more details of the problem (we use varnish downstream): 

It appears that immediately after mod_pagespeed was installed and the changes made to the default.vcl file on our site, we started seeing a huge spike in outbound traffic and server load, as seen in the graphs attached.

We have uninstalled mod_pagespeed and reverted the changes to Varnish, and the site is once again live. During this process, we noted that even after mod_pagespeed was uninstalled, the load remained high, indicating a potential problem in the Varnish ruleset provided at the following URL:https://developers.google.com/speed/pagespeed/module/downstream-caching

Thanks in advance. 

[Jeffrey Crowell]

It seems like your attachment was eaten by google groups/not attached. Can you reply with them again?

Also, what versions of mod_pagespeed, and varnish are your running, and do you see anything indicative of the problem in your apache error.log?

Jeff

[magewes]

Jeff

Thanks for the quick response - it is appreciated. Are you on the mod_pagespeed team at Google? 

I think you are onto something. 

Here is my attachments showing the spike in outbound traffic and server load: 

We were running:
- mod_pagespeed at it's latest stable version(https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_x86_64.rpm)
- Apache 2.2.15
- Varnish 3.0.7

There are some error logs relating to permissions on the caching directory:

"[Mon Aug 31 09:23:56 2015] [error] [mod_pagespeed 1.9.32.4-7251 @6031] Failed to make directory /var/cache/mod_pagespeed/[redacted]: Permission denied"
We will check this out before trying again. 

Based on the above, do you have other pointers/suggestions? 

We are on the Magento ecommerce platform, and utilising the very popular "Turpentine" extension to run Varnish. Have you had any experience with Turpentine & Mod_pagespeed?

Many thanks again in advance

Wes

[Jeff Kaufman]

"Failed to make directory /var/cache/mod_pagespeed/"

The user mod_pagespeed runs as needs to have full permissions over its cache.  Otherwise it will keep reoptimizing the same images over and over.  This could have caused your problem.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/23685d2a-0a80-4e86-a5ee-5e5f16dab2c9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.