Console feedback after a day and some hours of use.

[Jonathan Garbee]

First the effectiveness noticed before was spotty. It worked very well in some areas such as HTML minification and CSS/JS rewriting and optimization. The bad was images, it doesn't seem to ever want to touch images. For reasons I can't figure out.

Second, here is the full list from the console:

Notable issues (Click through for the links for info on how to fix these problems):

1. Cache misses: 66%
2. Image rewrite failures: 0.12%
3. Resources not loaded because of fetch failure: 0.00%
4. Resources not rewritten because domain wasn't authorized: 0.00%
5. Resources not rewritten because of restrictive Cache-Control headers: 0.00%
6. Cache lookups were expired: 0.00%
7. CSS files not rewritten because of parse errors: 0.00%
8. JavaScript minification failures: 0.00%
9. CSS combine opportunities missed: 0.00%

Third, I knew cache issues were a problem from looking into logs. However this seems mostly to be from the images that pagespeed doesn't seem to like working with me on. I'm clueless as to image rewrites, since that is actually the main issue I see it not doing yet it is reporting such a low failure rate.

My current debugging process for the cache has been to double check where it is being stored. That involves first checking the configuration to make sure it is right, then checking that folders permissions. It is owned by the apache user and php-fpm user, so it should be functional. Beyond that, I'm not sure why it wouldn't be writing to/pulling from the cache.

As far as debugging the image rewrite failures, I'm completely lost on how to debug that out. Judging from my error log if I can solve the cache issue then that should get solved as well. Finally for reference, here is an example of the error log " [Wed Jun 05 15:56:06 2013] [error] [mod_pagespeed 1.5.28.2-3084 @4294] Failed to stat /srv/web/etc/cache/pagespeed/rname/ic_tfmJjSHiymrVdpBINg_b/http,3A/,2Fwww.untamedworlds.com/uploaded/thumbnails/280_8556_180x250.jpg,40x,40_,55qdF2: No such file or directory "

Here is the requested information (perhaps a little more.) Please let me know if anymore information can be used at this stage and I'll get it as soon as I can.

[Shawn Ligocki]

Thanks Jonathan for the great write-up!

It does look like "Cache Misses" is a good place to start investigating here. Usually I would expect high cache misses to mean that either (a) your website was so large that most resources had not yet been stored in it or (b) your cache was too small and you are getting files evicted from it too often.

Are you getting a lot of these "Failed to stat" messages in your error log?

Is your cache directory being used? For example, what are the results of "du -s /srv/web/etc/cache/pagespeed/" and "find /srv/web/etc/cache/pagespeed/ -type f | wc -l"? What is your cache size set to? Any custom configuration? Can you post your config?

Also, does your console still look like this after several more days running?

Thanks,

-Shawn

[Jonathan Garbee]

I am getting 42 "Failed to stat" errors out of 749 pagespeed errors. So, not a lot in the grand scheme.

Command results are: du returned 95652 and find returned 8577.

Here is my entire pagespeed.conf: http://pastebin.com/7nBSKaDH .

The cache if I am looking at the right one should be the default of 102400 for files. No major custom configurations going on in pagespeed that I know of, just about everything is default except for the filters in use.

Current stats of console look the same :/. Sitting at:

1. Cache misses: 75%
2. Image rewrite failures: 0.17%

[Shawn Ligocki]

Aha, it looks like your cache is full. Can you increase the cache size, restart the server and report back if that helped?

[Jonathan Garbee]

Increased to 1024000 . It would be nice if a comment were added in explaining exactly what unit is being used for these fields, I first assumed (then dug around and found a reference to) KB, so I'm assuming now this is about 1GB of cache for files.

I will let you know Thursday or Friday what the results are, the site won't get much usage until those two days.

Cheers and let's hope this at least helps!

[Shawn Ligocki]

You mean ModPagespeedFileCacheSizeKb, right? The units are in the name :) Was that too subtle? Or is this a different issue?

[Jonathan Garbee]

No, I'm just blind! Thanks for pointing that out. (Long day here, new puppy still being trained. Please don't ask about the state of my mind right now.)

[Jonathan Garbee]

So, after the power going out for a while here are the results of about a week after increasing the cache:

1. Cache misses: 52%
2. Image rewrite failures: 0.03%
3. Cache lookups were expired: 0.03%
4. Resources not rewritten because domain wasn't authorized: 0.03%

Much better on the cache misses, although I'm considering increasing the allowed cache size even more. We have about 2GB of images which will be redone a few times each if the entire site gets hit multiple times, so might need to make it about 5GB since it is doing a jpeg and webp copy of each (even though optimized copies are much smaller than their original in some cases.)

[Daniel Bartholomew]

We struggle to right-size our caches as well.

We find sites with low cache hit ratios where the CPU gets smashed every cache purge, even when we have our cache set at 3GB. The documentation says that the cache clear evicts the oldest resources, not the least used resources - I was wondering if this might be a better solution?

Our strategy now is to just allocate a massive cache (have been thinking 100GB) because if the cache eviction happens to line up with a high traffic period then the users suffer.

--

Daniel Bartholomew

Technical Director & Founder

+61 488 215 220

dan...@squixa.com

[Shawn Ligocki]

Indeed this does look better, you might want to even go for 3-5x larger cache than data since we need to store the original image, and rewritten versions in the cache.

[Jonathan Garbee]

Um, wow. Going to a 9GB cache size has dropped the cache misses down tremendously. Here are the stats as of today:

1. Cache misses: 18%
2. Resources not rewritten because of restrictive Cache-Control headers: 1.16%
3. Resources not rewritten because domain wasn't authorized: 0.43%
4. Image rewrite failures: 0.09%

So, it looks like pagespeed never had an issue converting the images for me, it simply couldn't store them.

[Shawn Ligocki]

Great to see, hopefully that number will go down even further over time too. Ideally it should be almost 0 after the cache is warmed up.