pagespeed cache issues

[Les Fenison]

I have been having very bad caching issues and I have determined that it is pagespeed that is doing it.

The console shows this..

Resources not rewritten because of restrictive Cache-Control headers: 37.80%

Cache lookups that were expired: 25.13%

Headers with pagespeed OFF looks like this...

Date: Wed, 06 May 2015 18:12:04 GMT

Server: Apache

Cache-Control: max-age=3600, must-revalidate, public

User-Cache-Control: max-age=3600

Set-Cookie: PHPSESSID=ivm9mn1tt39htvnla3hd4bfdt1; path=/

Set-Cookie: beercount2=1; expires=Thu, 05-May-2016 18:12:04 GMT; path=/

Cache-Control: max-age=43200, public, must-revalidate

Expires: Wed, 20 May 2015 18:12:04 GMT

Vary: Accept-Encoding

Content-Length: 19147

Content-Type: text/html

But with pagespeed ON, it looks like this on the same page

Date: Wed, 06 May 2015 18:12:45 GMT

Server: Apache

Cache-Control: max-age=3600, must-revalidate, public

User-Cache-Control: max-age=3600

Set-Cookie: PHPSESSID=8b2s3p11tg34r57647r7pstvj7; path=/

Set-Cookie: beercount2=1; expires=Thu, 05-May-2016 18:12:46 GMT; path=/

X-Mod-Pagespeed: 1.9.32.3-4448

Cache-Control: max-age=0, no-cache, must-revalidate

Vary: Accept-Encoding

Content-Length: 49126

Content-Type: text/html; charset=utf-8

[Jeff Kaufman]

The log messages you're showing are talking about resources
(css/js/images) but the headers you're showing are for the html.

Could you share a link to your site so I can look and see which
resources you have are the ones with restrictive cache control
headers?

Your existing Cache-Control headers are also funky. You have both:

Cache-Control: max-age=3600, must-revalidate, public
Cache-Control: max-age=43200, public, must-revalidate

Which one do you want? Also note that must-revalidate here means it's
effectively max-age=0 already because you don't provide an ETag or
Last-Modified header that would allow revalidation.

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mod-pagespeed-discuss/6f98a8a3-7d94-4426-96d6-777ca708b97c%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Les Fenison]

the URL is http://beerrecipes.org  or if you want the full url to the file it is http://beerrecipes.org/index.php  You can also try http://beerrecipes.org/beerstyles.php which has this problem alot.  

I do need to clean up some of those extra cache-control stuff.  I was puting it everywhere trying to get it to cache and went overboard..  I have been adding cache-control to my php code, .htaccess, and the virtual server config trying to get it to cache.

Also, it never caches .woff files.  Is this normal?

[Jeff Kaufman]

Looking at http://beerrecipes.org/beerstyles.php I see a good score
on PageSpeed Insights with no messages about poor caching headers, and
manually looking at the source I also don't see resources
(css/images/js) with restrictive cache control headers that would
block PageSpeed from optimizing. In fact, it looks like PageSpeed
successfully optimized all the resources there.

On the other hand, if you're getting that report in the console that
does mean there are resources on some page that do have this problem.
Can you think of other types of resources that are on other pages that
I should look at?

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/5e29fa5b-2498-4f9e-be16-7db7ec8f6a53%40googlegroups.com.

[Les Fenison]

I have my log level set to info and I have been watching the logs.   Maybe I just don't know what to look for.  I have about 12 sites that currently have mod_pagespeed enabled and it could be a different one.    What should I be looking for in the logs that would indicate this kind of problem?

This is what I was thinking was it..

[Wed May 06 11:30:30 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23200] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 76 seconds.

[Wed May 06 11:30:30 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23207] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 76 seconds.

[Wed May 06 11:30:38 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23247] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 68 seconds.

[Wed May 06 11:30:56 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23214] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 50 seconds.

[Wed May 06 11:35:30 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23226] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 298 seconds.

[Wed May 06 11:35:45 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23241] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 283 seconds.

[Wed May 06 11:35:45 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23246] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 283 seconds.

[Wed May 06 11:43:51 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23225] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 297 seconds.

[Wed May 06 11:43:59 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23229] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 289 seconds.

[Wed May 06 11:43:59 2015] [info] [mod_pagespeed 1.9.32.3-4448 @26581] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 289 seconds.

[Wed May 06 11:46:27 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23207] HTTPCache key=http://beerrecipes.org/beerstyles.php fragment=beerrecipes.org: remembering not-cacheable status for 141 seconds.

[Jeff Kaufman]

On Wed, May 6, 2015 at 2:51 PM, Les Fenison <lfen...@gmail.com> wrote:
> [Wed May 06 11:30:30 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23200]
> HTTPCache key=http://beerrecipes.org/beerstyles.php
> fragment=beerrecipes.org: remembering not-cacheable status for 76 seconds.
> [Wed May 06 11:30:30 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23207]
> HTTPCache key=http://beerrecipes.org/beerstyles.php

> ...

This info message doesn't represent a problem. Those are HTML files,
and PageSpeed won't cache html even if cache control headers allow it.
In your case they don't allow it, because of how you have
must-revalidate set, but so few sites have truly cacheable html that
PageSpeed doesn't cache input html.

This probably explains your 37.8% unrewritten resources stat, though.
I suspect that statistic is getting bumped each time we emit that log
message, while the actually cacheable resources are staying in cache
and so only get counted once, when we first cache them. I'd have to
look at the code to be sure, but I don't find that stat alarmingly
high anymore. I think your site is doing very well.

>
>
> On Wednesday, May 6, 2015 at 11:40:43 AM UTC-7, Jeff Kaufman wrote:
>>
>> Looking at http://beerrecipes.org/beerstyles.php I see a good score
>> on PageSpeed Insights with no messages about poor caching headers, and
>> manually looking at the source I also don't see resources
>> (css/images/js) with restrictive cache control headers that would
>> block PageSpeed from optimizing. In fact, it looks like PageSpeed
>> successfully optimized all the resources there.
>>
>> On the other hand, if you're getting that report in the console that
>> does mean there are resources on some page that do have this problem.
>> Can you think of other types of resources that are on other pages that
>> I should look at?
>>

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/aeadb848-5cf0-48d5-ab9e-995830564f5e%40googlegroups.com.

[Les Fenison]

I haven't been able to get .woff font files to cache.  These are local files and my log is full of them.  

[Wed May 06 12:08:02 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23210] HTTPCache key=http://beerrecipes.org/foundation_icons/foundation-icons.woff fragment=beerrecipes.org: remembering not-cacheable status for 244 seconds.

[Wed May 06 12:08:36 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23241] HTTPCache key=http://beerrecipes.org/foundation_icons/foundation-icons.woff fragment=beerrecipes.org: remembering not-cacheable status for 210 seconds.

So what should I be looking for in the log that would indicate this kind of problem?   

I am trying to figure out what is making my cache go cold.  I get a great pagespeed report then a few hours later I go back and I am in the 70s until I run the report a few times and then I move back to the 90s.   It seems that once I have everything right it should stay that way but it doesn't.

[Jeff Kaufman]

On Wed, May 6, 2015 at 3:20 PM, Les Fenison <lfen...@gmail.com> wrote:
> [Wed May 06 12:08:02 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23210]
> HTTPCache key=http://beerrecipes.org/foundation_icons/foundation-icons.woff
> fragment=beerrecipes.org: remembering not-cacheable status for 244 seconds.

This is a slightly inaccurate info message. The caching headers on
your woff files are fine, the message is just telling you that
pagespeed doesn't cache woff files (and it doesn't know what they are
until it fetches them).

This shouldn't be a problem, since pagespeed doesn't optimize woff
files either and it only needs to cache the kinds of files it can
optimize.

> I am trying to figure out what is making my cache go cold. I get a great
> pagespeed report then a few hours later I go back and I am in the 70s until
> I run the report a few times and then I move back to the 90s. It seems
> that once I have everything right it should stay that way but it doesn't.
>

When it goes into the 70s, could you expand all the sections and take
a screenshot? There are many things it could be objecting to, and
this could suggest where to look to fix things.

>
> On Wednesday, May 6, 2015 at 12:02:05 PM UTC-7, Jeff Kaufman wrote:
>>
>> On Wed, May 6, 2015 at 2:51 PM, Les Fenison <lfen...@gmail.com> wrote:
>> > [Wed May 06 11:30:30 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23200]
>> > HTTPCache key=http://beerrecipes.org/beerstyles.php
>> > fragment=beerrecipes.org: remembering not-cacheable status for 76
>> > seconds.
>> > [Wed May 06 11:30:30 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23207]
>> > HTTPCache key=http://beerrecipes.org/beerstyles.php
>> > ...
>>
>> This info message doesn't represent a problem. Those are HTML files,
>> and PageSpeed won't cache html even if cache control headers allow it.
>> In your case they don't allow it, because of how you have
>> must-revalidate set, but so few sites have truly cacheable html that
>> PageSpeed doesn't cache input html.
>>
>> This probably explains your 37.8% unrewritten resources stat, though.
>> I suspect that statistic is getting bumped each time we emit that log
>> message, while the actually cacheable resources are staying in cache
>> and so only get counted once, when we first cache them. I'd have to
>> look at the code to be sure, but I don't find that stat alarmingly
>> high anymore. I think your site is doing very well.
>>

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/d5097237-7442-454e-8b33-fba077e52416%40googlegroups.com.

[Les Fenison]

It didn't take long to have this happen again.  Here is one example.  The only section that was having a problem was "

Eliminate render-blocking JavaScript and CSS in above-the-fold content"

http://test.themastersapartments.com/css,_foundation.min.css+foundation_icons,_foundation-icons.min.css+css,_app.min.css+slick,_slick.css+slick,_slick-theme.css.pagespeed.cc.BCAcuAuA2l.css

Often it is the optimize images where every image on the page needs to be reoptimized even though it was optimized perfectly an hour before.   The cache always goes cold within a few hours.   

On Thursday, May 7, 2015 at 5:07:41 AM UTC-7, Jeff Kaufman wrote:

On Wed, May 6, 2015 at 3:20 PM, Les Fenison <lfen...@gmail.com> wrote:
> [Wed May 06 12:08:02 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23210]
> HTTPCache key=http://beerrecipes.org/foundation_icons/foundation-icons.woff
> fragment=beerrecipes.org: remembering not-cacheable status for 244 seconds.

[Jeff Kaufman]

On Thu, May 7, 2015 at 2:55 PM, Les Fenison <lfen...@gmail.com> wrote:
> http://test.themastersapartments.com/css,_foundation.min.css+foundation_icons,_foundation-icons.min.css+css,_app.min.css+slick,_slick.css+slick,_slick-theme.css.pagespeed.cc.BCAcuAuA2l.css
>

This isn't a caching issue. That's a file produced by PageSpeed with
a completely warm cache, and repeated loads of
https://developers.google.com/speed/pagespeed/insights/?url=http%3A%2F%2Ftest.themastersapartments.com%2F
show that error every time.

(Insights reports that page as 88 on mobile and 95 on desktop, and
still shows that warning "render blocking css" warning.)

Do you have any examples of insights reporting files that either have
short cache lifetimes or files that should be optimized but aren't?

> On Thursday, May 7, 2015 at 5:07:41 AM UTC-7, Jeff Kaufman wrote:
>>
>> On Wed, May 6, 2015 at 3:20 PM, Les Fenison <lfen...@gmail.com> wrote:
>> > [Wed May 06 12:08:02 2015] [info] [mod_pagespeed 1.9.32.3-4448 @23210]
>> > HTTPCache
>> > key=http://beerrecipes.org/foundation_icons/foundation-icons.woff
>> > fragment=beerrecipes.org: remembering not-cacheable status for 244
>> > seconds.
>>
>>
>> > I am trying to figure out what is making my cache go cold. I get a
>> > great
>> > pagespeed report then a few hours later I go back and I am in the 70s
>> > until
>> > I run the report a few times and then I move back to the 90s. It seems
>> > that once I have everything right it should stay that way but it
>> > doesn't.
>> >
>>
>> When it goes into the 70s, could you expand all the sections and take
>> a screenshot? There are many things it could be objecting to, and
>> this could suggest where to look to fix things.
>>
>>
>>

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/965bc6bf-ac20-46e2-bb4a-7c984d11cf30%40googlegroups.com.

[Les Fenison]

Yes, here are two examples.

This first one shows a problem that is always a problem and that is anytime I include anything from Google such as maps, adsense, analytics, or a CDN, it hurts my score.  In this case tap targets because google maps has 3 links that are too close together which I have no control over.   

The Javascript and CSS were not an issue an hour before this but now they are.  The cache was warm and went cold.

The minify javascript and leverage browser caching I assume there is no way to fix since it is the google maps.

Also in this example, there is nothing about optimize images but if I wait another few hours, it will tell me to optimize images but it will be for the google map.  How can it be caching a compressed image one moment and not the next?

Then scroll to the bottom of this example for the next.

84 / 100Speed

Consider Fixing:

Reduce server response time

In our test, your server responded in 0.77 seconds. There are many factors that can slow down your server response time.Please read our recommendations to learn how you can monitor and measure where your server is spending the most time.

Hide details

Eliminate render-blocking JavaScript and CSS in above-the-fold content

Your page has 1 blocking CSS resources. This causes a delay in rendering your page.

None of the above-the-fold content on your page could be rendered without waiting for the following resources to load. Try to defer or asynchronously load blocking resources, or inline the critical portions of those resources directly in the HTML.

Optimize CSS Delivery of the following:

• http://test.oakglenapts.com/…1S5eiDuE.css.pagespeed.cf.zLsLUb-Adz.css

Hide details

Leverage browser caching

Setting an expiry date or a maximum age in the HTTP headers for static resources instructs the browser to load previously downloaded resources from local disk rather than over the network.

Leverage browser caching for the following cacheable resources:

• https://maps.googleapis.com/…,search&v=3.exp&language=en_US&region=us (30 minutes)

Hide details

Minify JavaScript

Compacting JavaScript code can save many bytes of data and speed up downloading, parsing, and execution time.

Minify JavaScript for the following resources to reduce their size by 817B (2% reduction).

• Minifying https://maps.gstatic.com/mapfiles/embed/20/10/init_embed.js could save 817B (2% reduction) after compression.

Hide details

6 Passed Rules

Show details

Download optimized image, JavaScript, and CSS resources for this page.

99 / 100User Experience

Consider Fixing:

Size tap targets appropriately

Some of the links/buttons on your webpage may be too small for a user to easily tap on a touchscreen. Considermaking these tap targets larger to provide a better user experience.

The following tap targets are close to other nearby tap targets and may need additional spacing around them.

• The tap target <a href="https://www.go…erms_maps.html">Terms of Use</a> is close to 1 other tap targets.
• The tap target <a href="//accounts.goo…uccess?pb=!1e2">Sign in</a> is close to 1 other tap targets.
• The tap target <div></div> and 1 others are close to other tap targets.

Hide details

In this next example,  we have many sites like this.   Where local images were optimized and cached and now they are not.   But after running the pagespeed insights a couple of time they are optimized again... for a while, then the cache goes cold.

85 / 100Speed

Consider Fixing:

Eliminate render-blocking JavaScript and CSS in above-the-fold content

Show how to fix

Reduce server response time

Show how to fix

Leverage browser caching

Setting an expiry date or a maximum age in the HTTP headers for static resources instructs the browser to load previously downloaded resources from local disk rather than over the network.

Leverage browser caching for the following cacheable resources:

• http://temper.com/images/pnw_300.jpg (10 minutes)
• http://temper.com/images/temper-logo.gif (10 minutes)

Hide details

Optimize images

Properly formatting and compressing images can save many bytes of data.

Optimize the following images to reduce their size by 1KiB (10% reduction).

• Losslessly compressing http://temper.com/images/temper-logo.gif could save 1KiB (10% reduction).

Hide details

[Jeff Kaufman]

As Josh wrote in response to your previous email, the PageSpeed
Insights complaints about Google Maps are not things mod_pagespeed can
fix automatically for you, because the changes need to happen on the
servers controlled by Maps.

As I wrote in my previous email, the "Your page has 1 blocking CSS
resources" error is also not a caching issue. This error may be fixed
by turning on prioritize_critical_css, or it may be that mod_pagespeed
can't fix this automatically for your page, but I still get that error
testing with a warm cache.

The two issues with temper.com do look like potential caching issues,
though. Did you warm the cache first, let it cool off, and then run
that test? Or is that test from an initial run before the cache was
warmed? Are those issues from running Insights directly against
temper.com or against some subpage?

One possibility is that mod_pagespeed's cache is too small for all the
content on your many sites, and you should increase it:

ModPagespeedFileCacheSizeKb SIZE

Where SIZE is about twice the size of all the content on all your
sites running PageSpeed on this machine.

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/0efdeff1-51a9-4d4c-beec-9041811f74f6%40googlegroups.com.

[Les Fenison]

Ok on the google maps thing.  That makes sense.

As for the file cache size.  I don't think it matters as I am using 2 memcached servers, however I have specified a cache size and here are my global settings for both.

ModPagespeedMemcachedServers 10.1.2.34:11211,10.1.2.33:11211

ModPagespeedDefaultSharedMemoryCacheKB 5000000

ModPagespeedCreateSharedMemoryMetadataCache "/var/cache/mod_pagespeed/" 5120000

ModPagespeedFileCacheSizeKb was commented out..  I didn't think this mattered anymore since using memcached

The memcached file size is huge.   4gb per server for a total of 8gb.   with 84% free.

On the temper.com site.  I believe I did allow the cache to cool off first.  That is my biggest misunderstanding issue with pagespeed is why a cache will cool off when all the caching headers specify an infinite amount of time and the cool off time is a matter of hours

[Jeff Kaufman]

With the temper.com example, was that on the main page or a subpage?
What url should I be using if I'm trying to reproduce this?

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/dedf8fd6-38b3-453e-a8a8-5693fa52000b%40googlegroups.com.

[Les Fenison]

The temper site isn't bad and is improving daily it seems.. It use to go down to 75 when the cache was cold and warm up to 98 or 99 for mobile.    Now, it drops to 80 when cold and 99 when warm.   The strange thing is that the  

render-blocking JavaScript and CSS keeps coming back when the cache is cold and goes away once it is warm.  The url was just temper.com the main page.

What did you think of my cache settings?  Was I doing that right?  Could it be better?