mod_pagespeed does not seem to run on first page load

[Keith Power]

I have just installed mod_pagespeed onto Apache 2.2.7. Website is http://mckconstruction.ie

The setting as follows in pagespeed.conf

<IfModule pagespeed_module>

    ModPageSpeed on

    ModPagespeedRewriteLevel CoreFilters

    ModPagespeedEnableFilters prioritize_critical_css

    ModPagespeedEnableFilters defer_javascript

    ModPagespeedEnableFilters sprite_images

    ModPagespeedEnableFilters convert_png_to_jpeg,convert_jpeg_to_webp

    ModPagespeedEnableFilters collapse_whitespace,remove_comments

</IfModule>

When I go to the website first time the page loads without compression, caching etc. I have tested this on multiple page speed testing service.  But if I refresh the page mod_pagespeed seems to work and the files get appended with the css/A.main.css.pagespeed.cf.sscqXtBd9o.css

https://developers.google.com/speed/pagespeed/insights/ I get the score 65 but if I do it a second time I get 85.

Also on http://www.webpagetest.org/

				Document Complete			Fully Loaded
	Load Time	First Byte	Start Render	DOM Elements	Time	Requests	Bytes In	Time	Requests	Bytes In
First View	1.575s	0.363s	0.892s	180	1.575s	13	286 KB	1.707s	14	287 KB
Repeat View	0.656s	0.307s	0.658s	180	0.656s	1	15 KB	0.656s	1	15 KB

 

[Bernard Gardner]

Hi Keith,

That’s expected behaviour, see the following documentation for more information and details on how to tune it.

https://developers.google.com/speed/pagespeed/module/system#rewrite_deadline

https://developers.google.com/speed/pagespeed/module/system#ipro_deadline

Bernard.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/ee0e9600-9bb3-48f6-a76b-dcda3206d935%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Keith Power]

Thanks Bernard for the links. I have looked over them but sorry I am still unsure what to change.

Am I right in saying that every page load should be compressed and not just when the page gets refreshed? Surely it should every time. And if so what do I change to make sure, I don't see what I need to do in the docs.

Yesterday I was getting 90/100. Pagespeed is very inconsistent but not sure what I need to change.

80 / 100Suggestions Summary - Desktop

Should Fix:

Enable compression

Show how to fix

Consider Fixing:

Eliminate render-blocking JavaScript and CSS in above-the-fold content

Show how to fix

Leverage browser caching

Show how to fix

Optimize images

Show how to fix

Minify CSS

Show how to fix

[Keith Power]

Sorry just another note. I have read a few more posts with a similar issue.

Am I right to say that when the first user visits the website mod_pagespeed does not compress and cache because of latency. Then it does the work afterwards so every other user after that will see the compressed and cached version?

For me every time I refresh the page on any device or browser it seems random if it compresses and caches all the files.

When I do PageSpeed Insights it says no compression or caching for the mobile section and all fine for the desktop. Does anyone know why that might be? Also every time I run PageSpeed Insights I get different results?

Thanks

[Jan-Willem Maessen]

Hi Keith -

I'd suggest looking at http://yoursite/mod_pagespeed_statistics and taking a look at your cache statistics.  If cache_hits is low and cache_misses is high, there may be a few causes:

* If cache_hits is near 0 it's likely your cache is configured improperly (permission problems are the likeliest culprit, particularly if you're running some form of SELinux).  Take a look at error_log and you should see a lot of log messages about this.

* If your resources have short expiration times, it might be that PageSpeed is forced to drop entries from its cache before they're re-used.  In general you want expiration times that are at least a bit longer than the interval between page accesses, and a good bit longer than that is better still.  This always balances against the cost of pushing updates; with longer cache times you might need to use the cache flush interface when you make changes to the css, js, or images on your site.

* If cache_hits is much smaller than cache_misses but everything has long expiration times, it's likely your cache is too small.  You should ideally allocate enough cache space to hold the optimized contents of your site.

-Jan

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/a2cf9752-305b-470b-9130-3db6478a9bad%40googlegroups.com.

[Keith Power]

Hi Jan,

I was thinking that it might be the cache size. I am running about 20 websites from this VPS and one of those is a Mangento install. So to solve this I set mod_pagespeed to only run for one domain. As soon as I did a reboot I saw much improved results.

However it was still inconsistent after 10 seconds when I load the page there is no compression or cache headers again. Then it is OK as I click through the pages. I can understand holding the cache for 10 secs but compression and headers should always be on.

I decided to do a test and disable mod_pagespeed and turn on mod_deflate and header setting from the .htaccess file. Now I have gone from a jumping up and down pagespeed of between 60 and 80 to a consistent 99. No matter how many times I run the tests now. I have used other speed test services to confirm.

**Note: I manually minified the files and optimised images, I know pagespeed is better as it automats all this.

I have also noticed that when I turn back on mod_pagespeed it overrides the .htaccess and the compression becomes inconsistent again.

[Jan-Willem Maessen]

So you're actually seeing gzip / mod_deflate turning on and off inconsistently with PageSpeed enabled?  That's *very* strange, and something is definitely amiss.  Are you losing compression only on html requests, or on css and js as well?  (Images generally rely on their own compression and don't use gzip on top of it.)

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/0cf4c0de-e42a-439a-8bc6-eb2004398d62%40googlegroups.com.

[Keith Power]

It happens on both js and css files. It is usually one or the other. The domain I have it on at the moment is http://hotelieracademy.com

Here is a check I have done on pagespeed insights and one I did straight after on http://gtmetrix.com/

69 / 100Speed

Should Fix:

Eliminate render-blocking JavaScript and CSS in above-the-fold content

Show how to fix

Enable compression

Compressing resources with gzip or deflate can reduce the number of bytes sent over the network.

Enable compression for the following resources to reduce their transfer size by 118.3KiB (82% reduction).

• Compressing http://www.hotelieracademy.com/css/style.min.css could save 118.3KiB (82% reduction).

Hide details

Enable gzip compression	B (80)	83%	Server	High
What does this mean? Compressing the following resources with gzip could reduce their transfer size by 54.0KiB (73% reduction). Compressing http://www.hotelieracademy.com/js/main.min.js could save 54.0KiB (73% reduction).

[Keith Power]

Sorry I should say that I have set to PassThrough at the moment to focus on the compression side. But say issue set to core also.

[Keith Power]

Just did another test today to track it over a couple of days and now PageSpeed Insights says that style.min.css is not compressed. I have made no changes to the code. PassThrough is still set.

66 / 100Speed

Should Fix:

Eliminate render-blocking JavaScript and CSS in above-the-fold content

Show how to fix

Enable compression

Compressing resources with gzip or deflate can reduce the number of bytes sent over the network.

Enable compression for the following resources to reduce their transfer size by 172.3KiB (79% reduction).

• Compressing http://www.hotelieracademy.com/css/style.min.css could save 118.3KiB (82% reduction).

• Compressing http://www.hotelieracademy.com/js/main.min.js could save 54KiB (73% reduction).

Hide details

[Jan-Willem Maessen]

Thanks, the fact that this is happening in PassThrough is of particular interest (and particularly baffling).  That's definitely a helpful fact to know.

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/b87b5752-f40b-434a-8660-bba82d3bb543%40googlegroups.com.

[Jan-Willem Maessen]

We've reproduced this (or something very like it) locally.  Thanks for going back and forth with us a couple of times, that was helpful in understanding the symptoms.  I've opened https://code.google.com/p/modpagespeed/issues/detail?id=1049 on this.

One thing that came up in my experiments was that I could play with my DEFLATE filter settings and get different behavior.  I haven't been able to make gzip *go away* when I've done so, but I have a hypothesis:

Sometimes the content-type seen by mod_pagespeed is different from the content-type that ultimately appears on the wire when Apache serves information (it can get altered by downstream filters and the like).  If that's happening here, it's possible we're failing to correctly leave the output filters alone.  I'm otherwise baffled why this would happen.

If this is true, you may be able to work around the problem by adding an additional DEFLATE filter setting like:

    AddOutputFilterByType DEFLATE text/javascript

If, say, text/javascript early in the filter flow was being transformed to application/javascript later on.

-Jan

[Keith Power]

Hi Jan,

Thanks also for being so engaging. One thing to note is that I see the same intermittant results running the core filters also. The last four days I have updated the .htaccess to compress the following files with no change. The deflate works perfectly if I disable mod_pagespeed with the .htaccess.

I am using CakePHP 2.2 not that it matters so much. I have seen the results on another website I am running on the VPS.

<IfModule mod_deflate.c>

    # Force compression for mangled `Accept-Encoding` request headers

    # https://developer.yahoo.com/blogs/ydn/pushing-beyond-gzipping-25601.html

    <IfModule mod_setenvif.c>

        <IfModule mod_headers.c>

            SetEnvIfNoCase ^(Accept-EncodXng|X-cept-Encoding|X{15}|~{15}|-{15})$ ^((gzip|deflate)\s*,?\s*)+|[X~-]{4,13}$ HAVE_Accept-Encoding

            RequestHeader append Accept-Encoding "gzip,deflate" env=HAVE_Accept-Encoding

        </IfModule>

    </IfModule>

    <IfModule mod_filter.c>

        AddOutputFilterByType DEFLATE "application/atom+xml" \

                                      "application/javascript" \

                                      "application/json" \

                                      "application/ld+json" \

                                      "application/manifest+json" \

                                      "application/rdf+xml" \

                                      "application/rss+xml" \

                                      "application/schema+json" \

                                      "application/vnd.geo+json" \

                                      "application/vnd.ms-fontobject" \

                                      "application/x-font-ttf" \

                                      "application/font-woff" \

                                      "application/x-font-otf" \

                                      "application/x-javascript" \

                                      "application/x-web-app-manifest+json" \

                                      "application/xhtml+xml" \

                                      "application/xml" \

                                      "font/eot" \

                                      "font/opentype" \

                                      "image/bmp" \

                                      "image/svg+xml" \

                                      "image/vnd.microsoft.icon" \

                                      "image/x-icon" \

                                      "text/cache-manifest" \

                                      "text/css" \

                                      "text/html" \

                                      "text/javascript" \

                                      "text/plain" \

                                      "text/vcard" \

                                      "text/vnd.rim.location.xloc" \

                                      "text/vtt" \

                                      "text/x-component" \

                                      "text/x-cross-domain-policy" \

                                      "text/xml"

    </IfModule>

Thanks

[Jan-Willem Maessen]

That looks like a pretty comprehensive set of content types, and includes text/plain, so I'd be really surprised if it's a change in content-type (and I can't think of any clever way for me to test this one way or the other).

The support for header mangling makes me suspicious: are you seeing a lot of mangled accept-encoding headers at the server?  Is there any way to tell if an incoming request has a mangled accept header?  I don't believe mod_pagespeed specifically accounts for such header mangling.  On the other hand, I still don't see how that would *prevent* downstream mod_deflate from correctly gzipping our results anyway.

You're only seeing these on js and css?  Or are you seeing failure to compress for html as well?  (When I looked at your site with wget similarly to the bug I filed, I only saw problems with style.min.css .)

-Jan

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/b7d8a992-1f02-4588-a5ae-0c6ebe0cc775%40googlegroups.com.

[Keith Power]

It mainly seems to be the .js compression that shows up in the different speed tests most often. I did another PageSpeed Insights website test again and the first test said that no cache header was set on the css and js files. 30 seconds latter I ran the test again and the main.min.js showed up as not being compressed.

I ran the test then in PageSpeed chrome extension and it also highlighted the main.min.js compression.

Running the css through redbot.org to check the header the first time it was gZipped but the second time one minute latter I got following. One other thing to note is the Etag is there. In my .htaccess I actually remove it. When the files does get gZipped it then has the Etag removed. The problem also seems to interfere with my .htaccess.

style.min.css: No gZip, Etag not removed and no PageSpeed entry

    HTTP/1.1 200 OK
    Date: Mon, 23 Feb 2015 23:27:31 GMT
    Server: Apache
    Content-Length: 146722
    Accept-Ranges: bytes
    Cache-Control: max-age=31536000
    Expires: Tue, 23 Feb 2016 23:00:22 GMT
    Vary: Accept-Encoding
    Etag: W/"PSA-vN7Xa1Tttv"
    Last-Modified: Mon, 23 Feb 2015 23:27:31 GMT
    X-Content-Type-Options: nosniff
    Keep-Alive: timeout=1, max=100
    Connection: Keep-Alive
    Content-Type: text/css

main.min.js: can take many tries to get a response with this file only, most tries get no response.

    HTTP/1.1 200 OK
    Date: Mon, 23 Feb 2015 23:12:38 GMT
    Server: Apache
    Content-Length: 75660
    Accept-Ranges: bytes
    Cache-Control: max-age=31536000
    Expires: Tue, 23 Feb 2016 23:01:33 GMT
    Vary: Accept-Encoding
    Etag: W/"PSA-CbloUJIigV"
    Last-Modified: Mon, 23 Feb 2015 23:12:38 GMT
    X-Content-Type-Options: nosniff
    Keep-Alive: timeout=1, max=100
    Connection: Keep-Alive
    Content-Type: application/javascript

Every other file type gives the expected response.

index.php

 HTTP/1.1 200 OK
    Date: Mon, 23 Feb 2015 23:33:50 GMT
    Server: Apache
    Vary: Accept-Encoding
    X-Mod-Pagespeed: 1.9.32.3-4448
    Cache-Control: max-age=0, no-cache
    Content-Encoding: gzip
    Content-Length: 7862
    Keep-Alive: timeout=1, max=100
    Connection: Keep-Alive
    Content-Type: text/html; charset=UTF-8

font-awesome.svg

  HTTP/1.1 200 OK
    Date: Mon, 23 Feb 2015 23:35:32 GMT
    Server: Apache
    Last-Modified: Fri, 13 Feb 2015 13:19:30 GMT
    Accept-Ranges: bytes
    Cache-Control: max-age=2592000
    Expires: Wed, 25 Mar 2015 23:35:32 GMT
    Vary: Accept-Encoding
    Content-Encoding: gzip
    Keep-Alive: timeout=1, max=100
    Connection: Keep-Alive
    Transfer-Encoding: chunked
    Content-Type: image/svg+xml

Conclusion:

With PageSpeed on, css and js only seem to be affected. All other files behave as expected.

[Keith Power]

I have just turned mod_pagespeed off and straight away I get the following for main.min.js

Note also the Etag is removed.

style.min.css

   HTTP/1.1 200 OK
    Date: Mon, 23 Feb 2015 23:56:07 GMT
    Server: Apache
    Last-Modified: Sun, 22 Feb 2015 15:56:17 GMT
    Accept-Ranges: bytes
    Cache-Control: max-age=31536000
    Expires: Tue, 23 Feb 2016 23:56:07 GMT
    Vary: Accept-Encoding
    Content-Encoding: gzip
    Content-Length: 25557

    Keep-Alive: timeout=1, max=100
    Connection: Keep-Alive
    Content-Type: text/css

main.min.js

HTTP/1.1 200 OK
    Date: Tue, 24 Feb 2015 01:51:47 GMT
    Server: Apache
    Last-Modified: Sun, 22 Feb 2015 15:20:46 GMT
    Accept-Ranges: bytes
    Cache-Control: max-age=31536000
    Expires: Wed, 24 Feb 2016 01:51:47 GMT
    Vary: Accept-Encoding
    Content-Encoding: gzip
    Content-Length: 20414

[Joshua Marantz]

Sorry I'm late to this thread -- just got back from vaca.  I'm not sure I understand yet if there's a bug here, but I wanted to chime in on a couple of points.

1. mod_pagespeed is not an alternative to mod_deflate.  Leave all your mod_deflate configuration in even when mod_pagespeed is on.  If a configuration does not turn on mod_deflate itself, then MPS will only deflate the files that it actually optimizes (which is 100% of the HTML, but the resources will only be served optimized on cache hits).
   
2. It looks like you are trying to get optimum PageSpeed Insights score when the server-side cache is cold.  I want to know why that's a goal for you.  Once you deploy your site, if you make your cache TTL for resources 10 minutes, and you get at least a couple of queries per minute, the server-side cache will never be cold (it auto-freshens cache entries at 80% of TTL).  Measuring PageSpeed Insights score with a cold server cache should only matter for sites that are rarely visited.

Having said that, it seems like there's some confusion about whether MPS is really delivering consistently optimized results even when the interval between requests is much smaller than the cache TTL.  Can this be clarified?

Thanks,

-Josh

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/ac473e54-d66d-404e-95c8-200bdcf49def%40googlegroups.com.

[Jan-Willem Maessen]

On Tue, Feb 24, 2015 at 9:42 AM, 'Joshua Marantz' via mod-pagespeed-discuss <mod-pagesp...@googlegroups.com> wrote:

Sorry I'm late to this thread -- just got back from vaca.  I'm not sure I understand yet if there's a bug here, but I wanted to chime in on a couple of points.

1. mod_pagespeed is not an alternative to mod_deflate.  Leave all your mod_deflate configuration in even when mod_pagespeed is on.  If a configuration does not turn on mod_deflate itself, then MPS will only deflate the files that it actually optimizes (which is 100% of the HTML, but the resources will only be served optimized on cache hits).
   
2. It looks like you are trying to get optimum PageSpeed Insights score when the server-side cache is cold.  I want to know why that's a goal for you.  Once you deploy your site, if you make your cache TTL for resources 10 minutes, and you get at least a couple of queries per minute, the server-side cache will never be cold (it auto-freshens cache entries at 80% of TTL).  Measuring PageSpeed Insights score with a cold server cache should only matter for sites that are rarely visited.

Having said that, it seems like there's some confusion about whether MPS is really delivering consistently optimized results even when the interval between requests is much smaller than the cache TTL.  Can this be clarified?

The big thing to note here is that even in high-intensity testing in PassThrough mode I was seeing style.min.css arrive uncompressed consistently every single time.  That seems problematic.

One thing I'll note here is that the ETags being sent (thanks for the header data) indicate in-place resource optimization is at least being attempted.  JeffTK was suspicious that the logic here may have missed an edge case, but we haven't reproduced the problem locally if so.  So far we've only succeeded in reproducing cases where files *won't* be compressed when mod_pagespeed is unplugged.

-Jan

 

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAGKR%2BEBAe%3D_Q0dotn_vGo9rg8C7K0WtOVSdSu%3D8z%3D1Xt%3DEQXkg%40mail.gmail.com.

[Jeff Kaufman]

Another weird thing in the headers; I see above for "style.min.css: No
gZip, Etag not removed and no PageSpeed entry":

Cache-Control: max-age=31536000

Expires: Tue, 23 Feb 2016 23:00:22 GMT
Vary: Accept-Encoding
Etag: W/"PSA-vN7Xa1Tttv"

And for "main.min.js: can take many tries to get a response with this

file only, most tries get no response."

Cache-Control: max-age=31536000

Expires: Tue, 23 Feb 2016 23:01:33 GMT
Vary: Accept-Encoding
Etag: W/"PSA-CbloUJIigV"

These two don't look like anything we should ever emit. Those our our
etags ("PSA") and they're not pagespeed resources, which means that
must have been put there by IPRO. But IPRO should never be serving
anything with a Cache-Control: max-age=31536000 because you might
later change your optimization settings.

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/CA%2B_-ou9ni04WhnsqCrmA3UGx%2BDnNrutoDpPQnWJpAA76USZWVw%40mail.gmail.com.

[Jeff Kaufman]

On Tue, Feb 24, 2015 at 10:23 AM, 'Jan-Willem Maessen' via
mod-pagespeed-discuss <mod-pagesp...@googlegroups.com> wrote:

> The big thing to note here is that even in high-intensity testing in
> PassThrough mode I was seeing style.min.css arrive uncompressed consistently
> every single time. That seems problematic.
>

Isn't that expected behavior? That's not a .pagespeed. resource, you
have IPRO off, and you didn't explicitly enable gzip in the config, so
it would never be gzipped.