Re: Individual Cache Flush (Individual TTL)

[Matt Atterbury]

IIRC we don't cache anything longer than its cache expiry header says to, so if you want CSS cached for less time reduce that (that's not something managed by mod_pagespeed, it's a normal part of Apache).

On Thu, Mar 7, 2013 at 2:38 AM, antituhan <dewan...@xtremenitro.org> wrote:

Hello there,

I've configured pagespeed using this configuration ...

        # Enable Filter
        ModPagespeedEnableFilters remove_quotes
        ModPagespeedEnableFilters insert_dns_prefetch
        ModPagespeedEnableFilters rewrite_images,sprite_images,resize_mobile_images
        ModPagespeedEnableFilters remove_comments,collapse_whitespace
        ModPagespeedEnableFilters lazyload_images
        ModPagespeedEnableFilters insert_dns_prefetch
        ModPagespeedEnableFilters move_css_to_head,move_css_above_scripts
        ModPagespeedEnableFilters make_google_analytics_async
        ModPagespeedEnableFilters canonicalize_javascript_libraries
        ModPagespeedEnableFilters elide_attributes
        ModPagespeedEnableFilters rewrite_style_attributes_with_url
        ModPagespeedEnableFilters trim_urls
        ModPagespeedEnableFilters recompress_images
        ModPagespeedEnableFilters recompress_jpeg,recompress_png
        ModPagespeedEnableFilters strip_image_meta_data

        ModPagespeedJpegRecompressionQuality -1
        ModPagespeedImageRecompressionQuality -1


And the cache flush every 3600000 ms (1 hour). Is it possible to make the cached flush by the file types ? (eg. css flush every 10 minutes, images every 30 minutes, js every 1 hour). Any hints? I need to make it individually because of my development team changes the css and pagepseed still cache it :(

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
"Klaatu barada nikto"                          (754) 444-6288

[Joshua Marantz]

Yeah -- I think you've conflated the frequency that mod_pagespeed cleans out the least-recently-used file-cache entries with the TTL (time-to-live).

The ModPagespeedFileCacheCleanIntervalMs is just a balance between how tightly you want mod_pagespeed to stick to its disk-space budget vs the CPU-time spent scanning the cache directories looking for old files.

The TTL is what you are thinking of, and that's usually controlled by a conf-file stanza like this:

    <FilesMatch "\.(jpg|jpeg|gif|png|js|css)$">

      Header set Cache-control "public, max-age=600"

    </FilesMatch>

You can tweak that as desired.

-Josh

[Joshua Marantz]

TTL is important for files that are rewritten, because mod_pagespeed uses that to determine how frequently to check back with the origin to see if the file has changed.

I am confused by the results you give above.  I guess www.like.me is not your real domain; at least I can't ping it.  facebox.js,Mjm.JXcwJDTwcQ.js looks like it came from a mod_pagespeed-rewritten URL but it's not quite right.  Is that the whole URL?

mod_pagespeed doesn't provide configuration for the TTL for rewritten files: it always makes them cacheable for a year, but it still supports the origin TTL.  The reason is that mod_pagespeed will in this case check back every 10 minutes to see whether facebook.js has changed.  If it has, it will rewrite the URL differently, because the MD5-sum of facebook.js will be different.

I don't know why you were seeing cache-control:max-age=0,no-cache for that URL.  That doesn't make sense to me but I suspect I'm not seeing the whole picture.

On Thu, Mar 7, 2013 at 9:44 AM, antituhan <dewan...@xtremenitro.org> wrote:

Hello J,

Thanks for your response, I thought the TTL only valid on non-rewritten files, for example like this :

[root@banehallow ~]# HEAD http://www.like.me/scripts/newlintasme_script/facebox.js
200 OK
Cache-Control: public, max-age=600

But, on the rewritten files, the cache is not valid ...

HEAD http://www.like.me/scripts/newlintasme_script/facebox.js,Mjm.JXcwJDTwcQ.js
200 OK
Cache-Control: max-age=0, no-cache

Is it normal? Or is there any hints to make the rewritten files follows apache TTL?

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
"Klaatu barada nikto"                          (754) 444-6288

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Joshua Marantz]

I think http://www.lintas.me/scripts/newlintasme_script/facebox.js,Mjm.JXcwJDTwcQ.js must be the result of some kind of JavaScript mangling problem.  Can you give the URL to the HTML that references it?

-Josh

On Thu, Mar 7, 2013 at 10:34 AM, antituhan <dewan...@xtremenitro.org> wrote:

Oh sorry, yes like.me is not my real domain, try lintas.me :) The modpagespeed is behind nginx reverseproxy....
Any hints J ?

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
"Klaatu barada nikto"                          (754) 444-6288

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Joshua Marantz]

Ah that looks much better.  That gives:

HTTP/1.1 200 OK

Server: lmws/1.0.12

Date: Thu, 07 Mar 2013 16:15:13 GMT

Content-Type: application/javascript

Connection: keep-alive

Expires: Fri, 07 Mar 2014 16:04:27 GMT

Cache-Control: max-age=31536000

Etag: W/"0"

Last-Modified: Thu, 07 Mar 2013 16:04:27 GMT

Content-Length: 25076

Vary: Accept-Encoding

Master: Sparda

This looks fine to me.  What's the problem?  If any of the constituent JS files are updated then within 10 minutes, mod_pagespeed will notice that and serve the combined file with: a new name, with "LtvLajeo7I" replaced by the md5-sum of the new combination. So it's safe to cache this combined file for a year, and browsers will still see any changes within 10 minutes.

-Josh

[Joshua Marantz]

On Thu, Mar 7, 2013 at 11:26 AM, antituhan <dewan...@xtremenitro.org> wrote:

Hmmm... So next 10 minutes, the mod_pagespeed will rewrite new hash?

Yes.

 

If I changes the content of the js, is the mod_pagespeed will point into new modified js files and rewrite it

Yes

 

or still old rewritten js files?

No.  The old cache entries may still be around after 10 minutes but no one will ever reference them again because they will have the old hash in the name, so the caches will eventually drop the content.

-Josh

[Matt Atterbury]

To clarify (correct me if I'm wrong Josh):

If I changes the content of the js, is the mod_pagespeed will point into new modified js files and rewrite it

Yes

Yes, after the 10 minutes has expired (*). If you tell mod_pagespeed the JS is valid for 10 minutes it will believe you, just like a browser would.

(*) Unless you're fetching directly from file using ModPagespeedLoadFromFile in which case I think we stat the file every time so see any changes immediately.

m.

[Joshua Marantz]

I always recommend using ModPagespeedLoadFromFile* when you can.  Using that means that your files will be updated immediately.  Even if you set a 10 minute TTL, your JS file updates will propagate to your HTML references immediately.

However, I'm not sure what the "issue" you think you have is.  In either case, the rewritten files will be served with 1-year TTL.  Without LoadFromFile, the origin JS will be checked for changes every 10 minutes.  With LoadFromFile, it will be checked on every access.

I've probably done a better explanation on the cache extension web page: please read this:  https://developers.google.com/speed/docs/mod_pagespeed/filter-cache-extend

On Thu, Mar 7, 2013 at 12:16 PM, antituhan <dewan...@xtremenitro.org> wrote:

So, should I enable ModPagespeedLoadFromFile or ModPagespeedMatchFromFile belongs to this issue?

--

[Joshua Marantz]

You can continue in this thread.

On Thu, Mar 7, 2013 at 10:17 PM, antituhan <dewan...@xtremenitro.org> wrote:

Okay J, thanks for your response. So, talk about ModPagespeedLoadFromFile, should I write new topic or just continue this one? I have some question related to LoadFromFile & MatchFromFile

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Joshua Marantz]

TTL = time-to-live.  That corresponds to the cache-control:max-age=xxx parameter in http 1.1, or the expires: header in http 1.0.

1-year TTL menas that browsers will keep the URL->content mapping for a year.

But if you change the content, then the URL will change as well, because we put the MD5-sum of the content into the URL.  In your URL above that's LtvLajeo7I.  With LoadFromFile, the second you update any one of those JS files, mod_pagespeed will notice and the next time the HTML is served, it will put a different hash in the URL.

-Josh

On Sat, Mar 9, 2013 at 1:41 PM, antituhan <dewan...@xtremenitro.org> wrote:

Well J,

I've enable ModPagespeedLoadFromFile, and the directive are like this

ModPagespeedLoadFromFile "http://www.thedomainabove.me/assets/" "/data/html/assets"

And if the rewrite rule like this
http://www.thedomainabove.me/assets,_foundation,_js,_jquery.offcanvas.js,Mjm.3_SXUjNO9U.js+assets,_foundation,_js,_jquery.placeholder.js,Mjm.oUKfTiV0Pi.js+scripts,_newlintasme_script,_facebox.js,Mjm.JXcwJDTwcQ.js+scripts,_newlintasme_script,_loader-v4.js,Mjm.YWCVCbeHTw.js.pagespeed.jc.LtvLajeo7I.js

Is it still direct to js/css file if I modified the original source? I mean, if I modified the source, the pagespeed rewrite's hash still the same or changed?
And if I use ModPagespeedLoadFromFile, should I enable the cache control? I don't understand with your statement "the rewritten files will be served with 1-year TTL", 1-year TTL? Could you explain it to me more clearly, J ?

Thank you :)

Pada Jumat, 08 Maret 2013 20:36:09 UTC+7, jmarantz menulis:

You can continue in this thread.

On Thu, Mar 7, 2013 at 10:17 PM, antituhan <dewan...@xtremenitro.org> wrote:

Okay J, thanks for your response. So, talk about ModPagespeedLoadFromFile, should I write new topic or just continue this one? I have some question related to LoadFromFile & MatchFromFile

Pada Jumat, 08 Maret 2013 0:34:15 UTC+7, jmarantz menulis:

I always recommend using ModPagespeedLoadFromFile* when you can.  Using that means that your files will be updated immediately.  Even if you set a 10 minute TTL, your JS file updates will propagate to your HTML references immediately.

However, I'm not sure what the "issue" you think you have is.  In either case, the rewritten files will be served with 1-year TTL.  Without LoadFromFile, the origin JS will be checked for changes every 10 minutes.  With LoadFromFile, it will be checked on every access.

I've probably done a better explanation on the cache extension web page: please read this:  https://developers.google.com/speed/docs/mod_pagespeed/filter-cache-extend

On Thu, Mar 7, 2013 at 12:16 PM, antituhan <dewan...@xtremenitro.org> wrote:

So, should I enable ModPagespeedLoadFromFile or ModPagespeedMatchFromFile belongs to this issue?

Pada Kamis, 07 Maret 2013 23:47:35 UTC+7, matterbury menulis:

To clarify (correct me if I'm wrong Josh):

If I changes the content of the js, is the mod_pagespeed will point into new modified js files and rewrite it

Yes

Yes, after the 10 minutes has expired (*). If you tell mod_pagespeed the JS is valid for 10 minutes it will believe you, just like a browser would.

(*) Unless you're fetching directly from file using ModPagespeedLoadFromFile in which case I think we stat the file every time so see any changes immediately.

m.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Joshua Marantz]

That isn't possible because mod_pagespeed explicitly prevents that by disabling mod_expires and mod_headers for delivery of .pagespeed. resources.  Why would you want to lower the TTL for .pagespeed. resources?   The md5-sum of the content is in the URL so it will never be stale.  That's the whole point of cache extension.

However, Apache *does* recognize the TTL in fhe FilesMatch block: that's how you communicate to mod_pagespeed how frequently to check the origin CSS & JS to see if they've changed.  The only exception is with LoadFromFile, when mod_pagespeed checks for resource updates on every request.

-Josh

On Sat, Mar 9, 2013 at 10:32 PM, antituhan <dewan...@xtremenitro.org> wrote:

Ah, ic. IMHO, my apache didn't recognize the TTL like this

        <FilesMatch "\.(js|css)">
                Header unset Etag
                Header set Cache-Control "public, max-age=600"
        </FilesMatch>

Because I'm using HTTP/1.0, not HTTP/1.1. Could I use mod_expires on apache to control the expires of file, like this one http://httpd.apache.org/docs/2.2/mod/mod_expires.html on pagespeed directive?

[Joshua Marantz]

Same deal as the other filters.  Without LoadFromFile, mod_pagespeed will use the expires/cache-control header for the resource, so for cache-control:max-age=600 it will be updated every 10 minutes.  With LoadFromFile, it will be checked on every HTML request.

-Josh

On Mon, Mar 11, 2013 at 3:58 AM, antituhan <dewan...@xtremenitro.org> wrote:

Hmmm ic, how about inline javascript? I've try to edit the js on root directory *is not defined at LoadFromFile directive*, here is my directive :

        ModPagespeedLoadFromFile "http://www.lintas.me/assets/" "/data/html/assets"
        ModPagespeedLoadFromFile "http://www.lintas.me/scripts/" "/data/html/scripts"
        ModPagespeedLoadFromFile "http://www.lintas.me/style/" "/data/html/style"

I've try to edit the inline js file on root, (eg. auth.php) but the js didn't update and still use the old rewritten data.
Should I enable this filter below ?

        ModPagespeedLoadFromFile "http://www.lintas.me/" "/data/html"

To avoid inline js/css and update regularly based on source file.

[Jimy Johny]

I'm using cdn to host all the css and js files. I want to set max-age=0 and s-maxage for one year. This is used as cdn gives first preference to s-maxage and also I could make sure browsers do not cache the files as browser gives preference to max-age. (I hope that's how it works) Which one will be used by mod_pagespeed ?

Regards,

JJ

[Joshua Marantz]

RE s-maxage: I don't think we do anything with that.  We'll ignore that on an origin resource, and we won't set it on a rewritten resource.  AFAICT, s-maxage is specific to resources that require HTTP authorization: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html .   mod_pagespeed will not optimize such resources.

RE: root config: you can do LoadFromFile on the root if you like.  This means mod_pagespeed will only rewrite resources that are found as files on your filesystem.  It won't use http fetching as a backup.  That might be fine.  As mentioned earlier, for resources loaded via LoadFromFile, mod_pagespeed will ignore Cache-Control directives at origin.  But mod_pagespeed will check for updates on every HTML request.  One thing to note: this would bypass "Cache-Control:private" so be careful if you have such resources not to issue a LoadFromFile directive at that directory or above.

-Josh

[Joshua Marantz]

The files that are rewritten by mod_pagespeed are cache-extended, so they are served for 1 year TTL independent of your Apache configuration.

There is no need for you to change the TTL.  Why do you think you want to do that?

On Sun, Mar 17, 2013 at 9:41 AM, antituhan <dewan...@xtremenitro.org> wrote:

Hi Josh,

Thanks for explaination about using LoadFromFile directive. Now, I've new question :D I send HEAD command to rewritten URL and clean URL.

[dominique@dominiQue ~]$ HEAD http://cdn.lintas.me/images/xspriteme1.png.pagespeed.ic.hUSzds-VD3.png
200 OK
Cache-Control: max-age=31536000
Connection: close
Date: Sun, 17 Mar 2013 13:29:00 GMT
Pragma: public
Accept-Ranges: bytes
ETag: W/"0"
Server: lmws/1.0.12
Content-Length: 104261
Content-Type: image/png
Expires: Mon, 17 Mar 2014 12:31:10 GMT
Last-Modified: Sun, 17 Mar 2013 12:31:10 GMT
Client-Date: Sun, 17 Mar 2013 13:26:58 GMT
Client-Peer: 117.102.117.90:80
Client-Response-Num: 1
Master: Sparda

[dominique@dominiQue ~]$ HEAD http://i.brta.in/images/spriteme1.png
200 OK
Cache-Control: max-age=604800
Cache-Control: public
Connection: close
Date: Sun, 17 Mar 2013 13:32:23 GMT
Pragma: public
Accept-Ranges: bytes
Server: lmws/1.0.12
Content-Length: 120465
Content-Type: image/png
Expires: Sun, 24 Mar 2013 13:32:23 GMT
Last-Modified: Thu, 14 Feb 2013 03:06:13 GMT
Client-Date: Sun, 17 Mar 2013 13:30:22 GMT
Client-Peer: 117.102.117.90:80
Client-Response-Num: 1

The un-compressed (un-modified files by pagespeed) are correct by using 1 week TTL. Btw, why the rewritten file by pagespeed didn't follow my apache rules? Should I modify the TTL on rewritten files by pagespeed?

--

[Joshua Marantz]

Glad we cleared that up :)

On Tue, Mar 19, 2013 at 10:21 AM, antituhan <dewan...@xtremenitro.org> wrote:

Ah damn :)) I've miss perception from the TTL. A TTL of rewritten files are set by default, but the TTL of original files. That's why I created this thread *blushing

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.