Tried to use loadfromfile and "restrictive Cache-Control headers" went through roof

[Nigel Aves]

I was looking at the configuration for LoadFromFile and realised that I had implemented incorrectly and everything was being fetched via HTTPS.  loadfromfile is (or should be) pointing to .jpg files in my galleries. 

Before the change in console I had "Resources not rewritten because of restrictive Cache-Control headers: 0.42%"

I then made the following change and added to the end of each "target" ... \\1   like this;

ModPagespeedLoadFromFile "https://gallery.soft-focus-imagining.com/ _data /"   "/home/soft-focus-imagining/domains/gallery.soft-focus-imagining.com/public_html/_data/\\1"

ModPagespeedLoadFromFileCacheTtlMs 31556952000

ModPagespeedImplicitCacheTtlMs 31556952000

But when I looked again after about 20 minutes, I was not seeing any images getting modified (typically converted to webp) but when I looked at the console it was showing this:

Resources not rewritten because of restrictive Cache-Control headers: 97.4%

So I reverted back. Any ideas as to what I did wrong? (and if I get it working what would I expect to see in the log file messages).

Nigel.

here is a typical output;

[Sat, 31 Oct 2020 04:17:00 GMT] [Info] [1379635] https://gallery.soft-focus-imagining.com/index/category/worn-out-by-time:0: Shrinking image `https://gallery.soft-focus-imagining.com/_data/i/galleries/worn-out-by-time/worn-out-by-time-colorado-003-sm.jpg' (116245 bytes) to `https://gallery.soft-focus-imagining.com/_data/i/galleries/worn-out-by-time/xworn-out-by-time-colorado-003-sm.jpg.pagespeed.ic.yfH51iLYcE.webp' (41680 bytes)

[Longinos]

Hi Nigel

Why have you added the "\\1" at the end?

These side of the filter parameter is a path in your filesystem... Have you a directory called "\\1"?

The filter:

ModPagespeedLoadFromFile "https://gallery.soft-focus-imagining.com/ _data/"   "/home/soft-focus-imagining/domains/gallery.soft-focus-imagining.com/public_html/_data/"

works as:

eache request made to https://gallery.soft-focus-imagining.com/ _data /any_file. pagespeed search for it at this directory  "/home/soft-focus-imagining/domains/gallery.soft-focus-imagining.com/public_html/_data/" , so a request made to  https://gallery.soft-focus-imagining.com/ _data/some-sub-directory/some_file is searched at  /home/soft-focus-imagining/domains/gallery.soft-focus-imagining.com/public_html/_data/some_directory/some_file.

If you whis only read jpg files the way to do is some like this:

ModPagespeedLoadFromFile "https://gallery.soft-focus-imagining.com/ _data/"   "/home/soft-focus-imagining/domains/gallery.soft-focus-imagining.com/public_html/_data/"

ModPagespeedLoadFromFileRuleMatch disallow .* (this disallow any file)
ModPagespeedLoadFromFileRuleMatch allow \.(js|css|jpg|jpeg|png|pdf|gif)(\?.*)?$ (this allow only files with these extensions)

The order matter. If you don´t use the disallow directive and only use the allow, any other file can be loaded, because no disallow is presente.

If you put the disallow in the last position, no files are allowed

About these other filters:

ModPagespeedLoadFromFileCacheTtlMs 31556952000

ModPagespeedImplicitCacheTtlMs 31556952000

When you load a file from the filesystem, no http request is made, so no http headers are in place then tread the loaded files as it have a cache-control: max-age= 31556952000.

And the other is tread any resource fetched by http request that don´t have a cache-control header put one with a max-age of  3155695200.

I think is not a good idea put 1 year here. Why? These resources are stored in the pagespeed cache for 1 year, The resource loaded from filesystem is renewed each time the file changes, same the resource fethed by http, if the hash changes the optimized resource is renewed, but maybe the old resources still in the pagespeed cache until it expires or is evicted to make space for new items, so a sort time is better.

These times must be a deal between space in the pagespeed cache and how much times the resource need to be renewed.

These 2 parameters are NOT the time in what the resource is stored in the browser cache, a optimized resorce have a ttl for the brownser for 1 year..

P.S  I see, you put "\\1" because you have see it in https://www.modpagespeed.com/doc/domains#ModPagespeedLoadFromFile, but is NOT the same directive.

You see it in a ModPagespeedLoadFromFileMatch filter. This filter permit regular expressions,  ModPagespeedLoadFromFile NOT. And \\1 is part of a regular expression.

[Nigel Aves]

Thanks for spotting my deliberate mistake! DUHHH .... this what happens when trying to improve things when tired and working with options that look alike :) You never can successfully check your own work to 100%, I really should have spotted that.

Thanks also for the long write-up. Very helpful and informative.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e50a000c-4b33-4330-b5d8-d06f5fa1aa4bn%40googlegroups.com.

[Longinos]

Glad to ear it helped, and sorry for my bad,bad english....