inconsistent number of webp images

194 views
Skip to first unread message

Ungov

unread,
Mar 17, 2022, 7:25:21 PM3/17/22
to mod-pagespeed-discuss
When I look at the source code of the homepage of my domain and CTRL+F "webp" to find how many webp images are on the page, the number is always changing

For example right now there is 31 results for webp images in the source code
Now I just force refresh with CTRL+F5 and the number went up to 41
I refreshed again and now it's 50
After many refreshes I see over 100 webp images
But if I come back after a while it will start from 0 again

Why is this happening? The page receives a lot of visitors, the cache headers are set correctly, and there is plenty of disk space for caching

Additionally, I tried logging in the pagespeed admin section but I always get a 403 Forbidden error using this code:
<Location /pagespeed_admin>
        Order allow,deny
        Allow from localhost
        Allow from *
        SetHandler pagespeed_global_admin
</Location>

Longinos

unread,
Mar 18, 2022, 5:13:37 AM3/18/22
to mod-pagespeed-discuss
Hi
Pagespeed by default don´t optimize ALL the images but 8. You need to set
ModPagespeedImageMaxRewritesAtOnce NumImages
The value -1 try to optimize ALL images, but if you have too many this has a cost in time waiting for the optimization.
Well... why the optimization restart after some time?
2 main reasons:
1.- cache-control: max-age value
2.- cache storage too short in size.

For 1, I can´t check the original cache-control header, I see wen the image is not optimized a cache-control: max-age 1 year, but is the original header or is set by Clodflare?
Have you set Cloudflare to respect original headers?
I see in the main request 2 cache-headers, f.e.

For 2, How have you set cache size?

ModPagespeedFileCachePath "/var/cache/pagespeed/"
ModPagespeedFileCacheSizeKb 102400
ModPagespeedFileCacheCleanIntervalMs 3600000
ModPagespeedFileCacheInodeLimit 500000

What values have you here?

Pagespeed admin access is configured wrong:

Allow from localhost only allow from localhost acces
Allow from * This is not a valid statement.
Allow from admit hostname (like localhost or www.myhost.com), ip ( like 123.123.123) or the statement "all"
So for permit acces from any site must be "Allow from all"

Ungov

unread,
Mar 18, 2022, 3:50:56 PM3/18/22
to mod-pagespeed-discuss
Hi,

I understand that Pagespeed doesn't optimize all of the images, but after refreshing the page tens of times and getting over 100 webp images in the source code, I would expect that if I refresh the page 1 hour later there will still be at least 100 webp images optimized

Cloudflare is set to respect original headers. Image cache expiry is set to 1 year in the htaccess file.

I have already tried to increase the cache size many times, it is probably overkill as is right now:
ModPagespeedFileCachePath            "/var/cache/mod_pagespeed/"
ModPagespeedFileCacheSizeKb          50000000
ModPagespeedFileCacheCleanIntervalMs 86400000
ModPagespeedFileCacheInodeLimit      5000000
ModPagespeedLoadFromFileCacheTtlMs 2419200000
ModPagespeedDownstreamCachePurgeLocationPrefix http://localhost:8443

Thanks for your help, I am now able to access the admin section. I'll let it publicly accessible for a couple of days if you need to check. I see a lot of errors "Too busy to rewrite image" and "served with only 90% rewriting done"

Longinos

unread,
Mar 19, 2022, 9:07:43 AM3/19/22
to mod-pagespeed-discuss
Hi
Why do you have this

ModPagespeedDownstreamCachePurgeLocationPrefix?

Have you a ngnix proxy cache or a varnish cache in front of your site?

These messages are not error but info:
Too busy to rewrite image: self explanatory, the server is busy rewriten other images (rewrite images, conver it to webp,etc,etc is too cpu expensive)
served with only 90% rewriting done well... the resource is not totally optimized when served.

In pagespeed_admin -> statistics:

file_cache_evictions
775620
file_cache_hits 146154
file_cache_inserts 807026

I think this will say the cache is not so big for the number of assets stored:
evictions are deletions to made site to new assets not by expiration.

In pagespeed_admin -> Configuration:
ImagePreserveURLs (ipu) True This makes image url don´t change, some filters are disabled, images not converted to webp..... ModifyCachingHeaders (mch) False This make pagespeed s unable to change the cache headers.

For now I can´t test much more as far as I go out of house...

Ungov

unread,
Mar 19, 2022, 4:15:08 PM3/19/22
to mod-pagespeed-discuss
Hello,

I'm not sure about  ModPagespeedDownstreamCachePurgeLocationPrefix. This is an old config. I used nginx cache previously but right now it is disabled, but I still use nginx proxy mode without caching. I tested both with and without "Serve static files directly by nginx" but it didn't change anything https://prnt.sc/hCjFn7DKzKXV

How can I see how much space is used by pagespeed cache? I tried doing "du -sh" on /var/cache/mod_pagespeed/ but it always fail with the error "Aborted (core dumped)" maybe because there are too many files in the folder. For now I raised cache size from 50GB to 100GB but the total disk space used by this domain in Plesk is only 10GB so the cache size should be enough

ImagePreserveURLs doesn't seem to be working because the webp filenames keep changing

Caching is set to 1 year through htaccess, what should I do with ModifyCachingHeaders ?

Longinos

unread,
Mar 20, 2022, 7:56:57 AM3/20/22
to mod-pagespeed-discuss
Hi
ModPagespeedDownstreamCachePurgeLocationPrefix is used when a cache (nginx or varnish) is in front of the web server, and in the pagespeed admint there are cache purge request (remember the messages "served with only 90% rewriting done"? these messages come from purge request, delete a cached page when only 90% is rewrited), so I think the proxy cache still here and working.

Should purge "https://www.no-gods-no-masters.com/women-tshirt-zad-partout-D01001235777P0347/" which was served with only 90% rewriting done.[Sun, 20 Mar 2022 11:49:40 GMT] [Info] [1905276] Purge url is http://localhost:8443/women-tshirt-zad-partout-D01001235777P0347/



Seem that in /var/cache/mod_pagespeed/  are stored non only 1 domain but all, in pagespeed_admin I can se request for more that 1 domain.
Pagespeed cache need to be 5x - 7x the size of files (f.e. 1 image can have many "versions" for diferent sizes, devices....)

ImagePreserveURLs is here... is all most strange it not works..... maybe there are diferent config files?

Well... caching headers, how it works (whe you fecht the site directly, no Cloudflare, no cache of any type)

1.- The original cache-control: max-age (the headers used if pagespeed is off) tell pagespeed how much time the optimized resource is stored in their cache.
If this header is st to 600, the optimized resource (these have pagespeed in the url) lives in the pagespeed cahe 600 (5minutes)

2.- If ModifyCachingHeaders is true, then pagespeed changes the cache-control: max-age to 1 year for the optimized resource, if the resource is not fully optimized
then it serves the original resiurce, with the original cache-control and add a s-max-age=10 to tell intermediates cahe (Cloudflare f.e) to not stored the unoptimized resource more that 10 sec.

3.- If you use LoadFromFile (and you do, allmost is in the config) then no http headers are in the resource request because is not http requested but loaded directly from disk.
In this case you must use LoadFromFileCacheTtlMs (you have it in the config set to 28 days) to tell pagespeed how much time store the optimized resource in their cache.
ModPagespeedImageMaxRewritesAtOnce NumImages

You need to increase this parameter, is not set and the default is 8.
Your pages have more than 50 images each, so rewriten 8 each request it takes some hits to have the images rewrited This makes the "too busy..." mesages.
Increase it progressively and take a look at CPU use and time to the page be served.
Try a start of 25, this takes 3-4 hits to get all images rewited, lest than the actual 12-13

Longinos

unread,
Mar 20, 2022, 9:09:36 AM3/20/22
to mod-pagespeed-discuss
Hi again

I think some config about the pagespeed config is wrong


file_cache_evictions
1042581
file_cache_hits 19467
file_cache_inserts 989448
file_cache_misses 544898

seems like each item inserted in the config is evicted.
Whitout the whole config, apache, nginx, pagespeed,.... I can´t say what is wrong here

Ungov

unread,
Mar 20, 2022, 8:42:45 PM3/20/22
to mod-pagespeed-discuss
Hello,

Yes there are 5 domains storing cache in /var/cache/mod_pagespeed/
I just updated the config to use  /var/cache/mod_pagespeed_ngnm/ for the main domain I'm testing with. So there should be plenty of dedicated space (100GB cache for 10GB of files)

Also added:
ModPagespeedLoadFromFileCacheTtlMs 31556952000 (1 year)
ModPagespeedImageMaxRewritesAtOnce 50

Full pagespeed config: https://pastebin.com/CAnX77py

I have not touched nginx/apache config. I'm not even sure where the config files are located. If you give me the exact path I can copy the config for you. I use CentOS 8.5 with Plesk

/etc/nginx/nginx.conf  ----> https://pastebin.com/kPFKV0JH
/etc/httpd/conf/httpd.conf  ----> https://pastebin.com/TK5nEnK3

Please note that I don't put my pagespeed config in the nginx/apache config file, I put it directly in the "additional directives" section of Plesk Apache & Nginx Settings
like this screenshot: https://ibb.co/2njgtJj

Longinos

unread,
Mar 21, 2022, 6:28:45 AM3/21/22
to mod-pagespeed-discuss
Hi
Now you have a new problem:
[Mon, 21 Mar 2022 10:26:13 GMT] [Error] [2531937] Failed to make directory /var/cache/mod_pagespeed_ngnm/v3: Permission denied
Check file owner and permisions on /var/cache/mod_pagespeed_ngnm

Longinos

unread,
Mar 21, 2022, 7:21:13 AM3/21/22
to mod-pagespeed-discuss
Hi
I take a look at the config files. Tell me if i´m wrong.

1.- You have apache+pagespeed running in localhost:8443
2.- You have nginx running main domain:443
3.-  apache+pagespeed load files from disk
4.-  nginx serve statics files

I that is rigth I don´t know why pagespeed is working with images as far as images are served by nginx with no pagespeed module.
So I don´t know why but seems some images are served directly from nginx (and no pagesped rewrited) and some images are served by apage and are rewrited by paespeed.
I think you need to deselect "Serve static  file directly from nginx" in the nginx section of plesk.

What about files here /etc/nginx/conf.d/*.conf;? Have you 1 file for each server nginx section?

For the pagespeed config:
Here is the
ModPagespeedImagePreserveURLs on
comment it put off.

About the pagespeed cache size:
There are 2 parameters that control the cache size:
ModPagespeedFileCacheSizeKb  # This control the size in Kb
ModPagespeedFileCacheInodeLimit# Ths control the inode number used.
So cache can be considered "full" used when the size of files are 95Gb OR when the number of files are 5.000.000 even if no 95Gb of space is occuped.
For example you may have 95 files of 1gb each and the cache is full,  or have 5.000.000 files of 1kb each (4Gb) and the cache is full.

Ungov

unread,
Mar 21, 2022, 4:12:34 PM3/21/22
to mod-pagespeed-discuss
Ok now I set the same file owner for "mod_pagespeed_ngnm" as "mod_pagespeed"  (apache:apache owner instead of root:root)

I have disabled "Serve static files directly by nginx"
Should I also disable "Smart static files processing" ? The description says "If turned off, Apache will process all requests for static files. nginx will only pass requests and responses without modification. Turn off this option only for troubleshooting."

Here are the files in /etc/nginx/conf.d/

[root@ns508279 cache]# cd /etc/nginx/conf.d/
[root@ns508279 conf.d]# ls
brotli.conf  ssl.conf  zz010_psa_nginx.conf  zz030_ext-monitoring.conf

Do you want me to disable ModPagespeedImagePreserveURLs?

Is there a way to know the current disk space usage of pagespeed cache and the current inode usage? I have changed it to:
ModPagespeedFileCacheInodeLimit      50000000

Ungov

unread,
Mar 21, 2022, 4:22:00 PM3/21/22
to mod-pagespeed-discuss
I found how to get the cache size and inode usage

[root@ns508279 /]# cd /var/cache
[root@ns508279 cache]# du -sh *
0       app-info
0       bpf
105M    dnf
0       httpd
0       krb5rcache
36K     ldconfig
0       libX11
1.8M    man
37G     mod_pagespeed
291M    mod_pagespeed_ngnm
0       nginx
0       PackageKit
13M     parallels_installer
0       private
0       yum
[root@ns508279 cache]# ls -id
3773759 .
[root@ns508279 cache]#

5 domains storing cache in /mod_pagespeed/ are using only 37G total and 3773759 inode
So the cache limit is definitely not exceeded

I can also see the size of /mod_pagespeed_ngnm/ is growing so pagespeed seems to be correctly saving the cached files now

Longinos

unread,
Mar 21, 2022, 8:46:49 PM3/21/22
to mod-pagespeed-discuss
Hi
Yes, try to unset "Smart static files processing" and see whats happen.
At a first glance nginx don´t need to serve statics files as far as pagespeed is not running here.
I see you restarted about 9:35p.m. CET and until now, 1:01 a.m CET file evictions are 0
Much more images get converted to webp in 2-3 hits.
And yes, disable ModPagespeedImagePreserveURLs
Whats the content of zz010_psa_nginx.conf?

Ungov

unread,
Mar 22, 2022, 2:30:27 AM3/22/22
to mod-pagespeed-discuss
Hi,
I disabled Smart Static Files Processing and ModPagespeedImagePreserveURLs

The webp images count on the homepage is rising much faster now. I think ModPagespeedImageMaxRewritesAtOnce helped a lot. I could try raising it higher, it raised a little bit the CPU usage and load average but not significantly https://prnt.sc/vSgiIaGV9M5D
But I would rather prefer to fix the root cause and not the symptoms

[root@ns508279 ~]# cd /etc/nginx/conf.d/
[root@ns508279 conf.d]# nano zz010_psa_nginx.conf
#DO NOT MODIFY THIS FILE BECAUSE IT WAS GENERATED AUTOMATICALLY,
#SO ALL YOUR CHANGES WILL BE LOST THE NEXT TIME THE FILE IS GENERATED.
include /etc/nginx/plesk.conf.d/server.conf;
include /etc/nginx/plesk.conf.d/webmails/*.conf;
include /etc/nginx/plesk.conf.d/vhosts/*.conf;
include /etc/nginx/plesk.conf.d/forwarding/*.conf;
include /etc/nginx/plesk.conf.d/wildcards/*.conf;


Maybe this can be useful?

/var/www/vhosts/system/domain.com/conf/httpd.conf :

/var/www/vhosts/system/domain.com/conf/vhost_ssl.conf :
^ this one is the pagespeed config

Longinos

unread,
Mar 22, 2022, 8:31:04 AM3/22/22
to mod-pagespeed-discuss
I still getting some strange issue
Some webp converted images have
content-type: image/jpeg
This is nor normal behaviuor, must be
content-type: image/webp

See these 2 images

Ungov

unread,
Mar 22, 2022, 9:10:53 PM3/22/22
to mod-pagespeed-discuss
The content-type bug seems to be due to Cloudflare. I paused Cloudflare and now the image you linked is showing webp content-type. I will leave Cloudflare disabled to continue the investigation.
https://prnt.sc/Ic49DBJYIyMt (on the left is jpg content-type before disabling CF, and on the right is webp content-type after disabling CF)

What can I do to make my config less complex? I didn't edit any files manually. I put all the config in Plesk Web Admin interface. This is a standard Plesk installation.

Longinos

unread,
Mar 23, 2022, 4:24:44 AM3/23/22
to mod-pagespeed-discuss
Hi

I think complexity comes from plesk.
As far as I can see you only need a web server without proxy.
The idea of using nginx+apache is not so bad, nginx serves statics files and manage user conections, in both is better than apache.
But using pagespeed introduces changes that these combination is useless if no are a server cluster o load balancers in place.
May be you need to re-tink the whole installation and use only apache or nginx. Pagespeed nginx module need to be compiled with nginx not as easy than apache
but is better serving statics files and managing connections, this not so importan as far as you use CF.

Well... now we are requesting to the original server :
Take a look at the main request (https://www.no-gods-no-masters.com/), there are 2 cache-control headers:
cache-control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0
cache-control: max-age=864000
You need to find from where comes the second header and unset it

Take a look at this file
https://www.no-gods-no-masters.com/images/homepage/trending_refugees_welcome.jpg from the headers look at is not served by apache but from nginx and then get not rewrited, never.
It preserves the original headers as if pagespeed don´t care about this file.

Despite no file evictions in the pagespeed cache, optimizations are lost after a bit of time, maybe not so fast as before, but they lost it.
Other strange thing, despite cache is storing assets, and not evicting, the hits are low, I think
file_cache_hits 78238
file_cache_inserts 816818

About retinking the site, are you able to set a stage site with only apache + pagespeed and a copy of the main domain?


Ungov

unread,
Mar 23, 2022, 5:25:04 PM3/23/22
to mod-pagespeed-discuss
Hello,

I found the cause of the double cache-control header, it was in the htaccess:
ExpiresActive On
ExpiresDefault "access plus 10 days"
I removed it and now the cache-control header with max-age=864000 is gone

The website is not ready to upgrade to nginx-only and it seems impossible to completely disable nginx proxy mode with Plesk according to this article: https://support.plesk.com/hc/en-us/articles/360006364914-How-to-make-a-website-be-processed-only-by-Apache-when-nginx-is-enabled-on-a-Plesk-server
So I followed the instructions in the link above and set the domain to use "Dedicated FPM application served by Apache" instead of served by nginx

However, as the article notes:
"After applying this solution, the website will still be shown as served by nginx in a web browser. This is due to the fact that nginx is working as the front-end proxy. The website will be processed by Apache in fact"

Thanks for the support, you are very helpful I appreciate it a lot!

Longinos

unread,
Mar 24, 2022, 4:43:14 AM3/24/22
to mod-pagespeed-discuss
Hi
I´m glad to help from a little union

Ungov

unread,
Mar 24, 2022, 5:21:46 AM3/24/22
to mod-pagespeed-discuss
However, the issue with the webp images is still not fixed :( Is there anything else I can investigate?

Ungov

unread,
Mar 25, 2022, 5:04:09 PM3/25/22
to mod-pagespeed-discuss
If the problem is related to Plesk, then I could contact Plesk and ask for support, but I'm not sure what should I ask them?

Longinos

unread,
Mar 26, 2022, 10:09:38 AM3/26/22
to mod-pagespeed-discuss
Hi
No, the problem is not with Plesk.
They do a good decision, serving statics assets from nginx and managing conections with it, and let apache to the background process.
But this a  web "normal" configuration. When you put pagespeed in the equation then things changes.
If you put apache + pagespeed, then statics assets need to be served by apache because are rewrited and these rewrited static contend served from the pagespeed cache.... from apache.
This makes the initial config netx to useless.
Pagespeed need hits on the web server it run t work, so if you put a cache in front then you need a shot time for the cache to mage request go to the backend and make pagespeed wor.
This config ( nginx proxy cache -> apache + pagespeed) makes sense in a very high used web server.
And Plesk make it a plus confuse thing creating a lot of config files, but is how it work. And you cn´t taylar these config files because Plesk overwrite it when you makes changes to the files.
So my tougths in this cases is making the whole proccess step by step:
1.- Create the apache web server and do it work well.
2.- Add the pagespeed module, and try the pagespeed directives snf configurations, see what they do and what the web need.
3.- Add the nginx proxy cache, see how it works and play with the config
4.-  Add CF to the whole thing.
But this a production installation, so changes can have non desired effects.

And this web have a heavy work to do because you have 100 images in each page (or near it)
So, what I think you need to do:
1.- Make sure all statics asset that pagespeed processes have a public cacheable header / cache-contol: max-age=XXXX, public)
the XXX not need to be 1 year because when pagespeed rewrites it then they have the 1 year. Try 2-3 days. 172800 - 259200, If you set it to 1 year this tell pagespeed to store the optimized resources for 1 year in the cache , why do you need to store in cache an asset not accessed by 2-3 days?
2.- Make sure ALL request go to the apache, nginx (seems that in plesk nginx can´t be bypassed) must be a transparent proxy
3.- For some time and as a "experiment" put ModPagespeedImageMaxRewritesAtOnce -1, and see if the times for rewrite all images are acceptable.
4.- For help in 4, put  RewriteDeadlinePerFlushMs 20 (the defaul is 10). This the time pagespeed used to to optimizations in a page before it flush the page to the client
5.- Take a look on how cache is working. Remember you have 2 caches in place SHMcache and filecache. Shmcache is a in-memory thing, so increase the size consumes RAM memory
6.- Consider intrduce redis in the config, this a ram database to store the cache.

Ungov

unread,
Mar 26, 2022, 5:20:17 PM3/26/22
to mod-pagespeed-discuss
Hi,

2- what is pagespeed directives snf configurations?
3- nginx cache is still disabled, I will enable it back only after I fixed the issues with pagespeed
4- Cloudflare is disabled but it didn't change anything
5- The double cache-control header was fixed, right now there is only 1 cache-control header. 1 year is fine with me because I want to always serve webp images and I have a lot of disk space I can use. Anyway I don't think it will change anything about the issue with inconsistent number of webp served
6- With ModPagespeedImageMaxRewritesAtOnce -1 I can see 287 webp images on the homepage after 3 page reload. Then if I wait 5 minutes and force refresh again, the webp images go down to zero again. It does help but as mentioned previously it seems like treating symptoms instead of fixing the underlying issue. Webp images will still constantly gets regenerated, which is what I'm trying to avoid/fix
7- Can Redis really help with the issue of webp images being constantly regenerated? Seems like it would only introduce another variable in my config and make it even more complex
8- How do I enable SHMcache? I get this error when adding it to my config through Plesk interface:
[mod_pagespeed 1.13.35.2-0 @706229] 'CreateSharedMemoryMetadataCache' is global and is ignored at this scope AH00526: Syntax error on line 15 of /var/www/vhosts/system/no-gods-no-masters.com/conf/vhost_ssl.conf: <VirtualHost> cannot occur within <VirtualHost> section

Longinos

unread,
Mar 27, 2022, 7:13:54 AM3/27/22
to mod-pagespeed-discuss
HI

2- what is pagespeed directives snf configurations?
snf mus be "and", my bad... a mistake

8:- Becacse this directive can´t be set in a <VirtualHost> section. This must be set like this:

ModPagespeedCreateSharedMemoryMetadataCache "/var/cache/pagespeed/" 51200
ModPagespeedCreateSharedMemoryMetadataCache "/var/cache/pagespeed-new/" 51200
<VirtualHost www.example.com:80>
 ModPagespeedFileCachePath "/var/cache/pagespeed/"
</VirtualHost>
 <VirtualHost alt.example.com:80>
ModPagespeedFileCachePath "/var/cache/pagespeed-new/"
 </VirtualHost>
This creates 2 SHMCache and is used in each <VirtualHost>. Yes, is a bit confising, you create a  CreateSharedMemoryMetadataCache but is used with  ModPagespeedFileCachePath)

Let me explain how pagespped cache is structured:
Basically there are 2 cache, metadata and html (no, not the html code but rewrited resources, webp images for example)
The metadata cache info about a resource, for example :
this image is converted in this webp
whit a size 400x400, is read from file, has a size of 8432, can be inlined as " RIFFD\r\000\000WEBPVP8 8......"
These info is stored in the SHMCache.
The webp himself is stored in the file cache.
If the metadata is not here, pagespeed don´t know what webp belongs to whats jpg file, even if the webp is in the file cache, so the optimization process restart to recreate the metadata cahe.
Where the metadata cache is stored? Well.... small piezes bellow 64Kb in the SHMCache (in the RAM memory) and pizes over 64Kb in the file cache (you can see an rname folfer under the
/var/cache/mod_pagespeed_ngnm/)
The most of the metadata are below the 64Kb limit, so are stored in the SHMCache in Ram, when evicted, the optimization process restar despite the optimized resource is in the file cache.

Your case:
You have now 2 files cache, 1 for the main domain and 1 for all others, if I remember:
/var/cache/mod_pagespeed/
/var/cache/mod_pagespeed_ngnm/
rigth?
then you can create a SHMCache for each
ModPagespeedCreateSharedMemoryMetadataCache " /var/cache/mod_pagespeed/ " 51200
ModPagespeedCreateSharedMemoryMetadataCache "  /var/cache/mod_pagespeed_ngnm/ " 51200
and in the
<VirtualHost main-domain>
ModPagespeedFileCachePath " /var/cache/mod_pagespeed_ngnm/"
  </VirtualHost>
<VirtualHost other-domain>
ModPagespeedFileCachePath " /var/cache/mod_pagespeed_ngnm/"
  </VirtualHost>

If you don´t have any ModPagespeedFileCachePath directive set in you config, there is a default one, with a 50Mb size.
You can modify the size with "ModPagespeedDefaultSharedMemoryCacheKB size-in-kb"

When you put redis (or memcached) in place, then you can make ALL the caches get stored in the redis (or memcached9 database in RAM memory

Longinos

unread,
Mar 27, 2022, 7:16:09 AM3/27/22
to mod-pagespeed-discuss
Hi
My bad , again

<VirtualHost other-domain>
ModPagespeedFileCachePath " /var/cache/mod_pagespeed/"
  </VirtualHost>

This is the rigth 2nd <VirtualHost>

Ungov

unread,
Mar 27, 2022, 6:24:51 PM3/27/22
to mod-pagespeed-discuss
This one took me a while to figure out where to put the  ModPagespeedCreateSharedMemoryMetadataCache directive because I can't put it with the rest of my config in Plesk Additional Directives section
So I tried to google where to put GLOBAL apache directives for Plesk but I get conflicting answers
Currently I put the SharedMemory directive in  /etc/httpd/conf/httpd.conf .... is it correct?


and yes, that is right, I use /var/cache/mod_pagespeed_ngnm/ for the main domain and /var/cache/mod_pagespeed/ for all other domains

The changes didn't fix the issues with the webp images. After some time all webp images are still lost on the homepage

I have also tested turning ModPagespeedImagePreserveURLs back on, but it's still not working

Ungov

unread,
Mar 27, 2022, 6:27:51 PM3/27/22
to mod-pagespeed-discuss
As for the ModPagespeedFileCachePath directive, I already have it in Plesk Apache Additional Directives GUI

So I didn't put it in  /etc/httpd/conf/httpd.conf because it's already in /var/www/vhosts/system/no-gods-no-masters.com/conf/vhost_ssl.conf

Longinos

unread,
Mar 28, 2022, 4:02:33 AM3/28/22
to mod-pagespeed-discuss
Hi
The http.conf is the main config file for apache, all other config files must be included here to work.
ModPagespeedCreateSharedMemoryMetadataCache must be before all <Vhost> are included so, I think, the rigth place is
in the http.conf before IncludeOptional conf.d/*.conf /Think this line includes the files where <Vhost> are defined)

For now I see that in the main domain it takes more time to lost the webp conversion.
Can you compare how much time take webp to be lost in the main domain and in others domains?

There is a parameter you can try:
ModPagespeedShmMetadataCacheCheckpointIntervalSec 600
This must be in the same site of ModPagespeedCreateSharedMemoryMetadataCache
This creates a disk copy of SHMCache then the optimizations survive server restart.
600 are 5min.

Longinos

unread,
Mar 28, 2022, 7:53:56 AM3/28/22
to mod-pagespeed-discuss
Hi

There is another image:
as you can see, is not webp converted but:
Metadata cache key:rname/aj_GtCzyBf1YlXRfF0-5CW8/https://www.no-gods-no-masters.com/images/texturebg.jpg@@a_
cache_ok:true
can_revalidate:false
partitions:partition {
optimizable: true
input { index: 0 type: FILE_BASED last_modified_time_ms: 1596189146000 filename: "/var/www/vhosts/no-gods-no-masters.com/public_html/images/texturebg.jpg" } optimized_image_type: 4
}
debug_message: "Image https://www.no-gods-no-masters.com/images/texturebg.jpg does not appear to need resizing."

There is a webp image in the metadata cache but is not served. Maybe the no rewrite images url is working here?

In other hand this image
cache_ok:true
can_revalidate:false
partitions:partition {
optimizable: false
 input { index: 0 type: FILE_BASED last_modified_time_ms: 1645526592000 filename: "/var/www/vhosts/no-gods-no-masters.com/public_html/images/homepage/trending_refugees_welcome.jpg" } }
other_dependency { type: FILE_BASED last_modified_time_ms: 1645526592000 filename: "/var/www/vhosts/no-gods-no-masters.com/public_html/images/homepage/trending_refugees_welcome.jpg" }
debug_message: "Image https://www.no-gods-no-masters.com/images/homepage/trending_refugees_welcome.jpg does not appear to need resizing."

As you can see no webp version is stored in the metadata cahe.
Why? well.... maybe the webp version is less than 10% in size that the jpg version, in such cases pagespeed don´t rewrite it.

Ungov

unread,
Mar 28, 2022, 5:32:07 PM3/28/22
to mod-pagespeed-discuss
I don't see any difference... After only a couple of minutes, webp images are still lost and regenerated :(

I added ModPagespeedShmMetadataCacheCheckpointIntervalSec 600 to my config

I also opened a ticket with Plesk support to confirm where to put directives in the global config and they told me to put it in /etc/httpd/conf.d/pagespeed.conf

Longinos

unread,
Mar 29, 2022, 3:32:28 AM3/29/22
to mod-pagespeed-discuss
Yes, webp is lost, but I think in the main menu takes more time than in the others domains.
No matter where the global directives are set, but out of <Vhost> declarations. In some way put it in a *.conf file that is included in the http.conf , but make sense use of pagespeed.conf to maintain all pagespeed directives toghether.

What the use of RAM in the server?
Can you try to increase SHMCache size?
Some like:
ModPagespeedCreateSharedMemoryMetadataCache "  /var/cache/mod_pagespeed_ngnm/ " 102400
My bet in this issue is the metadata cache is small for all the assets it need to store, so evict entries and this makes the lost of webp optimizations.


In other hand I see this:


This make me thinking that apache+pagespeed is running at http://localhost:7080 and then this is rewrited to  https://www.no-gods-no-masters.com/
But why the others domains aren´t?

Ungov

unread,
Mar 29, 2022, 5:53:53 PM3/29/22