Errors with new pagespeed version but not with older version

[Gorka]

Hello,

I just updated my pagespeed version to 1.13.35.2-0. Previously it worked without problems with same config, no warnings or errors. I have not changed the nginx configuration, I haven noted the previous pagespeed version so I can't add that info. Here are the errors:

2021/11/29 03:33:36 [error] 16790#17411: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 19 of 48 recent fetches fully successful; is fetching working? 2021/11/29 04:04:57 [error] 16785#17685: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 10 of 43 recent fetches fully successful; is fetching working? 2021/11/29 04:36:48 [error] 16794#17415: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 13 of 54 recent fetches fully successful; is fetching working? 2021/11/29 05:08:13 [error] 16783#18924: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 0 of 35 recent fetches fully successful; is fetching working? 2021/11/29 05:40:25 [error] 16785#17685: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 21 of 81 recent fetches fully successful; is fetching working? 2021/11/29 09:15:54 [error] 16783#18924: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 37 of 94 recent fetches fully successful; is fetching working? 2021/11/29 11:17:12 [error] 16789#16955: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 28 of 108 recent fetches fully successful; is fetching working? 2021/11/29 21:31:01 [error] 16789#16955: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 70 of 154 recent fetches fully successful; is fetching working?

I have tried this method:

https://www.modpagespeed.com/doc/faq#warning-fetch-rate but debug of filters doesn't show any errors.

Here is my ngnix conf:

http:

        pagespeed ListOutstandingUrlsOnError on;

        pagespeed FileCachePath /var/ngx_pagespeed_cache;

        pagespeed EnableCachePurge on;

        pagespeed PurgeMethod PURGE;

on server:

    pagespeed on;

    location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {

    access_log off;

    add_header "" "";

    }

    location ~ "^/pagespeed_static/" { access_log off; }

    location ~ "^/ngx_pagespeed_beacon$" { access_log off; }

Here is my nginx compilation params:

nginx version: nginx/1.21.4

built by gcc 10.2.1 20210110 (Debian 10.2.1-6)

built with OpenSSL 1.1.1k  25 Mar 2021

TLS SNI support enabled

configure arguments: --add-module=../naxsi/naxsi_src --with-cc-opt='-Wdate-time -D_FORTIFY_SOURCE=2 -Wno-error=date-time' --with-ld-opt=-Wl,-z,relro --prefix=/usr/share --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --modules-path=/usr/share/nginx/modules --build='VirtuBox Nginx-ee' --with-file-aio --with-threads --with-http_v2_hpack_enc --with-http_v2_module --with-http_ssl_module --with-pcre-jit --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_gzip_static_module --with-http_gunzip_module --with-http_mp4_module --with-http_sub_module --add-module=../ngx_http_substitutions_filter_module --add-module=../srcache-nginx-module --add-module=../ngx_http_redis --add-module=../redis2-nginx-module --add-module=../memc-nginx-module --add-module=../ngx_devel_kit --add-module=../set-misc-nginx-module --add-module=../ngx_http_auth_pam_module --add-module=../nginx-module-vts --add-module=../ipscrubtmp/ipscrub --add-module=../incubator-pagespeed-ngx-latest-beta --add-module=../echo-nginx-module --add-module=../headers-more-nginx-module --add-module=../ngx_cache_purge --add-module=../ngx_brotli --with-zlib=../zlib-cf --with-openssl-opt=enable-tls1_3 --sbin-path=/usr/sbin/nginx

Any ideas on how to debug this issue further?

[Longinos]

And in nginx log files?

Look for request with User Agent with the word "Serf" in it.

[Longinos]

And some more things:

If you put access_log off; in pagespeed location, you can´t see hothing in nginx location, so comment it, allmost for testing.

I can´t see https config here, so Serf may fail to access resources.

[Gorka]

I have enabled logging for all request now, we'll see, but AFAIK fetch will fail when getting the resources, not accesing them in pagespeed location right?

Regarding to HTTPS config, the previous nginx_pagespeed module stable was working with the same config, it's necessary to explicity set it on debian systems?

I found this info on pagespeed documentation:

"The default directory for Debian-based systems is /etc/ssl/certs, and there is no certificate file setting. On CentOS-based systems, the default directory is /etc/pki/tls/certs and default file is /etc/pki/tls/cert.pem."

I just added pagespeed SslCertDirectory /etc/ssl/certs; just in case, we'll see. 

Thanks for your time

[Gorka]

I'm having the same issue:

"2021/12/07 11:04:25 [error] 246131#246378: [ngx_pagespeed 1.13.35.2-0] PageSpeed Serf fetch failure rate extremely high; only 37 of 121 recent fetches fully successful; is fetching working?"

I don't see any mod_pagespeed / serf request on access.log.

[Longinos]

Hola

To enable https fetch these directives are needed:

pagespeed FetchHttps enable;

pagespeed SslCertDirectory directory;

https://www.modpagespeed.com/doc/https_support#https_fetch

Have you tried LoadFromFile?

https://www.modpagespeed.com/doc/https_support#load_from_file

Thsi need some more config options because files are loaded from disk and don´t have http headers.

And other thing to see:

https://yourdomain.com/?PageSpeedFilters=+debug

This put debug messages in the html code as comments, so maybe there are some clue for the 404.

El martes, 7 de diciembre de 2021 a las 11:43:38 UTC+1, Gorka escribió:

Ok, I have images with log off, now here is more info with 404 errors, probably because nginx misconfiguration. 

[Gorka]

I have found the problem:

My magento installation was changing image/cache names, so pagespeed was trying to fetch non existant images, they were not showing previously because I didn't clean magento cache in months. 

Few images references on CSS files were missing. 

So far it seems this is the reason. 

[Longinos]

Oh! I see....

Perhaps the file versioning?

Maybe related:

https://github.com/apache/incubator-pagespeed-mod/issues/1897

See the comments from ioweb-gr