Serf status 120171 (SSL communications related errors)

[Quinn Comendant]

We just migrated an existing configuration to a new server (centos6 → cents 7), and now are getting "Serf status 120171" errors. According to here [1] it means "SSL communications related errors". Here is the full error:

[Sun Jun 17 08:04:36.874056 2018] [pagespeed:error] [pid 11647] [mod_pagespeed 1.13.35.2-0 @11647] Serf status 120171(APR does not understand this error code) polling for 1 threaded fetches for 0.05 seconds

And here is a bunch of log entries from error_log and access_log sorted by time to show any related queries (although I don't see any correlation): <https://pastebin.com/raw/B6wDSGDK>

We are using mod_proxy together with mod_pagespeed to send requests to a back-end server using HTTPS.

Here's our config files:
- pagespeed.conf: <https://pastebin.com/FxisRgHT>
- VirtualHost with PageSpeed settings: <https://pastebin.com/Tq663k6g>

Here is the SSL certificate info of the back-end server from `openssl s_client … | openssl x509 …`: <https://pastebin.com/raw/L6GLLZvs> Anyways, the Qualys SSL Server Test gives https://www-origin.bikehugger.com an "A" rating, so I don't think the back-end server SSL setup is a problem (I could check if the proxy server doesn't support the used ciphers, but I doubt it).

I've tried setting FetchHttps to permissive modes, like this:

ModPagespeedFetchHttps enable,allow_self_signed,allow_unknown_certificate_authority,allow_certificate_not_yet_valid

I am using:

ModPagespeedRespectXForwardedProto on

The errors go away if I add this:

ModPagespeedMapOriginDomain "http://www-origin.bikehugger.com" "https://www.bikehugger.com"

(NB: I'm confused why mod_pagespeed is giving SSL errors if it is mod_proxy that is actually making the HTTPS requests to the back-end server. Or does modmod_pagespeed make its own requests too for src files? I guess so.)

Is there any way to get more details what the "SSL communications related" error is?

Quinn


[1] https://www.jefftk.com/p/serf-error-codes

[Otto van der Schaaf]

Fetch timed out: https://www.bikehugger.com/wp-content/uploads/2017/09/Exploro-dropped-chainstay-1024x768.jpg (connecting to:10.128.0.5:80) (1) waiting for 50 ms

Hmmm, https on port 80?

The module fetches its input css/js/css via http(s).
I think you need to authorize the https domain explicitly to bypass the loop back fetching that occurs by default. You need it because of respectxforwardedproto being turned on.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/20180617153600433423.88f868f0%40strangecode.com.
For more options, visit https://groups.google.com/d/optout.

[Quinn Comendant]

Hi Otto,

> Fetch timed out:
> https://www.bikehugger.com/wp-content/uploads/2017/09/Exploro-dropped-chainstay-1024x768.jpg
> (connecting to:10.128.0.5:80) (1) waiting for 50 ms

Indeed! https on 80‽

I was confused by this line for several reasons:

1. Why is mod_pagespeed requesting an asset from www.bikehugger.com? (It should request from www-origin.bikehugger.com.)

2. The error says "connecting to:10.128.0.5:80", but WHY? Doing a `curl -v` to https://www.bikehugger.com/… from there clearly shows a connection to 35.226.33.96 on port 443:

Trying 35.226.33.96...
Connected to www.bikehugger.com (35.226.33.96) port 443 (#0)
[…]

3. The IP 10.128.0.5 is the internal IP of the pagespeed proxy, essentially == localhost. No domains resolve to this IP. Why is mod_pagespeed sending a request there???

4. Why is it only that request timed out, out of the hundreds of requests possible at that moment? (There's many other images requested from https://www.bikehugger.com/…)

> you need to authorize the https domain explicitly to bypass the loop back fetching that occurs by default

Can you explain how to do that?

Thanks!

Quinn

[Otto van der Schaaf]

By default the module connects back to the inbound ip/port that was used, which is the one your proxy used in your case.

Also, I believe respectxforwarded proto isn’t automatically compensated for when turned on and effective for fetches initiated on behalf of a html response. Maybe we should issue a warning on that in the logs when these fetches fail.

In any case, to bypass loop back fetching, you can explicitly authorize a domain for fetching: https://www.modpagespeed.com/doc/domains#auth_domains

Otto

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/20180617162025500393.bb821574%40strangecode.com.

[Quinn Comendant]

On Sun, 17 Jun 2018 13:04:34 +0200, Otto van der Schaaf wrote:
> In any case, to bypass loop back fetching, you can explicitly
> authorize a domain for fetching:
> https://www.modpagespeed.com/doc/domains#auth_domains

Yes, clearly that was needed. Now I have this and it seems to be working:

ModPagespeed On
ModPagespeedDomain https://www.bikehugger.com
ModPagespeedMapOriginDomain https://www-origin.bikehugger.com *.bikehugger.com
ModPagespeedMapRewriteDomain www.bikehugger.com www-origin.bikehugger.com
ModPagespeedEnableFilters [...]

Anything you see missing?

Quinn