VirtualHost pagespeed settings issues & HTTPS fetch failed errors

[Ben Lam]

"MPS" refers to "mod_pagespeed"

Observations

While setting up MPS on a test server I noticed 2 particular issues (unrelated): 

1. VirtualHost settings are overwritten by pagespeed.conf (due to order of Includes in httpd.conf)
   
2. HTTPS fetch appears broken in MPS 1.12.34.2-0 / Apache 2.4.27, built from source

Questions

1. Could the virtual host section of the documentation use some improvement / clarification?
1. In particular, a note about order of Includes pagespeed.conf vs. Includes httpd-vhosts.conf directives in httpd.conf
2. Is HTTPSFetch broken in MPS 1.12.34.2-0?

From trial and error I've learned:

• most MPS settings put into httpd-vhosts.conf, like in the example vhost documentation, will be overwritten and ignored because:
• when MPS is installed, pagespeed.conf is included at the end of httpd.conf
• Apache's own inclusion of virtual hosts configuration (Include /etc/apache2/extra/httpd-vhosts.conf) is located above pagespeed's inclusion in httpd.conf, i.e. httpd-vhosts.conf is read before pagespeed.conf and most (all?) MPS settings inside httpd-vhosts.conf will be overwritten by those inside pagespeed.conf
• not sure why global settings are put in httpd-vhosts.conf in the example vhosts config.  Adding any lines outside the context of a particular <VirtualHost> directive means it'll be included into the global httpd.conf (if I understand correctly)
• even if a VirtualHost based MPS configuration is correct, having httpd-vhosts.conf included above/before pagespeed.conf in httpd.conf will lead to MPS not optimizing anything and there won't be any errors in Apache error.log since it's not doing anything

• in pagespeed.conf, setting ModPagespeed off, will shut off MPS for the entire Apache server, regardless of any ModPagespeed on lines included inside <VirtualHost> directives in httpd-vhosts.conf
• if you want to run MPS for only 1 virtualhost, you'll need to have MPS "on" globally, and include ModPagespeed off in each <VirtualHost> you want excluded

• https support for fetching static files for in-place rewriting seems broken on Apache 2.4.27 with MPS 1.12.34.2-0, (both built from source on Ubuntu 16.04LTS)
• See below for sample error
• Specifying IP address in the <VirtualHost> directive (i.e. <VirtualHost 173.255.236.40:443>) had no effect
• For example, the below block, included inside of a <VirtualHost> block, where the ModPagespeedLoadFromFile line is commented out, "Fetch failed" errors will populate the apache error log (error sample below)

    <IfModule pagespeed_module>

        ModPagespeed on

        AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html

        ModPagespeedFileCachePath "/var/cache/mod_pagespeed/"

        ModPagespeedLogDir "/var/log/pagespeed"

#       ModPagespeedLoadFromFile "https://parisbytrain.com/wp-content/uploads/" "/var/apache2/wp40-ml-parisbytrain.com/www/wp-content/uploads/"

        ModPagespeedSslCertDirectory "/etc/letsencrypt/live/parisbytrain.com"

        ModPagespeedSslCertFile "/etc/letsencrypt/live/parisbytrain.com/cert.pem"

    </IfModule>

Errors when relying on HTTPS to fetch origin files

[Fri Oct 13 16:14:10.266223 2017] [pagespeed:error] [pid 10019] [mod_pagespeed 1.12.34.2-0 @10019] https://parisbytrain.com/wp-content/uploads/2008/05/rer_map.jpg (connecting to:173.255.236.40):0: serf_context_run error status=20014 (Internal error (specific information not available))

[Fri Oct 13 16:14:10.266340 2017] [pagespeed:warn] [pid 10019] [mod_pagespeed 1.12.34.2-0 @10019] Fetch failed to start: https://parisbytrain.com/wp-content/uploads/2008/05/rer_map.jpg (connecting to:173.255.236.40)

• When ModPagespeedLoadFromFile  is used ('#' comment removed), MPS has no problems retrieving origin files and serving optimized images that are rewritten in HTML

To reiterate questions above:

1. Could the virtual host section of the documentation use some improvement / clarification?
1. In particular, a note about order of Includes pagespeed.conf vs. Includes httpd-vhosts.conf directives in httpd.conf
2. Is HTTPSFetch broken in MPS 1.12.34.2-0?

Thanks

[Otto van der Schaaf]

Re: https broken in 1.12.34.2

It looks like ModPagespeedSslCertDirectory and ModPagespeedSslCertFile need to be reconfigured to point to a directory containing the files for the trusted certificate authorities instead of the site certificates. I don't think https fetching is broken.

See https://www.modpagespeed.com/doc/https_support

Re: order of includes

I thought directives in virtualhost sections would override any global configuration regardless of declaration order. Your observations make me think that something odd is happening with "ModPagespeed on|off"'s processing. Could you file an issue for this over at https://github.com/pagespeed/mod_pagespeed/issues, preferably with a minimal configuration to reproduce?

Otto

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e850e4ac-47df-4fec-88b9-bd6c370d6359%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Lam]

Thanks Otto.

You're correct on the ModPagespeedSslCertDirectory needing the Certificate Authority directory, not the directory of your domain's keys.

I've submitted a pull request to re-iterate differences between CA directory and one's own domain SSL keys in the below mentioned block of documentation about HTTPS support.

I re-read the documentation several times and looked at the ModPagespeedSslCertDirectory settings several times and kept wondering "How am I supposed to tell MPS which key file to use?"  It simply didn't register that MPS is looking for something as generic as Certificate Authority keys as opposed to keys for my actual domain.  Intuitively, it didn't even occur to me MPS needs CA keys in order to perform HTTPS negotiation as a HTTPS client.

Apparently, I'm not alone.  The first two issues results for a Google search on the phrase "modpagespeed ModPagespeedSslCertDirectory" show each person making the same mistake, thinking the ModPagespeedSslCertDirectory is supposed to point to the directory housing their domain's SSL key, not the CA directory. (Example 1, Example 2).

Regarding the order of pagespeed.conf inclusion in httpd.conf and its effect on httpd-vhosts.conf MPS directives, I've created an issue:

https://github.com/pagespeed/mod_pagespeed/issues/1651

Thanks for your help,

Ben

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.