Understanding "domain" configuration

67 views
Skip to first unread message

Jose Luis Moya

unread,
Feb 4, 2022, 12:29:09 PMFeb 4
to mod-pagespeed-discuss
Hi. We've been suing mod_pagespeed for a while but I am now trying to optimize it a bit further and I am rather confuse regarding domain configuration.

In one of our setups we use modPageSpeed in a NGINX proxy server that connects to an Apache upstream . To improve deliverability of static content we serve a path "/ass" via a NGINX static server to improve speed and also as a pseudo CDN.

Being so https://our.domain.com/ass would upstream to our backend
While https://cdn.domain.com/ass would be served directly by a nginx server from the same location

We have modPageSpeed configured only at our.domain.com working correctly.
We now tried to add to its config:

pagespeed Domain https://cdn.domain.com/ass/;

Thinking that this would allow modPageSpeed to also rewrite this content but we are still getting at the console:


Is the logic of what I am trying to do wrong or am I missing something?
Do I need modpagespeed also activated at the cdn.domain.com nginx config?





Longinos

unread,
Feb 5, 2022, 6:34:40 AMFeb 5
to mod-pagespeed-discuss
Hi
Whitout a whole picture of the config is difficult to say.....
As a first glance these error must not happen if all is ok, but a 404 must be returned if pagespeed is not running at cdn.domain.com.
Think if a rewrited resource (some like https://cdn.domain.com/ass/skins/def/ico/apple-touch-icon-152x152.png.ic.pagespeed.HASH.png ) comes to the server and there is no pagespeed runnig here , the server must return a 404 NOT FOUND.
Have you enabled the admin pages?
At the bottom of the page you can see a Domain Layer section, here are the AUTH domains.

Maybe you have put the pagespeed Domain directive in the wrong place.

pagespeed Domain whateverdomain.com is set to allow pagespeed to rewrite resources of whateverdomain.com present in the html code otherwise they are not rewrited.
And whateverdomain.com MUST be capable to serve a pagespeed rewrited url aka pagespeed runs on it.

If no pagespeed Domain is present, pagespeed only rewrites the HTTP (not the HTTPS) resources from the domain that serves the html code.

Joshua Marantz

unread,
Feb 5, 2022, 11:29:52 AMFeb 5
to mod-pagesp...@googlegroups.com
On Sat, Feb 5, 2022 at 6:34 AM Longinos <longinos.int...@gmail.com> wrote:
Hi
Whitout a whole picture of the config is difficult to say.....
As a first glance these error must not happen if all is ok, but a 404 must be returned if pagespeed is not running at cdn.domain.com.
Think if a rewrited resource (some like https://cdn.domain.com/ass/skins/def/ico/apple-touch-icon-152x152.png.ic.pagespeed.HASH.png ) comes to the server and there is no pagespeed runnig here , the server must return a 404 NOT FOUND.

This is true for the origin server. However, if a CDN is acting as a reverse proxy and using an upstream with PageSpeed running, this should work fine; the server acting as the CDN does not have to be running PageSpeed to proxy PageSpeed-rewritten resources.

As Longinos said -- it's probably easier to diagnose the issue if you have more details about your topology.

Jose Luis Moya

unread,
Feb 5, 2022, 2:44:42 PMFeb 5
to mod-pagespeed-discuss
Thank you both for the reply. I'll try to clarify a bit further.

Same NGINX server running:
1.- proxy server upstream to internal 10.0.0.111 serving correctly my.pagespeed_domain.com 
HTML there contains source links to https://cdn.domain.com/ass resources

Pagespeed working fine configured like:

pagespeed Domain https://cdn.domain.com/ass;


2.- standard NGINX server for https://cdn.domain.com (currently not running pagespeed)

By the comments I understand I should get PageSpeed also running at cdn.domain.com or else pagespeed would never rewrite that contents at my.pagespeed_domain.com

Thanks.

JL

Joshua Marantz

unread,
Feb 5, 2022, 3:27:16 PMFeb 5
to mod-pagespeed-discuss
How does https://cdn.domain.com/ find it's content? Does it store locally on the file system? Or is it configured as a proxy?

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/b6ff41bf-b09c-4cd7-a870-ae3818223297n%40googlegroups.com.

LMC

unread,
Feb 6, 2022, 11:49:25 AMFeb 6
to mod-pagesp...@googlegroups.com
cdn.domain.com is on the same nginx proxy server but configured as a standard directory server, no upstream. 
JL


You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/B5lLUa-GgsU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAGKR%2BEB1rskmr6rXUfNGrTGM%2BJiqUS9ZK_rhUXNf_ngfy7caBA%40mail.gmail.com.

Joshua Marantz

unread,
Feb 6, 2022, 12:10:49 PMFeb 6
to mod-pagesp...@googlegroups.com
OK this explains it. You either need to:
or
  • configure cdn.domain.com as a reverse proxy (typical of CDN setups) where it fetches content from upstream, even if that upstream is running on the same physical host.
ngx_pagespeed should work fine with either choice.


Longinos

unread,
Feb 7, 2022, 3:56:58 AMFeb 7
to mod-pagespeed-discuss
I think I got it the first time: cdn is a file server that takes its resources directly from the file system so pagespeed need to run on it,
but the error should be 404 not the "No permision to rewrite" if the pagespeed Domain cdn.domain.com is set in my.pagespeed_domain.com.

LMC

unread,
Feb 7, 2022, 5:01:03 AMFeb 7
to mod-pagesp...@googlegroups.com
Yes correct. That was my initial guess and the reason for the question.

I'll test enabling it at the cdn domain too but I was expecting that being that on the same server as the proxy and enabled as a Domain var on the config, there should be no need.
I could want to "pagespeed it" when served from my.pagespeed_domain.com but maybe not when accesed direct. 

Thanks. 

JL



Longinos

unread,
Feb 8, 2022, 5:53:46 AMFeb 8
to mod-pagespeed-discuss
Ummm sme clarifycation:
If you serve html from my.pagespeed_domain.com that contain urls from cdn.mydomain.com
and in the pagespeed config on my.pagespeed_domain.com you have pagespeed Domain cdn.mydomain.com;

Then the urls from cdn.mydomain.com in the html served by my.pagespeed_domain.com got rewrited and cdn.mydomain.com
must be able to serve it.

what happens when this request comes to cdn.mydomain.com? If pagespeed is not running here a 404 error must be served cause cdn.mydomain.com know notthing about rewrited
resources.

This can be accomplissed by setting pagespped on; in each virtual host (each server block in nginx) or puttin it on the http block and then it is on in all server blocks.
To make pagespeed work on cdn.mydomain.com only to request comming from my.pagespeed_domain.com, you need to set pagespeed on; in eache server block and in the
cdn.mydomain.com some logic to put it  on and off testing the host part of the request, some like:
In the http block:
map $http_host $name {
          my.pagespeed_domain.com on;
         default unplugged;
}
in the cdn.mydomain.com server block
pagespeed $name;

But it this needed?

Jose Luis Moya

unread,
Feb 8, 2022, 1:53:13 PMFeb 8
to mod-pagespeed-discuss
Hi again... still not there I am afraid. This are the approx steps taken:

I've activated a different pagespeed config at my cdn.domain.com  pagespeed on and a different set of filters.
Verified by accessing: 
This shows correctly (active at that domain) at the config side:

Domain Lawyer 

I've then added to the main domain of my.pagespeed_domain.com the cdn domain.
Accessing https://my.pagespeed_domain.com/pagespeed_admin again  I see at the config display:

Domain Lawyer 
 https:// my.pagespeed_domain.com/ Auth OriginDomain:http://10.0.0.111/

But still getting on the message history of the main domain (same messages on both consoles actually):

[Tue, 08 Feb 2022 18:48:03 GMT] [Info] [12017] No permission to rewrite 'https://cdn.domain.com/gis/ass/skins/def/ico/apple-touch-icon-152x152.png'

Giving up for now but thought that would be the correct procedure.
Thanks.




Longinos

unread,
Feb 9, 2022, 3:53:13 AMFeb 9
to mod-pagespeed-discuss
Hi
If the message is in both consoles then pagespeed in cdn.domain.com is not working.
Can you post the nginx config? with anonimized IP and domain names.

Jose Luis Moya

unread,
Feb 9, 2022, 6:02:37 AMFeb 9
to mod-pagespeed-discuss
Not easy as this is a complex setup but I'll try to brief it related to pagespeed:

nginx.conf
        include /etc/nginx/al/pagespeed/core.conf;

core.conf
pagespeed off; 
# CACHE
pagespeed FileCachePath /tmp/ngx_pagespeed;
pagespeed FileCacheSizeKb            302400;
pagespeed FileCacheCleanIntervalMs   3600000;
pagespeed FileCacheInodeLimit        500000;
pagespeed LRUCacheKbPerProcess     1024;
pagespeed LRUCacheByteLimit        16384;

## PageSpeed admin pages configuration
pagespeed UsePerVhostStatistics on;

pagespeed Statistics on;
pagespeed StatisticsLogging on;

Then at server level my.pagespeed_domain.com (proxied)
pagespeed on; 

location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
        add_header "" "";
}
location ~ "^/pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon$" { }

include al/pagespeed/admin_ui.conf;

client_body_buffer_size 64k;
#filters activated
pagespeed EnableFilters responsive_images;
pagespeed EnableFilters convert_to_webp_lossless;
pagespeed EnableFilters insert_image_dimensions;
pagespeed EnableFilters recompress_images;
pagespeed EnableFilters recompress_jpeg;
pagespeed EnableFilters recompress_png;
pagespeed EnableFilters recompress_webp;
pagespeed EnableFilters convert_gif_to_png;
pagespeed EnableFilters strip_image_color_profile;
pagespeed EnableFilters strip_image_meta_data;
pagespeed EnableFilters resize_images;
pagespeed EnableFilters inline_preview_images;
pagespeed EnableFilters collapse_whitespace;
pagespeed EnableFilters extend_cache;
pagespeed EnableFilters extend_cache_css;
pagespeed EnableFilters extend_cache_images;
pagespeed EnableFilters extend_cache_scripts;
pagespeed EnableFilters sprite_images;
pagespeed EnableFilters convert_meta_tags;
pagespeed EnableFilters insert_dns_prefetch;
pagespeed SupportNoScriptEnabled false;


An at server level of  cdn.domain.com (direct)
pagespeed on; 

location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
        add_header "" "";
}
location ~ "^/pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon$" { }

include al/pagespeed/admin_ui.conf;

client_body_buffer_size 64k;

pagespeed Domain https://cdn.domain.com/gis;

#filters activated
pagespeed EnableFilters responsive_images;
pagespeed EnableFilters convert_to_webp_lossless;
pagespeed EnableFilters insert_image_dimensions;
pagespeed EnableFilters recompress_images;
pagespeed EnableFilters recompress_jpeg;
pagespeed EnableFilters recompress_png;
pagespeed EnableFilters recompress_webp;
pagespeed EnableFilters convert_gif_to_png;
...


Thanks

Longinos

unread,
Feb 10, 2022, 4:58:33 AMFeb 10
to mod-pagespeed-discuss
Hi
If these are your config then you need to add
pagespeed Domain https://cdn.domain.com/gis;
In the my.pagespeed_domain.com server block.
As I have posted in a email before, pagespeed by default is able to rewrite resources from the http domain from where the html is served, if you want resources
from other domains be rewrited, you must auth these domain in the domain from where the html is served.

So the config must be like:

in my.pagespeed_domain.com server block:
pagespeed Domain https://cdn.domain.com/gis;

in cdn.domain.com/gis server block:
pagespeed Domain https://cdn.domain.com/gis;
That´s is needed because the https.

Normally I use
pagespeed Domain http*://*.domain.com;
This cover http and https, and all subdomains belongs to domain.com
but maybe is not suitable for yours needs

Jose Luis Moya

unread,
Feb 10, 2022, 2:35:44 PMFeb 10
to mod-pagespeed-discuss
Hi. I missed that on the transcription of the config of my last email, sorry. It is actually exactly like what you suggested with the exception of the http* part.  This is, I am also including "pagespeed Domain https://cdn.domain.com/gis;" into the main domain, as well as now on the cdn.domain.com/gis domain.

Not working. I'll give it another try but as you mentioned the fact that I am seeing the same console log messages on both pagespeed admin pages looks confusing.
Thanks.






Longinos

unread,
Feb 10, 2022, 7:13:01 PMFeb 10
to mod-pagespeed-discuss
Hi
What you can see in the htm code when you make a request like
This put debug messages as html comments

Jose Luis Moya

unread,
Jul 6, 2022, 7:58:27 AMJul 6
to mod-pagespeed-discuss
I reopen this thread as I never managed to get this to work and now trying again.
I really don't know what else to try on this case or what we might be doing wrong.
As explained before currently

Domain Lawyer 

Config (as referred at https://cdn.domain.com/pagespeed_admin/ conf:
Domain Lawyer 


On eithers message history console:
[Wed, 06 Jul 2022 11:20:52 GMT] [Info] [13994] No permission to rewrite 'https://cdn.domain.com/def/css/vendor/bootstrap.min.css'


As a reply to last comment by @longinos
URLs gets rewritten without a param, served normally.

Any help of suggestion of what else to test would be mosty appreciated.
Thanks.

Jose Luis Moya

unread,
Jul 6, 2022, 11:51:20 AMJul 6
to mod-pagespeed-discuss
Finally found our problem that was related to a disallow rule that was too ample and affecting the domain rule. We were including this rules in all the pagespeed config files.

This is as for the example above a disallow rule of type */def/* was the one causing the "no permission error. 

Thanks.
Reply all
Reply to author
Forward
0 new messages