Understanding "domain" configuration

[Jose Luis Moya]

Hi. We've been suing mod_pagespeed for a while but I am now trying to optimize it a bit further and I am rather confuse regarding domain configuration.

In one of our setups we use modPageSpeed in a NGINX proxy server that connects to an Apache upstream . To improve deliverability of static content we serve a path "/ass" via a NGINX static server to improve speed and also as a pseudo CDN.

Being so https://our.domain.com/ass would upstream to our backend

While https://cdn.domain.com/ass would be served directly by a nginx server from the same location

We have modPageSpeed configured only at our.domain.com working correctly.

We now tried to add to its config:

pagespeed Domain https://cdn.domain.com/ass/;

Thinking that this would allow modPageSpeed to also rewrite this content but we are still getting at the console:

[Info] [31619] No permission to rewrite 'https://cdn.domain.com/ass/skins/def/ico/apple-touch-icon-152x152.png'

Is the logic of what I am trying to do wrong or am I missing something?

Do I need modpagespeed also activated at the cdn.domain.com nginx config?

[Longinos]

Hi

Whitout a whole picture of the config is difficult to say.....

As a first glance these error must not happen if all is ok, but a 404 must be returned if pagespeed is not running at cdn.domain.com.

Think if a rewrited resource (some like https://cdn.domain.com/ass/skins/def/ico/apple-touch-icon-152x152.png.ic.pagespeed.HASH.png ) comes to the server and there is no pagespeed runnig here , the server must return a 404 NOT FOUND.

Have you enabled the admin pages?

Take a look at https://our.domain.com/pagespeed_admin/config

At the bottom of the page you can see a Domain Layer section, here are the AUTH domains.

Maybe you have put the pagespeed Domain directive in the wrong place.

pagespeed Domain whateverdomain.com is set to allow pagespeed to rewrite resources of whateverdomain.com present in the html code otherwise they are not rewrited.

And whateverdomain.com MUST be capable to serve a pagespeed rewrited url aka pagespeed runs on it.

If no pagespeed Domain is present, pagespeed only rewrites the HTTP (not the HTTPS) resources from the domain that serves the html code.

[Joshua Marantz]

On Sat, Feb 5, 2022 at 6:34 AM Longinos <longinos.int...@gmail.com> wrote:

Hi

Whitout a whole picture of the config is difficult to say.....

As a first glance these error must not happen if all is ok, but a 404 must be returned if pagespeed is not running at cdn.domain.com.

Think if a rewrited resource (some like https://cdn.domain.com/ass/skins/def/ico/apple-touch-icon-152x152.png.ic.pagespeed.HASH.png ) comes to the server and there is no pagespeed runnig here , the server must return a 404 NOT FOUND.

This is true for the origin server. However, if a CDN is acting as a reverse proxy and using an upstream with PageSpeed running, this should work fine; the server acting as the CDN does not have to be running PageSpeed to proxy PageSpeed-rewritten resources.

As Longinos said -- it's probably easier to diagnose the issue if you have more details about your topology.

[Jose Luis Moya]

Thank you both for the reply. I'll try to clarify a bit further.

Same NGINX server running:

1.- proxy server upstream to internal 10.0.0.111 serving correctly my.pagespeed_domain.com 

HTML there contains source links to https://cdn.domain.com/ass resources

Pagespeed working fine configured like:

pagespeed MapOriginDomain http://10.0.0.111 https://my.pagespeed_domain.com ;

pagespeed Domain https://cdn.domain.com/ass;

2.- standard NGINX server for https://cdn.domain.com (currently not running pagespeed)

this produces at my.pagespeed_domain.com/pagespeed_admin/

[Info] [31619] No permission to rewrite 'https://cdn.domain.com/ass/skins/def/ico/apple-touch-icon-152x152.png'

By the comments I understand I should get PageSpeed also running at cdn.domain.com or else pagespeed would never rewrite that contents at my.pagespeed_domain.com

Thanks.

JL

[Joshua Marantz]

How does https://cdn.domain.com/ find it's content? Does it store locally on the file system? Or is it configured as a proxy?

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/b6ff41bf-b09c-4cd7-a870-ae3818223297n%40googlegroups.com.

[LMC]

cdn.domain.com is on the same nginx proxy server but configured as a standard directory server, no upstream. 

JL

You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/B5lLUa-GgsU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAGKR%2BEB1rskmr6rXUfNGrTGM%2BJiqUS9ZK_rhUXNf_ngfy7caBA%40mail.gmail.com.

[Joshua Marantz]

OK this explains it. You either need to:

• enable ngx_pagespeed on the virtual-host running cdn.domain.com

or

• configure cdn.domain.com as a reverse proxy (typical of CDN setups) where it fetches content from upstream, even if that upstream is running on the same physical host.

ngx_pagespeed should work fine with either choice.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAEaxWA3tmes5%2Bt%3DD_AV5Ks%2B%2Brk%3D2K6z_yyS6tAcZena97CwmCw%40mail.gmail.com.

[Longinos]

I think I got it the first time: cdn is a file server that takes its resources directly from the file system so pagespeed need to run on it,

but the error should be 404 not the "No permision to rewrite" if the pagespeed Domain cdn.domain.com is set in my.pagespeed_domain.com.

[LMC]

Yes correct. That was my initial guess and the reason for the question.

I'll test enabling it at the cdn domain too but I was expecting that being that on the same server as the proxy and enabled as a Domain var on the config, there should be no need.

I could want to "pagespeed it" when served from my.pagespeed_domain.com but maybe not when accesed direct. 

Thanks. 

JL

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/fa2850fe-1b1b-42f7-a8d5-fe8b48063c5fn%40googlegroups.com.

[Longinos]

Ummm sme clarifycation:

If you serve html from my.pagespeed_domain.com that contain urls from cdn.mydomain.com

and in the pagespeed config on my.pagespeed_domain.com you have pagespeed Domain cdn.mydomain.com;

Then the urls from cdn.mydomain.com in the html served by my.pagespeed_domain.com got rewrited and cdn.mydomain.com

must be able to serve it.

Think: If you have some like https://cdn.mydomain.com/xsome-image.jpg.ic.pagespeed.some-hash-value.webp in the html served by my.pagespeed_domain.com,

what happens when this request comes to cdn.mydomain.com? If pagespeed is not running here a 404 error must be served cause cdn.mydomain.com know notthing about rewrited

resources.

This can be accomplissed by setting pagespped on; in each virtual host (each server block in nginx) or puttin it on the http block and then it is on in all server blocks.

To make pagespeed work on cdn.mydomain.com only to request comming from my.pagespeed_domain.com, you need to set pagespeed on; in eache server block and in the

cdn.mydomain.com some logic to put it  on and off testing the host part of the request, some like:

In the http block:

map $http_host $name {

          my.pagespeed_domain.com on;

         default unplugged;

}

in the cdn.mydomain.com server block

pagespeed $name;

But it this needed?

[Jose Luis Moya]

Hi again... still not there I am afraid. This are the approx steps taken:

I've activated a different pagespeed config at my cdn.domain.com  pagespeed on and a different set of filters.

Verified by accessing: 

https://cdn.domain.com/pagespeed_admin

This shows correctly (active at that domain) at the config side:

Domain Lawyer 

 https://cdn.domain.com/gis/

I've then added to the main domain of my.pagespeed_domain.com the cdn domain.

Accessing https://my.pagespeed_domain.com/pagespeed_admin again  I see at the config display:

Domain Lawyer 

 http://10.0.0.111/ 

 https://cdn.domain.com/gis/ Auth 

 https:// my.pagespeed_domain.com/ Auth OriginDomain:http://10.0.0.111/

But still getting on the message history of the main domain (same messages on both consoles actually):

[Tue, 08 Feb 2022 18:48:03 GMT] [Info] [12017] No permission to rewrite 'https://cdn.domain.com/gis/ass/skins/def/ico/apple-touch-icon-152x152.png'

Giving up for now but thought that would be the correct procedure.

Thanks.

[Longinos]

Hi

If the message is in both consoles then pagespeed in cdn.domain.com is not working.

Can you post the nginx config? with anonimized IP and domain names.

[Jose Luis Moya]

Not easy as this is a complex setup but I'll try to brief it related to pagespeed:

nginx.conf

        include /etc/nginx/al/pagespeed/core.conf;

core.conf

pagespeed off; 
# CACHE
pagespeed FileCachePath /tmp/ngx_pagespeed;
pagespeed FileCacheSizeKb            302400;
pagespeed FileCacheCleanIntervalMs   3600000;
pagespeed FileCacheInodeLimit        500000;
pagespeed LRUCacheKbPerProcess     1024;
pagespeed LRUCacheByteLimit        16384;

## PageSpeed admin pages configuration
pagespeed UsePerVhostStatistics on;

pagespeed Statistics on;
pagespeed StatisticsLogging on;

Then at server level my.pagespeed_domain.com (proxied)

pagespeed on; 

location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
        add_header "" "";
}
location ~ "^/pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon$" { }

include al/pagespeed/admin_ui.conf;

client_body_buffer_size 64k;

pagespeed MapOriginDomain http://10.0.0.111 https://my.pagespeed_domain.com;

#filters activated
pagespeed EnableFilters responsive_images;

pagespeed EnableFilters convert_to_webp_lossless;
pagespeed EnableFilters insert_image_dimensions;
pagespeed EnableFilters recompress_images;
pagespeed EnableFilters recompress_jpeg;
pagespeed EnableFilters recompress_png;
pagespeed EnableFilters recompress_webp;
pagespeed EnableFilters convert_gif_to_png;
pagespeed EnableFilters strip_image_color_profile;
pagespeed EnableFilters strip_image_meta_data;
pagespeed EnableFilters resize_images;
pagespeed EnableFilters inline_preview_images;
pagespeed EnableFilters collapse_whitespace;
pagespeed EnableFilters extend_cache;
pagespeed EnableFilters extend_cache_css;
pagespeed EnableFilters extend_cache_images;
pagespeed EnableFilters extend_cache_scripts;
pagespeed EnableFilters sprite_images;
pagespeed EnableFilters convert_meta_tags;
pagespeed EnableFilters insert_dns_prefetch;
pagespeed SupportNoScriptEnabled false;

An at server level of  cdn.domain.com (direct)

pagespeed on; 

location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
        add_header "" "";
}
location ~ "^/pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon$" { }

include al/pagespeed/admin_ui.conf;

client_body_buffer_size 64k;

pagespeed Domain https://cdn.domain.com/gis;

#filters activated
pagespeed EnableFilters responsive_images;

pagespeed EnableFilters convert_to_webp_lossless;
pagespeed EnableFilters insert_image_dimensions;
pagespeed EnableFilters recompress_images;
pagespeed EnableFilters recompress_jpeg;
pagespeed EnableFilters recompress_png;
pagespeed EnableFilters recompress_webp;
pagespeed EnableFilters convert_gif_to_png;

...

Thanks

[Longinos]

Hi

If these are your config then you need to add

pagespeed Domain https://cdn.domain.com/gis;

In the my.pagespeed_domain.com server block.

As I have posted in a email before, pagespeed by default is able to rewrite resources from the http domain from where the html is served, if you want resources

from other domains be rewrited, you must auth these domain in the domain from where the html is served.

So the config must be like:

in my.pagespeed_domain.com server block:

pagespeed MapOriginDomain http://10.0.0.111 https://my.pagespeed_domain.com;

pagespeed Domain https://cdn.domain.com/gis;

in cdn.domain.com/gis server block:

pagespeed Domain https://cdn.domain.com/gis;

That´s is needed because the https.

Normally I use

pagespeed Domain http*://*.domain.com;

This cover http and https, and all subdomains belongs to domain.com

but maybe is not suitable for yours needs

[Jose Luis Moya]

Hi. I missed that on the transcription of the config of my last email, sorry. It is actually exactly like what you suggested with the exception of the http* part.  This is, I am also including "pagespeed Domain https://cdn.domain.com/gis;" into the main domain, as well as now on the cdn.domain.com/gis domain.

Not working. I'll give it another try but as you mentioned the fact that I am seeing the same console log messages on both pagespeed admin pages looks confusing.

Thanks.

[Longinos]

Hi

What you can see in the htm code when you make a request like

https://my.pagespeed_domain/?PageSpeedFilters=+debug

This put debug messages as html comments

[Jose Luis Moya]

I reopen this thread as I never managed to get this to work and now trying again.

I really don't know what else to try on this case or what we might be doing wrong.

As explained before currently

Config (as referred at https://my.pagespeed_domain/pagespeed_admin/ conf:

Domain Lawyer 

http://cdn*.domain.com/ Auth 

 http://cdn.domain.com/ Auth 

 https://cdn.domain.com/ Auth

Config (as referred at https://cdn.domain.com/pagespeed_admin/ conf:

Domain Lawyer 

 http://cdn.domain.com/ Auth 

 https://cdn.domain.com/ Auth 

On eithers message history console:

[Wed, 06 Jul 2022 11:20:52 GMT] [Info] [13994] No permission to rewrite 'https://cdn.domain.com/def/css/vendor/bootstrap.min.css'

As a reply to last comment by @longinos

URLs gets rewritten without a param, served normally.

Any help of suggestion of what else to test would be mosty appreciated.

Thanks.

[Jose Luis Moya]

Finally found our problem that was related to a disallow rule that was too ample and affecting the domain rule. We were including this rules in all the pagespeed config files.

This is as for the example above a disallow rule of type */def/* was the one causing the "no permission error. 

Thanks.