Which makes more sense?

[Saumya Majumder]

Hi Otto,

few months back when I was started using pagespeed you suggested me to use LoadFromFileMatch as I was running a https site and it is more efficient.

At the same time you have given me the following to add so that pagespeed on works on the following file formats:

pagespeed LoadFromFileRuleMatch disallow .*;
pagespeed LoadFromFileRuleMatch allow \.css$;
pagespeed LoadFromFileRuleMatch allow \.jpe?g$;
pagespeed LoadFromFileRuleMatch allow \.png$;
pagespeed LoadFromFileRuleMatch allow \.gif$;
pagespeed LoadFromFileRuleMatch allow \.js$;

Now in terms of static files there is more than just css, js, jpg, png, gif etc. 

Here are some common static file types:

• Images: png, jpg, svg, gif, tif
• Stylesheets: css
• Javascript: js
• Video and Audio: flv (Flash), mp4 (HTML5 videos), mov (QuickTime), wmv (Windows Media), mp3 and wav
• Web Fonts: eot, ttf, otf, cff, afm, lwfn, ffil, fon, pfm, pfb, woff, svg, std, pro, xsf, and more…
• Other Formats: pdf, doc, docx, ppt, pptx, xls, xlsx, epub, odt, odp, ods, txt, rtf, zip

most of which is neglected by pagespeed as it is only considering css, js, jpg, png, gif. So, I was thinking will this following will be more efficient than the one mentioned above?

pagespeed LoadFromFileRuleMatch disallow \.php$;

rather than allowing just a bunch of file format because if I try to try an allow statement for all static file, it will be a long config file. So, I was wondering whether I just deny php file and allow other static files (which includes all static files).

Q// Why do you want to do that?

The reason I want to do that is because my cdn is now passes through pagespeed config so that all the pagespeed compressed file can be used through cdn. So, now only the files that is in the allow list is allowed to have cdn url. Rest file gets push via the main server. So, I was thinking if I allow all static files but deny php files in that case all of my static files will have cdn url in it.

What do you guys think? 

[Joshua Marantz]

I wonder whether it is feasible for you to put the pagespeed-optimizable resources in a different subdirectory (in the URL scheme and the file path), so you can use a clean 'LoadFromFile' rather than large numbers of regexps.  That would be fastest.  You can imagine that evaluating each URL with over 6 regexes is not free, from a CPU or latency perspective.

I also wanted to dig into what you meant by "my cdn now passes through pagespeed config".  It would be preferable to have your CDN proxy and cache the .pagespeed. resources, filling their cache by pulling from your origin, where ngx_pagespeed can interpret and serve the .pagespeed. URLs.  Having them pass through all the requests to your origin defeats the purpose of a CDN.

-Josh

[Saumya Majumder]

Hi Josh,

the first option will be tricky for me as I'm running a wordpress site. Also you didn't shared your opion on disallowing just php while allowing rest of the static files. Whats your thought on that.

What I meant by cdn passes through pagespeed is that if I want to use cdn along with pagespeed optimized resource, I must need to configure cdn inside the pagespeed conf like we did earlier. If I try to use plugin like CDN Enabler, then all of my resources get passed though cdn properly, but then I loose the magic of pagespeed.

So, again whats your thought on disallowing just php while allowing rest of the static files.

[Otto van der Schaaf]

Whitelisting is safer, but blacklisting can be more convenient. But when you use blacklisting you need to know exactly what you are doing and remember that you set it up like that when making future changes to the website.  You don't want to map files that you rather not have served to the public. Php scripts would be a good example of that. As Josh pointed out, evaluating 6 regular expressions is not free, but perhaps you could make the current blacklisting configuration a bit more efficient, I think that it can be rewritten to a single regex.

I'm not sure that the LoadFromFile configuration is related to what gets rewritten to the cdn though, as you have rewrite_domains enabled. Can you share an example html fragment where content you would have expected to does not get rewritten? 

--
You received this message because you are subscribed to the Google Groups "ngx-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ngx-pagespeed-di...@googlegroups.com.
Visit this group at https://groups.google.com/group/ngx-pagespeed-discuss.
For more options, visit https://groups.google.com/d/optout.

[Saumya Majumder]

Sure Otto, I would be glad to share an example. Let's take a look at this blog post: https://www.isaumya.com/how-to-enable-now-on-tap-with-google-apps-for-work-account/

Now when you scroll down you will see that there are a few video files running through the HTML5 video. Now when you right click on any of that video and do an inspect element, you will see that the mp4 files are getting pushed from my server and not the cdn.

Now now you tell me what you suggest? Keep things as it is or remove the css,js,jpg checks and disallow php simply.

As I mostly work on PHP, there is no other server side file format to disallow. Maybe HTML too if you want. 

Looking forward to your suggestion

[Otto van der Schaaf]

Well, I already shared my thoughts on blacklisting vs whitelisting, and deciding what you want is up to you.

For the m4 files, you may want to check out UrlValuedAttributes:

https://developers.google.com/speed/pagespeed/module/domains#url-valued-attributes

Otto

--

[Otto van der Schaaf]

... sorry, that went out to soon. Next to UrlValuedAttributes, rewriting hyperlinks would also be good to check out as some are referenced in "a href"' attributes:

https://developers.google.com/speed/pagespeed/module/filter-domain-rewrite#configuration

But in this case you do not want to rewrite *all* hyperlinks, so some experimentation with the MapRewriteDoman configuration may be required to get this to work like you want.

Otto 

[Saumya Majumder]

OK, so if I understand you correctly, you want me to do like this:

pagespeed UrlValuedAttribute video;  //This is for the mp4 files 

and maybe wanna tryout:

pagespeed DomainRewriteHyperlinks on; // But this is risky as it will rewrite normal hyperlinks too which are not images or static files

instead of just disallowing php files and allowing the rest.

[Otto van der Schaaf]

I think that modifying your LoadFromFile configuration will not change what gets rewritten to the CDN. Enabling DomainRewriteHyperlinks will, but when you enable that you have to take care because you do not want to rewrite all your hyperlinks to the CDN.

As for the UrlValuedAttribute configuration you posted, that does not seem like it would work to me. In this case, you want the nested "source" tag, and you also need to add the attribute that contains the url value plus how you want PageSpeed to treat its value. For example:

pagespeed UrlValuedAttribute source src hyperlink;

According to the docs I linked: source/src should already be treated as a hyperlink so you should not have to add it. This step is only needed for non-standard attributes (if you have any, and you want them be rewritten to the CDN).

Otto

--