InPlaceResourceOptimization + ModPagespeedMapRewriteDomain

Nicolás Mercado

unread,

Sep 28, 2015, 2:44:26 PM9/28/15

to mod-pagespeed-discuss

Hi.

I have successfully got ModPagespeedMapRewriteDomain to work so that it modifies the url of all optimized resources in the HTML to point to the CDN domain.

However, when loading pictures via javascript, the domain doesn't get rewritten, leading to pictures being loaded through our main domain (not CDN-distributed).

Any clues on how to make ModPagespeedMapRewriteDomain work?

Here is my current pagespeed.conf:

<IfModule !mod_version.c>
  LoadModule version_module modules/mod_version.so
</IfModule>

<IfVersion < 2.4>
  LoadModule pagespeed_module modules/mod_pagespeed.so
</IfVersion>
<IfVersion >= 2.4.2>
  LoadModule pagespeed_module modules/mod_pagespeed_ap24.so
</IfVersion>

<IfModule pagespeed_module>
  # Turn on mod_pagespeed. To completely disable mod_pagespeed, you
  # can set this to "off".
  ModPagespeed on

  # Direct Apache to send all HTML output to the mod_pagespeed
  # output handler.
  AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html

  # The ModPagespeedFileCachePath and
  # ModPagespeedGeneratedFilePrefix directories must exist and be
  # writable by the apache user (as specified by the User
  # directive).
  ModPagespeedFileCachePath            "/var/mod_pagespeed/cache/"
  ModPagespeedMemcachedServers "localhost:11211"


  # Override the mod_pagespeed 'rewrite level'. The default level
  # "CoreFilters" uses a set of rewrite filters that are generally
  # safe for most web pages. Most sites should not need to change
  # this value and can instead fine-tune the configuration using the
  # ModPagespeedDisableFilters and ModPagespeedEnableFilters
  # directives, below. Valid values for ModPagespeedRewriteLevel are
  # PassThrough, CoreFilters, TestingCoreFilters and AllFilters.
  #
  # ModPagespeedRewriteLevel PassThrough

  # Explicitly disables specific filters. This is useful in
  # conjuction with ModPagespeedRewriteLevel. For instance, if one
  # of the filters in the CoreFilters needs to be disabled for a
  # site, that filter can be added to
  # ModPagespeedDisableFilters. This directive contains a
  # comma-separated list of filter names, and can be repeated.
  #
  # ModPagespeedDisableFilters rewrite_images

  # Explicitly enables specific filters. This is useful in
  # conjuction with ModPagespeedRewriteLevel. For instance, filters
  # not included in the CoreFilters may be enabled using this
  # directive. This directive contains a comma-separated list of
  # filter names, and can be repeated.
  #
  ModPagespeedEnableFilters rewrite_javascript,rewrite_css,rewrite_images,combine_javascript,combine_css,inline_javascript,inline_css,inline_preview_images,extend_cache,collapse_whitespace,elide_attributes,lazyload_images,inline_google_font_css,make_google_analytics_async,convert_jpeg_to_progressive,prioritize_critical_css,rewrite_domains

  ModPagespeedImageRecompressionQuality -1

  # ModPagespeedDomain
  # authorizes rewriting of JS, CSS, and Image files found in this
  # domain. By default only resources with the same origin as the
  # HTML file are rewritten. For example:
  #
  ModPagespeedDomain OUR_MAIN_DOMAIN
  ModPagespeedDomain CDN_DOMAIN
  ModPagespeedDomain cdn.jsdelivr.net
  ModPagespeedDomain fonts.googleapis.com
  ModPagespeedMapRewriteDomain CDN_DOMAIN OUR_MAIN_DOMAIN
  ModPagespeedInPlaceResourceOptimization on
  ModPagespeedFetchHttps enable

  
  #
  # This will allow resources found on http://cdn.myhost.com to be
  # rewritten in addition to those in the same domain as the HTML.
  #
  # Wildcards (* and ?) are allowed in the domain specification. Be
  # careful when using them as if you rewrite domains that do not
  # send you traffic, then the site receiving the traffic will not
  # know how to serve the rewritten content.

  # Other defaults (cache sizes and thresholds):
  #
  # ModPagespeedFileCacheSizeKb          102400
  # ModPagespeedFileCacheCleanIntervalMs 3600000
  # ModPagespeedLRUCacheKbPerProcess     1024
  # ModPagespeedLRUCacheByteLimit        16384
  ModPagespeedCssInlineMaxBytes        16384 
  ModPagespeedImgInlineMaxBytes        151000
  ModPagespeedJsInlineMaxBytes         4096
  # ModPagespeedCssOutlineMinBytes       3000
  # ModPagespeedJsOutlineMinBytes        3000

  # Bound the number of images that can be rewritten at any one time; this
  # avoids overloading the CPU.  Set this to 0 to remove the bound.
  #
  # ModPagespeedImgMaxRewritesAtOnce        8

  # When Apache is set up as a browser proxy, mod_pagespeed can record
  # web-sites as they are requested, so that an image of the web is built up
  # in the directory of the proxy administrator's choosing.  When ReadOnly is
  # on, only files already present in the SlurpDirectory are served by the
  # proxy.
  #
  # ModPagespeedSlurpDirectory ...
  # ModPagespeedSlurpReadOnly on

  # The maximum URL size is generally limited to about 2k characters
  # due to IE: See http://support.microsoft.com/kb/208427/EN-US.
  # Apache servers by default impose a further limitation of about
  # 250 characters per URL segment (text between slashes).
  # mod_pagespeed circumvents this limitation, but if you employ
  # proxy servers in your path you may need to re-impose it by
  # overriding the setting here.  The default setting is 1024
  # characters.
  #
  # ModPagespeedMaxSegmentLength 250

  # Uncomment this if you want to prevent mod_pagespeed from combining files
  # (e.g. CSS files) across paths
  #
  # ModPagespeedCombineAcrossPaths off

  # Whether or not to log timing information for rewriting filters.
  # Off by default to keep logs uncluttered.
  #
  # ModPagespeedLogRewriteTiming on


  # Enables server-side instrumentation and statistics.  If this rewriter is
  # enabled, then each rewritten HTML page will have instrumentation javacript
  # added that sends latency beacons to /mod_pagespeed_beacon.  These
  # statistics can be accessed at /mod_pagespeed_statistics.  You must also
  # enable the mod_pagespeed_statistics and mod_pagespeed_beacon handlers
  # below.
  #
  # ModPagespeedEnableFilters add_instrumentation

  # This handles the client-side instrumentation callbacks which are injected
  # by the add_instrumentation filter.
  # You can use a different location by adding the ModPagespeedBeaconUrl
  # directive; see the documentation on add_instrumentation.
  <Location /mod_pagespeed_beacon>
        SetHandler mod_pagespeed_beacon
  </Location>

  # Uncomment the following line if you want to disable statistics entirely.
  #
  # ModPagespeedStatistics off

  # This page lets you view statistics about the mod_pagespeed module.
  <Location /mod_pagespeed_statistics>
      Order allow,deny
      # You may insert other "Allow from" lines to add hosts you want to
      # allow to look at generated statistics.  Another possibility is
      # to comment out the "Order" and "Allow" options from the config
      # file, to allow any client that can reach your server to examine
      # statistics.  This might be appropriate in an experimental setup or
      # if the Apache server is protected by a reverse proxy that will
      # filter URLs in some fashion.
      Allow from localhost
      SetHandler mod_pagespeed_statistics
  </Location>
</IfModule>

Jeff Kaufman

unread,

Sep 28, 2015, 3:13:15 PM9/28/15

to mod-pagespeed-discuss

You have something like this, right?

<script>
(function () {
var s = document.createElement('script');
s.type = 'text/javascript';
s.async = true;
s.src = '//www.example.com/js/jquery-ui-1.8.16.custom.min.js';
var x = document.getElementsByTagName('script')[0];
x.parentNode.insertBefore(s, x);
})();
</script>

PageSpeed isn't able to substitute cdn.example.com for www.example.com
in js like this because there's not a safe way to figure out what's a
url. So we leave js alone.

You'll need to modify your js yourself if you want it to load things
asynchronously from your cdn. Sorry!

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mod-pagespeed-discuss/e4c740f1-4476-46ad-90c0-046a5b2e7ab0%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Nicolás Mercado

unread,

Sep 28, 2015, 4:15:21 PM9/28/15

to mod-pagespeed-discuss

Well. We use requireJS a lot, so there is a lot of code like the one you describe.

We also have a picture slider that loads pictures on-demand.

For all these resources, PS is not replacing www.mydomain.com with cdn.mydomain.com, like it does with resources loaded by the HTML.

So there is no way I can get requireJS and the slider to work with memcached & minified js, optimized pictures, etc.

Thanks!

Nicolás Mercado

unread,

Sep 28, 2015, 4:16:17 PM9/28/15

to mod-pagespeed-discuss

Sorry. Last line was a question:

So there is no way I can get requireJS and the slider to work with memcached & minified js, optimized pictures, etc ?

Joshua Marantz

unread,

Sep 28, 2015, 4:18:26 PM9/28/15

to mod-pagespeed-discuss

Can you give a few examples of your requireJs usage?

-Josh

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e87c94ea-b7c0-4820-8a1d-3b24e13046e3%40googlegroups.com.

Nicolás Mercado

unread,

Sep 28, 2015, 4:33:45 PM9/28/15

to mod-pagespeed-discuss

Our javascript file in http://mydomain.com/js/myscript.js, uses requireJS to load script on demand.

require(["myscript"], function() {
    alert("My Script loaded successfully. Let's do some stuff with it!");
});

However, myscript.js is not being loaded from the CDN, but from the primary domain.

It's weird because, for example, whitespace removal and minification are working, it's just that the domains are not getting replaced to use the CDN.

Maybe the issue is caused because the requireJS lib is hosted on jsdelivr?

I did set cdn.jsdelivr.net in the allowed domains with "ModPagespeedDomain cdn.jsdelivr.net", but still not working.

Thanks for your help!

El lunes, 28 de septiembre de 2015, 17:18:26 (UTC-3), jmarantz escribió:

Can you give a few examples of your requireJs usage?

-Josh

On Mon, Sep 28, 2015 at 4:16 PM, Nicolás Mercado <nicolas.fern...@gmail.com> wrote:

Sorry. Last line was a question:
So there is no way I can get requireJS and the slider to work with memcached & minified js, optimized pictures, etc ?

El lunes, 28 de septiembre de 2015, 17:15:21 (UTC-3), Nicolás Mercado escribió:
Well. We use requireJS a lot, so there is a lot of code like the one you describe.
We also have a picture slider that loads pictures on-demand.
For all these resources, PS is not replacing www.mydomain.com with cdn.mydomain.com, like it does with resources loaded by the HTML.
So there is no way I can get requireJS and the slider to work with memcached & minified js, optimized pictures, etc.
Thanks!

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

Joshua Marantz

unread,

Sep 28, 2015, 4:42:34 PM9/28/15

to mod-pagespeed-discuss

Is there some reason you can't use:

require(["http://mydomain.com/js/myscript.js"], function() {

alert("My Script loaded successfully. Let's do some stuff with it!");

});

On Mon, Sep 28, 2015 at 4:33 PM, Nicolás Mercado <nicolas.fern...@gmail.com> wrote:

Our javascript file in http://mydomain.com/js/myscript.js, uses requireJS to load script on demand.

require(["myscript"], function() { alert("My Script loaded successfully. Let's do some stuff with it!"); });

However, myscript.js is not being loaded from the CDN, but from the primary domain.
It's weird because, for example, whitespace removal and minification are working, it's just that the domains are not getting replaced to use the CDN.
Maybe the issue is caused because the requireJS lib is hosted on jsdelivr?
I did set cdn.jsdelivr.net in the allowed domains with "ModPagespeedDomain cdn.jsdelivr.net", but still not working.

Thanks for your help!

El lunes, 28 de septiembre de 2015, 17:18:26 (UTC-3), jmarantz escribió:

Can you give a few examples of your requireJs usage?

-Josh

On Mon, Sep 28, 2015 at 4:16 PM, Nicolás Mercado <nicolas.fern...@gmail.com> wrote:

Sorry. Last line was a question:
So there is no way I can get requireJS and the slider to work with memcached & minified js, optimized pictures, etc ?

El lunes, 28 de septiembre de 2015, 17:15:21 (UTC-3), Nicolás Mercado escribió:
Well. We use requireJS a lot, so there is a lot of code like the one you describe.
We also have a picture slider that loads pictures on-demand.
For all these resources, PS is not replacing www.mydomain.com with cdn.mydomain.com, like it does with resources loaded by the HTML.
So there is no way I can get requireJS and the slider to work with memcached & minified js, optimized pictures, etc.
Thanks!

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e87c94ea-b7c0-4820-8a1d-3b24e13046e3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/f494e1e8-25a5-4feb-9ce2-76646804f5ca%40googlegroups.com.

Nicolás Mercado

unread,

Sep 28, 2015, 5:07:56 PM9/28/15

to mod-pagespeed-discuss

I guess you mean 'http://mycdndomain.com/js/myscript.js'.

Doing that leads to the resource being loaded via the CDN but not getting optimized (neither combined, nor minified, nor inlined).

What I want to do is to load the optimized resources from the CDN (which doesn't run PS).

The only way I could make that happen was using ModPagespeedMapRewriteDomain. That way, in the original html, the src url reads: http://mydomain.com/,

PS optimizes the script, then replaces the URL to http://myCDNdomain.com and then the optimized resource is loaded from the CDN (with a miss in the middle). This worked great for resources loaded by the HTML, but doesn't work for resources loaded via JS. URLs don't get replaced and the resource is loaded from the old, slow main server (optimized, but from the slower server)

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e87c94ea-b7c0-4820-8a1d-3b24e13046e3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

Jeff Kaufman

unread,

Sep 29, 2015, 8:18:11 AM9/29/15

to mod-pagespeed-discuss

We might be able to extend PageSpeed to identify these urls to move
them between domains, but we're not going to be able to combine or
inline them. Combining it would mean turning:

require(["a.js"]), function () {
foo();
});
require(["b.js"]), function() {
bar();
});

into

require(["a.js+b.js"], function() {
foo();
bar();
});

and similarly inlining it would mean turning:

require(["a.js"]), function () {
foo();
});

into

[contents of a.js]
foo();

These both seem hard to recognize and apply without breaking pages.

On Mon, Sep 28, 2015 at 5:07 PM, Nicolás Mercado

>>>>> an email to mod-pagespeed-di...@googlegroups.com.

>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/mod-pagespeed-discuss/e87c94ea-b7c0-4820-8a1d-3b24e13046e3%40googlegroups.com.
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "mod-pagespeed-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an

>>> email to mod-pagespeed-di...@googlegroups.com.

>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/mod-pagespeed-discuss/f494e1e8-25a5-4feb-9ce2-76646804f5ca%40googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to mod-pagespeed-di...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/feaa0ca1-fcc1-467b-95a4-3143053be7a5%40googlegroups.com.

Reply all

Reply to author

Forward