mod_pagespeed cache clearing

1,073 views
Skip to first unread message

Dmitrii Kustov

unread,
Nov 25, 2014, 1:35:27 PM11/25/14
to mod-pagesp...@googlegroups.com
Hello. After spending lots of time in searching for the answer to how to clear server-side cache, the only way i was able to find was to do it through manual removal cache folder from server. And that's not cool.

For whatever reason, even cmd sudo touch isn't working, nor pagespeed admin pages purges. it still would load old cached files.

Mighty Internet says adding parameter like '?ver=1' at the end of name of file helps, but that's ridiculous, as well as manually deleting cache folder.

Is it my side bug or pagespeed's? or server-settings?

P.S. using apache 2.2 and latest mozilla

Joshua Marantz

unread,
Nov 25, 2014, 1:44:31 PM11/25/14
to mod-pagespeed-discuss
which version of mod_pagespeed are you running?

Removing the cache folder from the disk is not only uncool, it also will not remove the in-memory copies that PageSpeed keeps or any that are residing in memcached.

-Josh


--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/43f84f59-65f2-4fc6-874d-d1bc1a6d530b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dmitrii Kustov

unread,
Nov 25, 2014, 2:08:01 PM11/25/14
to mod-pagesp...@googlegroups.com
pagespeed version: 1.9.32.2-4321


On Tuesday, November 25, 2014 12:44:31 PM UTC-6, jmarantz wrote:
which version of mod_pagespeed are you running?

Removing the cache folder from the disk is not only uncool, it also will not remove the in-memory copies that PageSpeed keeps or any that are residing in memcached.

-Josh

On Tue, Nov 25, 2014 at 1:35 PM, Dmitrii Kustov <dmi...@hyperlinksmedia.com> wrote:
Hello. After spending lots of time in searching for the answer to how to clear server-side cache, the only way i was able to find was to do it through manual removal cache folder from server. And that's not cool.

For whatever reason, even cmd sudo touch isn't working, nor pagespeed admin pages purges. it still would load old cached files.

Mighty Internet says adding parameter like '?ver=1' at the end of name of file helps, but that's ridiculous, as well as manually deleting cache folder.

Is it my side bug or pagespeed's? or server-settings?

P.S. using apache 2.2 and latest mozilla

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

Joshua Marantz

unread,
Nov 25, 2014, 2:59:32 PM11/25/14
to mod-pagespeed-discuss
What does your pagespeed.conf setting look like?  I just want to make sure you are aware that there's a legacy model of touching a file to clear the whole cache, which is the default.  Then there is a URL purging mechanism that has to be enabled by a pagespeed.conf setting.


-Josh

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/00b65e26-f77d-4fb7-a215-9020f1316a81%40googlegroups.com.

Dmitrii Kustov

unread,
Nov 25, 2014, 3:01:31 PM11/25/14
to mod-pagesp...@googlegroups.com
LoadModule pagespeed_module modules/mod_pagespeed.so
<IfModule pagespeed_module>
    # Turn on mod_pagespeed. To completely disable mod_pagespeed, you
    # can set this to "off".
    ModPagespeed on

    # We want VHosts to inherit global configuration.
    # If this is not included, they'll be independent (except for inherently
    # global options), at least for backwards compatibility.
    ModPagespeedInheritVHostConfig on

    # Direct Apache to send all HTML output to the mod_pagespeed
    # output handler.
    AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html

    # If you want mod_pagespeed process XHTML as well, please uncomment this
    # line.
    # AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER application/xhtml+xml

    # The ModPagespeedFileCachePath directory must exist and be writable
    # by the apache user (as specified by the User directive).
    ModPagespeedFileCachePath            "/var/www/mod_pagespeed/cache/"
    ModPagespeedEnableCachePurge On
    ModPagespeedPurgeMethod PURGE

    # Override the mod_pagespeed 'rewrite level'. The default level
    # "CoreFilters" uses a set of rewrite filters that are generally
    # safe for most web pages. Most sites should not need to change
    # this value and can instead fine-tune the configuration using the
    # ModPagespeedDisableFilters and ModPagespeedEnableFilters
    # directives, below. Valid values for ModPagespeedRewriteLevel are
    # PassThrough, CoreFilters and TestingCoreFilters.
    #
    # ModPagespeedRewriteLevel PassThrough

    # Explicitly disables specific filters. This is useful in
    # conjuction with ModPagespeedRewriteLevel. For instance, if one
    # of the filters in the CoreFilters needs to be disabled for a
    # site, that filter can be added to
    # ModPagespeedDisableFilters. This directive contains a
    # comma-separated list of filter names, and can be repeated.
    #
    # ModPagespeedDisableFilters rewrite_images

    # Explicitly enables specific filters. This is useful in
    # conjuction with ModPagespeedRewriteLevel. For instance, filters
    # not included in the CoreFilters may be enabled using this
    # directive. This directive contains a comma-separated list of
    # filter names, and can be repeated.
    #
    # ModPagespeedEnableFilters rewrite_javascript,rewrite_css
    # ModPagespeedEnableFilters collapse_whitespace,elide_attributes

    # ModPagespeedDomain
    # authorizes rewriting of JS, CSS, and Image files found in this
    # domain. By default only resources with the same origin as the
    # HTML file are rewritten. For example:
    #
      ModPagespeedDomain *.googleapis.com
      ModPagespeedMapOriginDomain "http://fonts.googleapis.com" "https://fonts.googleapis.com"
    #
    # This will allow resources found on http://cdn.myhost.com to be
    # rewritten in addition to those in the same domain as the HTML.
    #
    # Wildcards (* and ?) are allowed in the domain specification. Be
    # careful when using them as if you rewrite domains that do not
    # send you traffic, then the site receiving the traffic will not
    # know how to serve the rewritten content.

    # Other defaults (cache sizes and thresholds):
    #
    # ModPagespeedFileCacheSizeKb          102400
    # ModPagespeedFileCacheCleanIntervalMs 3600000
    ModPagespeedFileCacheCleanIntervalMs  180000
    # ModPagespeedLRUCacheKbPerProcess     1024
    # ModPagespeedLRUCacheByteLimit        16384
    # ModPagespeedCssFlattenMaxBytes       2048
    # ModPagespeedCssInlineMaxBytes        2048
    # ModPagespeedCssImageInlineMaxBytes   2048
    # ModPagespeedImageInlineMaxBytes      2048
    # ModPagespeedJsInlineMaxBytes         2048
    # ModPagespeedCssOutlineMinBytes       3000
    # ModPagespeedJsOutlineMinBytes        3000
    ModPagespeedFetchHttps enable
    # Limit the number of inodes in the file cache. Set to 0 for no limit.
    # The default value if this paramater is not specified is 0 (no limit).
    ModPagespeedFileCacheInodeLimit        500000

    # Bound the number of images that can be rewritten at any one time; this
    # avoids overloading the CPU.  Set this to 0 to remove the bound.
    #
    # ModPagespeedImageMaxRewritesAtOnce      8

    # You can also customize the number of threads per Apache process
    # mod_pagespeed will use to do resource optimization. Plain
    # "rewrite threads" are used to do short, latency-sensitive work,
    # while "expensive rewrite threads" are used for actual optimization
    # work that's more computationally expensive. If you live these unset,
    # or use values <= 0 the defaults will be used, which is 1 for both
    # values when using non-threaded MPMs (e.g. prefork) and 4 for both
    # on threaded MPMs (e.g. worker and event). These settings can only
    # be changed globally, and not per virtual host.
    #
    # ModPagespeedNumRewriteThreads 4
    # ModPagespeedNumExpensiveRewriteThreads 4


    # Settings for image optimization:
    #
    # Jpeg recompression quality (0 to 100, -1 strips metadata):
    # ModPagespeedJpegRecompressionQuality -1
    #
    # Percent of original image size below which optimized images are retained:
    # ModPagespeedImageLimitOptimizedPercent 100
    #
    # Percent of original image area below which image resizing will be
    # attempted:
    # ModPagespeedImageLimitResizeAreaPercent 100

    # When Apache is set up as a browser proxy, mod_pagespeed can record
    # web-sites as they are requested, so that an image of the web is built up
    # in the directory of the proxy administrator's choosing.  When ReadOnly is
    # on, only files already present in the SlurpDirectory are served by the
    # proxy.
    #
    # ModPagespeedSlurpDirectory ...
    # ModPagespeedSlurpReadOnly on

    # The maximum URL size is generally limited to about 2k characters
    # due to IE: See http://support.microsoft.com/kb/208427/EN-US.
    # Apache servers by default impose a further limitation of about
    # 250 characters per URL segment (text between slashes).
    # mod_pagespeed circumvents this limitation, but if you employ
    # proxy servers in your path you may need to re-impose it by
    # overriding the setting here.  The default setting is 1024
    # characters.
    #
    # ModPagespeedMaxSegmentLength 250

    # Uncomment this if you want to prevent mod_pagespeed from combining files
    # (e.g. CSS files) across paths
    #
    # ModPagespeedCombineAcrossPaths off

    # Renaming JavaScript URLs can sometimes break them.  With this
    # option enabled, mod_pagespeed uses a simple heuristic to decide
    # not to rename JavaScript that it thinks is introspective.
    #
    # You can turn this off to let mod_pagespeed rename all JS files.
    ModPagespeedAvoidRenamingIntrospectiveJavascript on

    # Certain common JavaScript libraries are available from Google, which acts
    # as a CDN and allows you to benefit from browser caching if a new visitor
    # to your site previously visited another site that makes use of the same
    # libraries as you do.  Enable the following filter to turn on this feature.
    #
    # ModPagespeedEnableFilters canonicalize_javascript_libraries

    # The following lines configure libraries that are recognized by
    # canonicalize_javascript_libraries.  These will have no effect unless you
    # enable this filter (generally by uncommenting the last line in the
    # previous stanza).  It simply provides a sensible default configuration
    # when the filter is switched on.
    # The format is:
    #    ModPagespeedLibrary bytes md5 canonical_url
    # Where bytes and md5 are with respect to the *minified* JS; use
    # js_minify --print_size_and_hash to obtain this data.
    # Note that we can register multiple hashes for the same canonical url;
    # we do this if there are versions available that have already been minified
    # with more sophisticated tools.
    ModPagespeedLibrary 105527 ltVVzzYxo0 //ajax.googleapis.com/ajax/libs/prototype/1.6.1.0/prototype.js
    ModPagespeedLibrary 92501 J8KF47pYOq //ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js
    ModPagespeedLibrary 141547 GKjMUuF4PK //ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js
    ModPagespeedLibrary 43 1o978_K0_L http://www.modpagespeed.com/rewrite_javascript.js

    # Explicitly tell mod_pagespeed to load some resources from disk.
    # This will speed up load time and update frequency.
    #
    # This should only be used for static resources which do not need
    # specific headers set or other processing by Apache.
    #
    # Both URL and filesystem path should specify directories and
    # filesystem path must be absolute (for now).
    #
    # ModPagespeedLoadFromFile "http://example.com/static/" "/var/www/static/"


    # Enables server-side instrumentation and statistics.  If this rewriter is
    # enabled, then each rewritten HTML page will have instrumentation javacript
    # added that sends latency beacons to /mod_pagespeed_beacon.  These
    # statistics can be accessed at /mod_pagespeed_statistics.  You must also
    # enable the mod_pagespeed_statistics and mod_pagespeed_beacon handlers
    # below.
    #
    # ModPagespeedEnableFilters add_instrumentation

    # The add_instrumentation filter sends a beacon after the page onload
    # handler is called. The user might navigate to a new URL before this. If
    # you enable the following directive, the beacon is sent as part of an
    # onbeforeunload handler, for pages where navigation happens before the
    # onload event.
    #
    # ModPagespeedReportUnloadTime on

    # Uncomment the following line so that ModPagespeed will not cache or
    # rewrite resources with Vary: in the header, e.g. Vary: User-Agent.
    # ModPagespeedRespectVary on

    # This handles the client-side instrumentation callbacks which are injected
    # by the add_instrumentation filter.
    # You can use a different location by adding the ModPagespeedBeaconUrl
    # directive; see the documentation on add_instrumentation.
    <Location /mod_pagespeed_beacon>
          SetHandler mod_pagespeed_beacon
    </Location>

    # Uncomment the following line if you want to disable statistics entirely.
    #
    # ModPagespeedStatistics off

    # This page lets you view statistics about the mod_pagespeed module.
    <Location /mod_pagespeed_statistics>
        Order allow,deny
        # You may insert other "Allow from" lines to add hosts you want to
        # allow to look at generated statistics.  Another possibility is
        # to comment out the "Order" and "Allow" options from the config
        # file, to allow any client that can reach your server to examine
        # statistics.  This might be appropriate in an experimental setup or
        # if the Apache server is protected by a reverse proxy that will
        # filter URLs in some fashion.
        Allow from localhost
        Allow from 127.0.0.1
        SetHandler mod_pagespeed_statistics
    </Location>

    # Uncomment the following line if you want to enable statistics logging.
    # ModPagespeedStatistics is required to be enabled.
    #
    # ModPagespeedStatisticsLogging on
    #
    # The base filename to use to store logged statistics.
    # Required if logging is enabled.
    #
    # ModPagespeedStatisticsLoggingFile "@@MOD_PAGESPEED_STATS_LOG@@"
    #
    # The interval at which statistics will be logged, in milliseconds.
    # Optional; default is 3000.
    #
    # ModPagespeedStatisticsLoggingIntervalMs 3000

    # If both of the below are set, the console will use offline copies of the
    # files needed for the Google Chart Tools API rather than connecting to the
    # Internet to obtain them. This is experimental, as the only supported
    # loading mechanism for the Chart Tools API requires an Internet connexion.
    #
    # Where to find an offline copy of the CSS file required for the Google
    # Chart Tools API. At the time of writing, the Google Chart Tools API CSS
    # file can be found at:
    # https://ajax.googleapis.com/ajax/static/modules/gviz/1.0/core/tooltip.css
    #
    # ModPagespeedStatisticsLoggingChartsCSS http://example.com/charts.css
    #
    # Where to find an offline copy of the JS file required for the Google
    # Chart Tools API. At the time of writing, the Google Chart Tools API JS
    # file can be found at:
    # https://www.google.com/uds/api/visualization/1.0/d7d36793f7a886b687850d2813583db9/format+en,default,corechart.I.js
    #
    # ModPagespeedStatisticsLoggingChartsJS http://example.com/charts.js

    # This page lets you view a graphical console displaying statistics about
    # the mod_pagespeed module.
    <Location /mod_pagespeed_console>
        Order allow,deny
        # This can be configured similarly to mod_pagespeed_statistics above.
        Allow from localhost
        Allow from 127.0.0.1
        SetHandler mod_pagespeed_console
    </Location>
   <Location /pagespeed_admin>
        Order allow,deny
        Allow from localhost
        SetHandler pagespeed_admin
   </Location>

    # Page /mod_pagespeed_message lets you view the latest messages from
    # mod_pagespeed, regardless of log-level in your httpd.conf
    # ModPagespeedMessageBufferSize is the maximum number of bytes you would
    # like to dump to your /mod_pagespeed_message page at one time,
    # its default value is 100k bytes.
    # Set it to 0 if you want to disable this feature.
    ModPagespeedMessageBufferSize 100000

    <Location /mod_pagespeed_message>
        Allow from localhost
        Allow from 127.0.0.1
        SetHandler mod_pagespeed_message
    </Location>
    <Location /mod_pagespeed_referer_statistics>
        Allow from localhost
        Allow from 127.0.0.1
        SetHandler mod_pagespeed_referer_statistics
    </Location>
</IfModule>


Dmitrii Kustov
Software Engineer
Hyperlinks Media, LLC
5150 Franz Road, Suite 100
Katy, Texas 77493

Phone: 281-693-5372
Recycle The World

--
You received this message because you are subscribed to a topic in the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mod-pagespeed-discuss/1R1TLQvWKp8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/CAGKR%2BEDH_St%2BCvYz%3DRL-K2niZsQw9Ezv-ZO7KnEUTnSJdxHWKQ%40mail.gmail.com.

Dmitrii Kustov

unread,
Nov 25, 2014, 3:01:32 PM11/25/14
to mod-pagesp...@googlegroups.com
On Tuesday, November 25, 2014 1:59:32 PM UTC-6, jmarantz wrote:
What does your pagespeed.conf setting look like?  I just want to make sure you are aware that there's a legacy model of touching a file to clear the whole cache, which is the default.  Then there is a URL purging mechanism that has to be enabled by a pagespeed.conf setting.


-Josh
On Tue, Nov 25, 2014 at 2:08 PM, Dmitrii Kustov <dmi...@hyperlinksmedia.com> wrote:
pagespeed version: 1.9.32.2-4321

On Tuesday, November 25, 2014 12:44:31 PM UTC-6, jmarantz wrote:
which version of mod_pagespeed are you running?

Removing the cache folder from the disk is not only uncool, it also will not remove the in-memory copies that PageSpeed keeps or any that are residing in memcached.

-Josh

On Tue, Nov 25, 2014 at 1:35 PM, Dmitrii Kustov <dmi...@hyperlinksmedia.com> wrote:
Hello. After spending lots of time in searching for the answer to how to clear server-side cache, the only way i was able to find was to do it through manual removal cache folder from server. And that's not cool.

For whatever reason, even cmd sudo touch isn't working, nor pagespeed admin pages purges. it still would load old cached files.

Mighty Internet says adding parameter like '?ver=1' at the end of name of file helps, but that's ridiculous, as well as manually deleting cache folder.

Is it my side bug or pagespeed's? or server-settings?

P.S. using apache 2.2 and latest mozilla

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsubscri...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

Joshua Marantz

unread,
Nov 25, 2014, 3:04:25 PM11/25/14
to mod-pagespeed-discuss
So what do you get as a response when you use the CURL command to purge a file?  What do you see in http://localhost/pagespeed_admin/cache#purge_cache

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/c2445d94-a68f-44bd-b70a-54c999bef1e5%40googlegroups.com.

Dmitrii Kustov

unread,
Nov 25, 2014, 5:13:20 PM11/25/14
to mod-pagesp...@googlegroups.com

ok, so, whenever i do curl purge through cmd, it says purge successful. And it works, as well as if i do it in browsers' address line.

whenever i go to that link, i see the same stuff as at pagespeed documentation page

although i catch these errors in my browers JS console log:

The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol. cache

ReferenceError: wa is not defined

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

Dmitrii Kustov

unread,
Nov 25, 2014, 5:22:03 PM11/25/14
to mod-pagesp...@googlegroups.com
So, since i can purge cache through using browers' address, the only question is when google guys gonna fix pagespeed admin cache page, so i can do it by clicking that button, not using get request.

Joshua Marantz

unread,
Nov 25, 2014, 5:24:09 PM11/25/14
to mod-pagespeed-discuss
RE character-encoding errors:  -- I didn't see that in Chrome. Which browser shows you that?

RE: Reference error -- I added https://code.google.com/p/modpagespeed/issues/detail?id=1025  it looks like this prevents the purge UI from working.  This must be a regression, and we don't really have a good automated testing mechanism for the purge UI.  Sorry about that.

When your purged via the curl UI (which is tested with every commit) did that work for you?

-Josh


To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/c10fb62d-6559-4979-a6db-448f8f545f8d%40googlegroups.com.

Dmitrii Kustov

unread,
Nov 26, 2014, 9:12:47 AM11/26/14
to mod-pagesp...@googlegroups.com
Thanks for adding the code issue topic.

As for character encoding error: it shows it to me only at standard firefox console, nor firebug, nor chrome doesnt. so, it might be browser's console thing or something.

Do you mean by curl UI pagespeed admin pages? if yes, then no, it doesnt work for me, that where i get those console errors.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

Joshua Marantz

unread,
Nov 26, 2014, 9:24:11 AM11/26/14
to mod-pagespeed-discuss
What I meant was:

When you execute the PURGE request via CURL, rather than by the UI, does it work?  That is,does it effectively get the old cached resource purged so you serve the new value to clients after you run it?

Again I apologize for releasing the broken UI.  We need to add more manual UI testing of our release packages.

-Josh

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/1778f19a-057d-45d3-825b-574ea21a760e%40googlegroups.com.

Joshua Marantz

unread,
Nov 26, 2014, 12:47:50 PM11/26/14
to mod-pagespeed-discuss
As reported in  https://code.google.com/p/modpagespeed/issues/detail?id=1025 comment 3, a workaround to the admin UI problem is:

You can work around the bug by adding the 'debug' filter to the location blocks for your admin handlers in your pagespeed.conf like this

    <Location /pagespeed_admin>
       Order allow,deny
        Allow from localhost
        Allow from 127.0.0.1
	SetHandler pagespeed_admin
	ModPagespeedEnableFilters debug
    </Location>
    <Location /pagespeed_global_admin>
        Order allow,deny
        Allow from localhost
        Allow from 127.0.0.1
	SetHandler pagespeed_global_admin
	ModPagespeedEnableFilters debug
    </Location>

This avoids the JavaScript error on this page.

We need to understand why the optimized JavaScript is failing.
Reply all
Reply to author
Forward
0 new messages