Upstream cache PURGEs

[Luci]

I am trying to understand how the purges are triggered.

For some reason, I am seeing less than 20% successful purges:

downstream_cache_purge_attempts:                 1856
successful_downstream_cache_purges:               373

Unless the upstream cache expires these pages (which shouldn't be the case), why would mod_pagespeed try to PURGE pages it never served before? I would have thought for a PURGE request to be triggered, mod_pagespeed would first have to serve a version of the page at the first cache miss and then PURGE it when a better version becomes available.

[Anupama Dutta]

Thanks for trying out the caching feature. Purges should not be getting issued for pages that were never served before. One possible reason for the discrepancy you are seeing is that some purges are failing due to incorrect vcl configuration. Could you recheck the purge handling setup in varnish and make sure that you have all of the recommended settings for 1) ACLs for purges, 2) vcl_hit redefinition and 3) vcl_miss redefinition and 4) vcl_recv block for purge handling?

Also, if you could enable info logs in apache for a short duration and check the apache error log, you should find lines starting with "Purge url is" for every attempted PURGE. You could grep out these lines and check if these seem to be pointing to your Varnish server (host:port) correctly. If any of the PURGE requests are failing, the 3 or 4 lines following the "Purge url is" line should indicate if the purge failed due to the connection being refused etc. Could you check and let us know what you find?

Thanks,

Anupama.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 

[Matt Atterbury]

Is it possible that the varnish cache is so small that the pages are being evicted by varnish before the PURGE is sent by mod_pagespeed?

--
"Klaatu barada nikto"                          (754) 444-6288

[Anupama Dutta]

The vcl snippet being suggested for vcl_miss should return a 200 even when the page-to-be-purged is not present in the varnish cache, and these should get counted as successful_purges. So, I don't think that explains why the purge_attempts are so much larger than successful_purges.

Anupama

[Luci]

I'm using nginx not varnish and the cache size should be big enough. This returns 200 for ok and 404 for not found. Could it be different user-agent versions of the same page that are not accounted for separately?

[Anupama Dutta]

Just to confirm, you are using mod_pagespeed with nginx proxy_cache - correct?

If yes, you can check the proxy_cache access_log to see what kind of PURGE URLs are returning 404s. You should be able to grep for whatever ModPagespeedDownstreamCachePurgeLocationPrefix you have used in your config.

If the access log does not help, you can still use the instructions I gave for checking the apache mod_pagespeed logs to understand what is happening to the PURGEs. Re-pasting:

'Also, if you could enable info logs in apache for a short duration and check the apache error log, you should find lines starting with "Purge url is" for every attempted PURGE. You could grep out these lines and check if these seem to be pointing to your Varnish server (host:port) correctly. If any of the PURGE requests are failing, the 3 or 4 lines following the "Purge url is" line should indicate if the purge failed due to the connection being refused etc. Could you check and let us know what you find?'

As long as you are using the same cache key ($ps_capability_list$1$is_args$args) in both the proxy_cache_key and the proxy_cache_purge lines, the User-Agents will get accounted for in the same way. This is because we store and pass along the incoming request headers (with UA) to subsequent PURGE requests for the page (if any).

Thanks,

Anupama.

[Luci]

Yes, nginx proxy_cache.

The PURGE urls are correctly formed and located, so there is no problem there.

I also use the same cache key in both locations, however, it is an extended version of what your sample config suggested - proxy_cache_key "$scheme$host$request_uri$ps_capability_list$1$is_args$args"; Could this be a problem?

Didn't get a chance to do more debugging otherwise...

[Anupama Dutta]

On Thu, Jul 25, 2013 at 7:16 PM, Luci <lu...@aura.travel> wrote:

Yes, nginx proxy_cache.

The PURGE urls are correctly formed and located, so there is no problem there.

I also use the same cache key in both locations, however, it is an extended version of what your sample config suggested - proxy_cache_key "$scheme$host$request_uri$ps_capability_list$1$is_args$args"; Could this be a problem?

As far as I can see, this should not be a problem. Of course, I think $request_uri would be equal to $1, unless you are using a special location matching block which makes these take different values. But, this shouldn't cause any of the purge attempts to fail. In case the other debugging instructions that I gave don't reveal any problems, do share your proxy_cache config and relevant parts of your mod_pagespeed config (if not confidential), so that I can try to figure out other possible reasons.

Thanks for your prompt responses and effort towards identifying any problems with this feature - this is very helpful to us.

Thanks,

Anupama.

[Anupama Dutta]

No. The cache.flush file is for the pagespeed-internal cache only. The only way to make the counts go to 0 would be to restart the server. Alternately, you can just track the deltas from the point at which you do the manual purge for your proxy_cache files, and see whether they are growing disproportionately or not.

On Thu, Jul 25, 2013 at 7:42 PM, Lucian Kafka <lu...@aura.travel> wrote:

I may manually purge everything and start again to see if this is still happening.

Does sudo touch /var/cache/mod_pagespeed/cache.flush clear the 'PURGE' accounting on the mod_pagespeed side?

--

Lucian Kafka AURA Travel - holiday rentals T: 02 8002 5402 | F: 02 8002 5401 www.aura.travel

--
Anupama

[Luci]

Interesting enough I just noticed that the PURGE request gets logged before the page GET request in nginx access_log. I wonder if the cache only caches a page once the socket sub-request is closed, but mod_pagespeed PURGES upstream during the open request?

aura.travel 122.202.67.148 - - [26/Jul/2013:11:18:45 +1000] "PURGE /accommodation/sa/goolwa/cape-house.html HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) mod_pagespeed/1.6.29.5-3346" "-"

aura.travel 180.76.5.183 - - [26/Jul/2013:11:18:45 +1000] "GET /accommodation/sa/goolwa/cape-house.html HTTP/1.1" 200 5528 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-"

[Lucian Kafka]

I may manually purge everything and start again to see if this is still happening.

Does sudo touch /var/cache/mod_pagespeed/cache.flush clear the 'PURGE' accounting on the mod_pagespeed side?

[Nithish Reddy]

I am trying to purge cache keys in nginx:

  location  /invalidateNginxCache/ {

                proxy_pass http://localhost:8000;

                proxy_cache zone_cache;

                proxy_cache_key $arg_url;

                proxy_cache_purge $purge_method;

        }

but it is not working ? can any one give solution?

Request for purge : curl -X PURGE -D – "http://localexample.com/invalidateNginxCache/?url="localexample.com/public.a.css""