Feature request? Priming Varnish cache - issue purge requests prior to prime requests

[Geraint Michael]

Hi.

I have moved from a simple priming script using CURL to OCP, for a moderately busy university website.  The cache in use is Varnish (3.0.6).  

OCP has been a great improvement for priming, but due to the volume of content, number of editors and frequency of updates we're still a way off a perfect solution.

At present we run one process to purge (large ~3k) sets of pages from the cache (from a generated sitemap, using the varnish client), and then we run OCP to prime (from the same sitemap).  These processes are run every 15 minutes, if a trigger is present (trigger dropped on updates/publish, with different triggers and sitemaps for different sets of pages).

What strikes me is that OCP could issue the purge request (with PURGE method, eg: curl -X PURGE http://domain.com/page) as well as issuing the request that primes the page - this would save running the first process and would result in a much tighter prime process - pages would only be purged just before they are primed, which would lead to a better experience for the audience.

I know it's very cache specific and that it would slow OCP down if used, but varnish is excellent so extending support for it may be no bad thing..  what do you think?  

I'd be very happy to elaborate or discuss further - the current setup seems to be frustrating for both administrators and the visitors and I'm stuck trying to time the clockwork to keep everyone happy..

Gez.

[Patrick Mylund Nielsen]

Hi Gez,

Interesting. Looking at some different Varnish examples, it seems the method naming is arbitrary, so what do you think about a command line option like this?

  --methods "method1,method2,..."   default: "GET"     Comma-separated list of HTTP methods to perform (in order) on each URL being primed. For use with e.g. Varnish or Squid ("PURGE,GET")

--
You received this message because you are subscribed to the Google Groups "Optimus Cache Prime" group.
To unsubscribe from this group and stop receiving emails from it, send an email to optimus-cache-p...@googlegroups.com.
To post to this group, send email to optimus-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/optimus-cache-prime.
For more options, visit https://groups.google.com/d/optout.

[Geraint Michael]

Yes, a sequential list of methods would seem like a good way to approach it.

That said - I have had a thought: I could use a OCP with a custom user-agent and have varnish determine this as a purge request (rather than using the request method), moreover varnish could then return a lookup, creating a new cached version - all done in one single OCP run with varnish doing the work.

--
You received this message because you are subscribed to a topic in the Google Groups "Optimus Cache Prime" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/optimus-cache-prime/rt2MLcrNiYU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to optimus-cache-p...@googlegroups.com.

To post to this group, send email to optimus-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/optimus-cache-prime.
For more options, visit https://groups.google.com/d/optout.

--

----------------------------------------------------------------------------------------------------

Geraint Michael - Web Developer & Timelapse Photographer

tel: 07810543858

photo: omnimatic.co.uk

web: monomatic.co.uk

----------------------------------------------------------------------------------------------------