Pagespeed better upstream or downstream of Varnish?

[Callum Macdonald]

We're running Magento on top of PHP / Apache. Then we added mod_pagespeed. Finally, we cache and serve all that with Varnish.

Magento is helluva slow. We currently cache the pagespeed optimised output with Varnish. That's mostly because pagespeed is an Apache module, so that's where it fit in our stack.

I'm now wondering if we'd be better to add Varnish between Magento and pagespeed. We could cache the unoptimised Magento output in Varnish, then use a second Apache site to apply the pagespeed optimisations.

As I see the pros / cons:

+ Browser specific optimisations are all handled by pagespeed, simpler Varnish logic

+ No chance of partially optimised pages in Varnish cache

- Pagespeed would be repeatedly optimising the same pages, redundant work

- Two requests every time, client -> pagespeed -> Varnish

Another side effect is that Pagespeed would cache it's own optimised .pagespeed.hash urls. Not sure if this is a good thing or not. The advice on the downstream-caching docs[1] says let pagespeed cache these urls to avoid cache bloat. We have plenty of memory, and I wonder if Varnish is better at caching these assets than pagespeed.

My gut reaction is that we'd get a simpler setup with minimally slower performance. Any other opinions on the list? What have I missed / mistaken in the reasoning?

Cheers - Callum.

[1] https://developers.google.com/speed/pagespeed/module/downstream-caching

[Joshua Marantz]

I like your analysis.  I suspect the optimal answer depends on your traffic and your site.

If you have huge amounts of traffic and your site changes very little, then having Varnish keep optimized versions for each UA-class makes a lot of sense to me.  There's no question that Varnish can serve HTML responses from its cache faster than PageSpeed can parse and reserialize the HTML.  That would favor having Varnish between PageSpeed and the user.

However if you have small amounts of traffic and your site changes constantly, then the Varnish hit-rate might too low due to the UA-class fragmentation, and you wouldn't save PageSpeed that much work.  And we've worked hard to try to minimize the CPU overhead of re-optimizing HTML on each request.

I think the only way to know the best answer is to experiment based on your site.  And if you do such experiments, then we'd love to hear more about what you learn!

I was not sure what you meant, exactly, by "Pagespeed would cache it's own optimised .pagespeed.hash urls".  I think it will do that in either case.

-Josh

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/d8db5b52-0eea-42b5-9117-1cd0419dd1eb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Callum Macdonald]

We have low traffic, infrequent page changes. We artificially warm our Varnish caches to keep 90% of our pageviews always in the cache. We can afford to keep all 20k pages, with 30 variants of each, in Varnish. The challenge is building the infrastructure to crawl all those pages and periodically refresh them!

 

I was not sure what you meant, exactly, by "Pagespeed would cache it's own optimised .pagespeed.hash urls".  I think it will do that in either case.

I meant that with pagespeed between Varnish and client, the .pagesped.hash. urls would never reach Varnish, they'd come directly from pagespeed's own cache. I thought we were currently serving these from Varnish, but we're not, we're actually using pagespeed to rewrite them all onto a CDN with pagespeed as its upstream, so this is a moot point.

Thanks for the feedback. I'll follow up if we decide to change our current architecture. The decision is put the UA fragmentation into Varnish, or put pagespeed in front of Varnish.

Cheers - Callum.