How does Wagtail work with Django's CACHES setting?

[Tim Allen]

I have looked through the documentation which has plenty of information on Wagtail's front-end cache invalidation, but my team isn't quite to the level of needing Varnish yet.

I'm curious how Wagtail works with Django's CACHES setting. I noted that Redis is recommended, but regardless if you're using Redis or memcached or even database caching, I'm wondering how Wagtail interacts with Django's CACHES.

Is any Wagtail model class that inherits from Page cached? Are they automagically invalidated when an edit occurs? Is there anything I need to do to get Wagtail to use Django's CACHES, rather than running database queries each time a page is viewed? Since there aren't necessarily user defined Views with Wagtail, I'm a little bit lost without @cache_page. 

Thanks for any information you can provide.

Regards,

Tim

[Tobias McNulty]

Hi, Tim!

Good question. I wasn't sure about this myself so I did a little digging and here's what I found:

1. Django's manual caching facility is used for two things in Wagtail: the sitemap and the root URL for sites, so there is some benefit to setting CACHES. There may be other reasons, I'm not sure.
   
2. As you suggest, using cache_page doesn't really seem possible because there's not a simple way (that I can see) to wrap as_view(), as suggested here. The possible exception to this would be copying wagtailcore.urls into your project and wrapping the "serve" view with cache_page().
3. What may also be possible (and I did some basic testing with) is to use the per-site cache, i.e., cache middleware. This won't achieve exactly what you're looking for because there's no invalidation, but for high-traffic sites where you want to avoid the complexity of Varnish or something similar and can afford a small delay in updates (say, 30-60 seconds) this might just do the trick. In local testing this eliminated all but the SQL query to retrieve the Site for me, and the cache middleware should respect many of the same headers that a dedicated frontend cache would.

I'm curious if the per-site cache would work for you. If not, it may be possible for page-level caching to be be added to Wagtail itself, or at least made a little easier for site implementors to add themselves.

Let us know what you end up doing!

Cheers,

Tobias

--
You received this message because you are subscribed to the Google Groups "Wagtail support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wagtail+unsubscribe@googlegroups.com.
To post to this group, send email to wag...@googlegroups.com.
Visit this group at https://groups.google.com/group/wagtail.
To view this discussion on the web, visit https://groups.google.com/d/msgid/wagtail/8b214ec0-573c-4ca2-93a3-9c511b1bd97b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tobias McNulty
Chief Executive Officer

tob...@caktusgroup.com
www.caktusgroup.com

[Tim Allen]

Thanks for the digging, Tobias! I've started looking into this myself, since a site-wide cache won't really work for us. Basically, I'm trying to find the middle ground of eliminating database calls by dumping to memcached / redis through the Django cache, without having to cache the entire site.

I've got a fairly complex Wagtail set up (index pages made up of sub pages, snippets, and such) that might prove a good proving ground for this. Over holiday break, I may start digging in and see if this would be easy enough to wire up as an option on a per-page basis.

Regards,

Tim

On Wednesday, December 14, 2016 at 5:53:15 PM UTC-5, Tobias McNulty wrote:

Hi, Tim!

Good question. I wasn't sure about this myself so I did a little digging and here's what I found:

1. Django's manual caching facility is used for two things in Wagtail: the sitemap and the root URL for sites, so there is some benefit to setting CACHES. There may be other reasons, I'm not sure.
   
2. As you suggest, using cache_page doesn't really seem possible because there's not a simple way (that I can see) to wrap as_view(), as suggested here. The possible exception to this would be copying wagtailcore.urls into your project and wrapping the "serve" view with cache_page().
3. What may also be possible (and I did some basic testing with) is to use the per-site cache, i.e., cache middleware. This won't achieve exactly what you're looking for because there's no invalidation, but for high-traffic sites where you want to avoid the complexity of Varnish or something similar and can afford a small delay in updates (say, 30-60 seconds) this might just do the trick. In local testing this eliminated all but the SQL query to retrieve the Site for me, and the cache middleware should respect many of the same headers that a dedicated frontend cache would.

I'm curious if the per-site cache would work for you. If not, it may be possible for page-level caching to be be added to Wagtail itself, or at least made a little easier for site implementors to add themselves.

Let us know what you end up doing!

Cheers,

Tobias

On Wed, Dec 14, 2016 at 4:04 PM, Tim Allen <fli...@peregrinesalon.com> wrote:

I have looked through the documentation which has plenty of information on Wagtail's front-end cache invalidation, but my team isn't quite to the level of needing Varnish yet.

I'm curious how Wagtail works with Django's CACHES setting. I noted that Redis is recommended, but regardless if you're using Redis or memcached or even database caching, I'm wondering how Wagtail interacts with Django's CACHES.

Is any Wagtail model class that inherits from Page cached? Are they automagically invalidated when an edit occurs? Is there anything I need to do to get Wagtail to use Django's CACHES, rather than running database queries each time a page is viewed? Since there aren't necessarily user defined Views with Wagtail, I'm a little bit lost without @cache_page. 

Thanks for any information you can provide.

Regards,

Tim

--
You received this message because you are subscribed to the Google Groups "Wagtail support" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wagtail+u...@googlegroups.com.

To post to this group, send email to wag...@googlegroups.com.
Visit this group at https://groups.google.com/group/wagtail.
To view this discussion on the web, visit https://groups.google.com/d/msgid/wagtail/8b214ec0-573c-4ca2-93a3-9c511b1bd97b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sævar Öfjörð Magnússon]

Hi Tim

I've used the cache template tag with success. You just have to be careful with your cache key so that the cache is invalidated when something is changed.

See https://docs.djangoproject.com/en/1.10/topics/cache/#template-fragment-caching for more information.

An example that works for many scenarios:

base.html

{% load cache %}

{% cache 300 base request.path request.user.username self.pk self.latest_revision_created_at|date:"c" %}
{% block content %}
...
{% endblock %}
{% endcache %}

The first parameter is the cache timeout in seconds. The rest of the parameters are there to ensure that each page is unique in the cache, and also unique for logged in users.

Please note that previews do not work in this scenario if the preview has been cached already. In that case I've added a timestamp to the serve_preview method so that each preview is unique.

Of course, then you would have to add this to all your pages, or have them inherit some BasePage model.

from datetime import datetime

class BasePage(Page):

    class Meta:

        abstract = True

    def serve_preview(self, request, mode_name):

        request.preview_timestamp = datetime.now()

        return super(BasePage, self).serve_preview(request, mode_name)

class HomePage(BasePage):
    ... rest of your page code

and in base.html 

{% cache 300 base request.path request.user.username self.pk self.latest_revision_created_at|date:"c" request.preview_timestamp %}

Hope this helps.

- Sævar

[Tim Allen]

Thank you Sævar, that's a great idea. I had been digging in from the other end - from views - and hadn't considered the options at the template layer. I think this option will do what we need well enough for now!

Happy holidays, everyone. Regards,

Tim

--
You received this message because you are subscribed to a topic in the Google Groups "Wagtail support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wagtail/SZzBmwf_q0U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wagtail+unsubscribe@googlegroups.com.

To post to this group, send email to wag...@googlegroups.com.
Visit this group at https://groups.google.com/group/wagtail.

To view this discussion on the web, visit https://groups.google.com/d/msgid/wagtail/4a25c953-24ed-4853-8221-623353977721%40googlegroups.com.