Database router with read replica

[Matt Pegler]

I'm hoping for some advice on how to utilize read replica databases with Django.  I searched the django-users archive but didn't find much discussion about this.  I also can't find many blog posts or discussion elsewhere.  Do people have experience or general advice on using read replicas with Django?

I am hoping to implement a database router that will intelligently use a read replica while maintaining consistency despite replica lag.  I have a couple things I'd like another opinion on:

First is handling replication lag.  Our read replica typically lags ~20ms behind master, so we want to route all queries that occur after a write within the same request to the master database.  I believe this can be accomplished using a thread local in the database router that is reset after each request using a middleware.  Does that seem like a reasonable approach?

Second is handling atomic blocks.  As far as I can tell, Django will not automatically route all queries within an atomic block to the same database, so this needs to be handled within the router.  Is connection.in_atomic_block a public API, or is there a better way to tell if we're within an atomic block?

Thanks,

Matt

[Antonis Christofides]

Hi,

When I once faced this problem I used django-multidb-router. It uses the term "pinning" for the fact that after a query to the master database it "pins" itself to the master database, i.e. uses the master database exclusively for the rest of the request. IIRC it works as you say, with a thread-local variable. It wasn't doing everything I wanted, and merging my pull requests didn't go fast enough, but I maintained my own fork which worked fine for the project I was working on.

Even if you decide not to use it, it will give you ideas on how to implement the things you need.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CA%2BSd1We3Yq37fYsZu1rVJcxizty6NNG4q-EeVp83wT7aeuh8pQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.