Adding SASL support to Pylibmc backend

264 views
Skip to first unread message

Walter Scheper

unread,
Nov 22, 2011, 11:20:09 AM11/22/11
to django-d...@googlegroups.com
Hello devs,

A few weeks ago, I began converting a django based web site for
deployment on the hosting service Heroku. As part of this process, I
looked into how to configure Memcached on Heroku and found that they
require that you access memecached through a binary, SASL
authenticated interface. With some more research I found that pylibmc
added support for SASL authentication in the 1.2.0 release[1], which
Django supports. I also found a ticket, #15815[2], adding support for
the binary protocol (which is also required for SASL authentication).
However, there seems to be some question as to how the pylibmc
specific options should be handled in the CACHES setting.

I think the most consistent form of the CACHES setting for this would look like:
"""
CACHES = {
'default': {
'BACKEND' : 'django.core.cache.backends.memcached.PyLibMCCache',
'LOCATION' : '127.0.0.1:11211',
'OPTIONS' : {
'BINARY': True, # optional, as pylibmc automatically sets
binary=true if SASL authentication is used
'USERNAME': 'user',
'PASSWORD': 'secret',
'BEHAVIORS': dict(tcp_nodelay=True, ketama=True),
},
},
}
"""

Then fairly simple change to the _cache method of PylibMCCache backend
will enable both of these features, though I'm unsure what level of
tests django would require before accepting such a patch. Also should
django inform the user if the version of pylibmc does not support the
username and password arguments to the Client __init__ method, or just
raise the raw exception?

If there's general support for adding this feature the PylibMCCache
backend, and a consensus around the CACHES setting, then I will open a
new ticket and generate a patch.

Thanks,
Walter

1. https://github.com/lericson/pylibmc#readme
2. https://code.djangoproject.com/ticket/15815

Jason Mayfield

unread,
Nov 22, 2011, 7:38:19 PM11/22/11
to Django developers
It's funny you posted this today, as I was just about to post
something similar, for the same genesis: Heroku.

When installing pylibmc, it wants to compile, which blows up because
of missing headers in the standard Heroku environment. However, they
are in the process of testing an update to their Python support such
that if they detect that you are loading pylibmc, they will bootstrap
libmemcached for you and everything is gravy, at least as far as
compiling goes.

Of course, you still need to have a SASL client to talk to the single
memcached provider currently on Heroku, so you also need to use a
custom backend for it. I'd written one myself, wrapping Django's
PylibMCCache backend and overriding the _cache method to pass in the
username and password parameters (which on Heroku are provided as
environment variables once you turn on the memcache add-on). One of
the guys at Heroku has also written that support into a fork of the
external django-pylibmc module as well.

What all this blather is leading up to is my support (such as it is)
for the idea of including the username and password parameters in the
PylibMCCache client. I'd be happy to assist Walter in doing so.

Although to be devil's advocate, I've basically just proven that we
don't strictly *need* this change to core (given the existence of the
external backend). Plus, of course, since Heroku is on EC2, we can
also use other external memcache servers using the vanilla client; for
example, Amazon's ElasticCache.

Best,
Jason

Reply all
Reply to author
Forward
0 new messages