Implementing A/B Testing in Django

[John Boxall]

I've got a new home page and I'd like to test it's performance in
getting users to sign up compared to my existing home page. In Django
terms I have two templates that I would like my existing view to
alternate between - I want to make whatever template the viewer sees
"sticky" - they see the same template if they come to the page again -
Also I want to measure how many users reach the conversion page - both
these requirements will require access to the request's session
object.

So basically I need to dynamically alter templates for existing views
and record the relevant information on the session (and store the
results somewhere)

A couple ideas I've gone through:

1) Using a process_view Middleware to dynamically insert a
template_name argument into views. This would seem to work but the
function would have to accept a template_name parameter and not alter
it inside to work. Also wouldn't work with FlatPages (or any other
view that doesn't accept a template_name parameter).

2) Using a template_loader to dynamically load templates. This awesome
because it would work with anything that uses template loaders. The
problem with this is it doesn't seem like there is any time when you
have access to the session object after the template_loader decides on
the template name. You could always hack it onto a thread locals but
I'd rather now.


Does anyone have any other ideas or suggestions about how to
dynamically show templates to users in a pluggable way while at the
same time measuring what template the user sees?

Cheers,

John

[TiNo]

Does anyone have any other ideas or suggestions about how to
dynamically show templates to users in a pluggable way while at the
same time measuring what template the user sees?

It's not about doing it in Django, but you know that Google Website Optimizer does this, and provides you out of the box with goal-conversions, and all other pretty analytical stuf?

[John Boxall]

Thanks Tino,

I am aware of GWO - just thought it would be a cool project to try!

It looks like the local threads / request object available in the
template loader trick is going to work - just gotta clear up some
possible caching issues then it'll be ready for release.

Cheers,

John

[John Boxall]

Using template_loader idea I've developed a little pluggable app for
conducting basic A/B tests. You can grab it here:

http://github.com/johnboxall/django-ab/tree/master

John