How to record conversions?

[Arvind Kalyan]

Hi, 

I've set up proctor-pipet. I can record which user is assigned what choice of an experiment by logging in the /groups/identity API. How do I record conversions (if the user actually clicked on a button, say) How do you people do it? I was thinking of writing a /convert API, which is POSTed from the client side, with the action of the user. Is there some kind of inbuilt mechanism that I missed?

Thanks!

-- Arvind

[Jack Humphrey]

Proctor is fairly agnostic about how you record user actions that you want to measure. Pipet provides the /groups/identify API endpoint so that you can retrieve the group assignment data and use it to control behavior and log user actions.

At Indeed, we have lots of different log events, and we include group assignment in those log events. We then correlate and process those logs into Imhotep (https://github.com/indeedeng/imhotep) datasets, which give us the ability to analyze things like conversions.

A simplified example:

1. A user views a job. Based on the user's identifiers, we determine that she is assigned to bucket 2 in the "applybuttontst" group. We show a variation on the Apply button as a result.

2. We log event "viewjob" with lots of metadata, including a tracking key for the event and the groups. Included in the groups metadata is the string "applybuttontst1" showing her membership in the group.

3. User clicks the Apply button. We log event "applyclick", again with plenty of metadata, including the tracking key of the preceding "viewjob" event.

4. We have a Hadoop job that builds an Imhotep dataset about "viewjob" events. Part of that job is to correlate "applyclick" events into that dataset.

5. Using IQL, we can then do things like query "from viewjob yesterday today group by grp in 'applybuttontst0,applybuttontst1' select apply/count()" to compare apply rates in control vs. test.

Your idea of doing the logging with a remote endpoint ("/convert") is definitely feasible -- the important thing is that you have the ability to tie your events together for analysis.

Jack

[Arvind Kalyan]

Thanks Jack!