Integrate Optimizely with Snowplow

[Jeffrey Lek]

Hi Snowplowers,

We do AB testing through Optimizely and we try to analyze these tests as much as possible. In Universal Analytics we've set up the plugin for Optimizely, sending the testing details to custom dimensions.

I'd like to do the same using Snowplow, but since a standard plugin isn't avaible (yet) I'm having trouble doing this. Is there anyone here with a solution for this issue? I'm using GTM as well and I was already thinking of passing some info through the dataLayer, but I haven't found a good way to do this. Any help would be great.

Thanks,

Jeffrey

[Robert Kingston]

Hi Jeffrey,

We also use Optimizely and other tools in Snowplow to great effect. For a simple test I've found the easiest approach is to fire a structured event with the following details:

Category: experiment

Action: {{test name}}

Label: {{test treatment}}

You can do this with this dataLayer push we created off the back of Optimizely's JS API, just remember to update the experiment ID variable for the test you're using:

var experimentId = 23333431896;

if (window['optimizely'].variationMap.hasOwnProperty(experimentId)) {
    var test = window['optimizely'].data.experiments[experimentId].name;
    var treatment = window['optimizely'].data.state.variationNamesMap[experimentId];
    if (test && treatment) {
        window['dataLayer'].push({
            'event': 'experiment running',
            'splitTestName': test,
            'splitTestVariation': treatment
        });
    }
}

I won't cover the GTM setup since it sounds like you've got this part sorted.

Alternative approaches are:

Unstructured events for each exposure

We use this for tests where we want to send additional data through from other split testing tools. It's usually overkill for most tests though.

In-test contexts with each event after exposure

We haven't tried this but I suppose it would also be possible to attach a context to each event with the experiments a user is in. I think this is unnecessary and will only increase your collection/storage costs for no real benefit. 

Would love to hear your thoughts on the above.

Cheers,

Rob

[Yali Sassoon]

Hi Jeffrey,

We generally recommend that you capture A/B testing data as custom context. That means you can capture all the A/B testing data with every event that occurs when a test is live, making it easy to analyse the data later.

In the case of Optimizely in particular, you have their Data Object available so it is really a question of :

1. Figuring out what data from that object you want to pass into Snowplow. You could potentially pass the complete object, but this can be an enormous amount of data. At the other extreme, Rob suggests passing in just the experimentId and the variation.
   
2. Create a custom context schema (and associated jsonpath file and sql table definition) for the contextual data that you want to track with each event. (You'd need to have Iglu setup.)
   
3. Configure variables in GTM to populate with the relevant values from the Optimizely data object
4. Update your Snowplow tracking code in GTM to spot if those variable values have been set, and if so, to send the additional custom context with any Snowplow events that are fired.

As an aside - we would like to build out-of-the-box integration with Optimizely - so it becomes a switch you set when you initialize the Javascript tracker, that would then automatically grab the relevant data points out of the Optimizely data object. In order to do this, however, we need to understand what data points Optimizely users want to record from that data object - if there's a consensus, then building out this support should be reasonably straightforward. So my question is what data points would you like? If the differnet snowplow users on this chain agree, we can proceed to build out that functionality.

All the best,

Yali

--
You received this message because you are subscribed to the Google Groups "Snowplow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snowplow-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Co-founder
Snowplow Analytics
The Roma Building, 32-38 Scrutton Street, London EC2A 4RQ, United Kingdom
+44 (0)203 589 6116
+44 7841 954 117
@yalisassoon

[Ty Wilson]

Glad that others are thinking in the same space.  We do AB testing primarily with internal tools, but also with Optimizely.  We follow a very similar approach to Rob by grabbing (1) the test name and (2) the variant/treatment in a structured event.

At analysis time we stitch this data to our sessions aggregate table, so that we can track all behavior for sessions in that test condition.  Yali, I can see how custom contexts would make this easier but its not something that we have set up currently.

Best,

Ty

[jba...@kapow.com]

I am very interested in this topic as well.  I am still getting snowplow set up but have used optimizely before as well as other 3rd party and internal solutions and recognize the benefits of being able to analyze session impact after the fact without pre-defining each specific goal would be super helpful.

[Alex Dean]

Great - we hope to be able to share some news on this soon!

Alex

--
You received this message because you are subscribed to the Google Groups "Snowplow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snowplow-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Co-founder
Snowplow Analytics
The Roma Building, 32-38 Scrutton Street, London EC2A 4RQ, United Kingdom

+44 (0)203 589 6116
+44 7881 622 925
@alexcrdean