Interpreting different types of predictor variables

[Paul Mainwood]

Hi there,

First, thank you very much for making such a powerful and user-friendly package freely available (CausalImpact and bsts).  Hugely valuable to those trying to apply state space models without that much technical support.

I have a conceptual question about  interpretation, when you use different types of predictor variables.

Let's say I want to understand whether some promotional materials in a retail store help sell a branded product.  (e.g., some banners, samplers of the product, and so on).

Case 1: I have a bunch of stores that are similar to my test stores, but didn't have the intervention.  I can use sales in these stores as predictor variables -- this is gold standard, and looks like putting a lot of (helpful) mathematical formalism around a simple A/B test.  I say helpful, mainly because the package will weight the sales from the stores that behave most like the target store via the spike/slab approach, which is an advance on a simple comparison of the two classes.

Case 2: I don't have any stores available that don't have the promotional activity, but I do have a variety of variables like the following: Price series (inversely correlated with sales), High visibility feature space occupied by the product (positively correlated with sales), temperature (positively correlated).  And these are all independent of the promotional activity.

Case 3: I have both the "hold-out" stores and these additional variables available, and use both as predictor variables.

To me, case 1 and 2 look qualitatively different, with case 3 mixing the two types.  But am I right in saying that the CausalImpact package will look at all of them in the same way -- constructing as best it can the "counterfactual" sales time series for the store (by using the price, feature space and temperature to give its best guess) -- and comparing that to the actuals.

Overall question, is it equally valid to use CausalImpact in all three cases, despite the conceptual difference in the predictor variables?  And how would you interpret the results given?

Paul

[Kay Brodersen]

Hi Paul,

Case 1 and 2 are based on predictors that may have very different flavours from a study-design point of view. But they both yield valid inferences under the CausalImpact framework as long as:

• your regressors weren't themselves affected by the intervention;
• the relationship between your regressors and the outcome series wouldn't have been affected by the intervention had the intervention not taken place.

If both datasets are available, I would suggest comparing the results to see whether they provide mutually supporting evidence.

Kay

--
You received this message because you are subscribed to the Google Groups "CausalImpact" group.
To unsubscribe from this group and stop receiving emails from it, send an email to causalimpact...@googlegroups.com.
To post to this group, send email to causal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/causalimpact/9d931c0e-8e5d-415f-9fcc-5e9ba1ac2399%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.