Is there a Notion of Test Suites or Related tests?

[Joe Bell]

Our analytics team is interested in having a "holdout group", for example when bucketing users a certain percentage of traffic will not enter any tests.  Obviously there will be a small percentage of traffic this is true for, but I think their goal is to have this be a larger percentage than what it would be naturally.

We can imagine maybe writing a proctor test that buckets traffic this way and then having a requirement for every other test be that the user is not in the "holdout group".  However, it seems like a pain to have to write that for every test.

What I sort of want is to have a nested test approach, where only if the user is not randomly bucketed into the holdout group, then they are exposed to a test.  

Is there support for this in proctor that I'm not seeing?

[Ketan Gangatirkar]

Hi, Joe.  At present, Proctor does not support this kind of holdout group.  We've kicked the idea around from time to time, and we have a few approaches that could work, but we've never seriously pursued this.

--
You received this message because you are subscribed to the Google Groups "indeedeng-proctor-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to indeedeng-proctor-users+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Ketan Gangatirkar
ke...@indeed.com