Production grade, Gradual rollout and basic A/B testing

[ittai zeidman]

Hi,

I stumbled on to this project while reading and thinking about feature toggles and I have to say I'm impressed.

It seems like a lot of aspects of this problem have been thought of so first of all well done.

I have two questions after reading the documentation:

1. Is togglz production grade? Do you know if anyone is using it in production?

2. What do you think about adding the ability to have a feature active not to a list of users but to a percentage of the users?

This can facilitate gradual rollout (no users, selected users, 10% of users, all users) as well as basic A/B testing (set the percentage to 50% and see the new against the old).

Implementation wise it seems a basic implementation might be to add a check in the DefaultFeatureManager which does modulo on basis of the username hashcode (I think it distributes normally). Of course an enum would be added in FeatureState to whether this is active for all/ name based/ percentage based.

If you think that's interesting I might even try to contribute this change myself.

Thanks a lot in advance,

Ittai

[Christian Kaltepoth]

Dear Ittai,

thank you very much for your email and your feedback. :)

Regarding your first question. Yes, Togglz is definitely ready for production. Our company is using Togglz in all new projects. And we also integrated it into some existing projects that are currently in production. As Togglz is a more or less new project, I cannot tell for sure if there are many users out there using it by now. I can only say, that we do it with huge success. :)

Regarding your second point. Yes, this sound like a very interesting idea. We typically roll out features to some beta testers first and after that to all other users at once. However, it would be very cool to support such gradual rollouts in the future.

Thinking about this a bit more I'm not 100% sure, if the percentage-based decisions should really be implemented directly into the DefaultFeatureManager. I think there could be many different other strategies for rolling out features. Like for example by geographical region, by company department or whatever. It would be cool to have some SPI so users could simply extend Togglz to support new strategies. I'm thinking about something like a FeatureActivator interface, that would tell whether a feature is active or not for a given FeatureUser and a FeatureState. This could also be used to implement such gradual rollouts.

I'm currently in the process of releasing Togglz 1.1.0.Final which will be ready by the end of this week. After 1.1.0 has been pushed out, I will definitely start to experiment with this idea.

Thank you so much for your feedback.

Best regards

Christian Kaltepoth

2012/10/24 ittai zeidman <itt...@gmail.com>

--
 
 
 

--
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal

[Ittai Zeidman]

Hi Christian,

The SPI solution sounds great.

I thought of the DefaultFeatureManager as that is where the decisions are currently being made (checking if the list is empty and so on).

I'll try to use togglz soon and be in touch.

Ittai 

P.S. the spring factory proxy is an amazing tool to lousy couple the decisions.

ב-25 באוק 2012, בשעה 12:13, Christian Kaltepoth <chri...@kaltepoth.de> כתב/ה:

--
 
 
 

[Christian Kaltepoth]

Hey Ittai,

a short status update. I finished a first draft of a SPI to implement custom activation strategies. The usecase you described could be implemented like this:

public class GradualActivationStrategy implements ActivationStrategy {

    @Override

    public boolean isActive(FeatureState state, FeatureUser user) {

        Double threshold = state.getAttribute("percent"); // not implemented yet

        if(threshold == null) {

            return true;

        }

        

        double p = user.getName().hashCode() % 100;

        return p < threshold;

    }

}

What do you think? Actually a strategy implementation would tell Togglz whether a feature is active or not based on the current FeatureUser and the FeatureState.

Unfortunately there are still a lot of things to do:

• The strategy should declare what parameters are required (like the percentage in your example)
• Users should be able to select a strategy and specify the parameters using the Togglz Admin Console
• The State Repositories have to persist these custom parameters
• I think there have to be some kind of converter mechanism so that parameters always have a specific type (double, string, int, whatever)

You see, there is still much to do. But I'm working on it. Just want to hear what you think about the SPI from a user perspective. :)

Best regards

Christian

2012/10/25 Ittai Zeidman <itt...@gmail.com>

[ittai zeidman]

Hi Christian,

The SPI looks great.

I agree that the complex part would be type-safety and convergence with the admin-ui.

One example I saw for such extensibility was in spring security oauth where  they added a map of string to string for additional attributes and left it to the implementors to  handle the type-safety. That would be a less desirable solution but maybe a more practical one to begin with. BTW that map was serialized as json to their jdbc repository.
Another issue is how do implementations register themselves so that the the ui-admin can display them as options.

If you'd like to continue bouncing ideas I'm happy to help.

Ittai

--
 
 
 

[Christian Kaltepoth]

Hi Ittai,

thanks for your feedback. Yeah, I think you are right. Maybe type-safety is a bit over-engineered at this point. I think a simple map containing strings would be fine for now. Do you have any reference to the Spring Security extension point you wrote about? I would be interested to see how the API looks like.

Regarding registration: I think the strategy could tell about the parameters that are required. The strategy itself will be looked up using the ServiceLoader mechanism provided by the JDK. So the SPI could look like this:

public class GradualActivationStrategy implements ActivationStrategy {

    @Override

    public boolean isActive(FeatureState state, FeatureUser user) {

        String threshold = state.getAttribute("percent");

        // ....

    }

    public StrategyParameter[] getParameters() {

      return new StrategyParameter[] {

            StrategyParameter.create("percent").named("Rollout Percentage").matching("[0-9]+")

      };

    }

}

The nice thing here is that the strategy could require the parameter to match a certain pattern. This way the user is forced to enter a valid number although the conversion to an integer is done later. This way it is not soooo bad that the strategy has to do the conversion itself.

I think it could work this way. :)

Christian

2012/11/1 ittai zeidman <itt...@gmail.com>

[ittai zeidman]

Hi Christian,

Just to verify:

create("percent") => "percent" is the key of the parameter

named("Rollout Percentage") => "Rollout Percentage" is the label in the admin ui

matching("[0-9]+") => "[0-9]+" is a regex which will be used in the admin ui for that input 

Am I correct in my understanding?

I think that's excellent. Type safety is better but should probably be deferred for the moment.

Took a bit of digging around for the extension point (those projects are not currently "mounted" on my workspace) but I found it:

interface for pojo

https://github.com/SpringSource/spring-security-oauth/blob/master/spring-security-oauth2/src/main/java/org/springframework/security/oauth2/provider/ClientDetails.java

impl pojo

https://github.com/SpringSource/spring-security-oauth/blob/master/spring-security-oauth2/src/main/java/org/springframework/security/oauth2/provider/BaseClientDetails.java

impl service which handles pojo (look for loadClientByClientId)

https://github.com/SpringSource/spring-security-oauth/blob/master/spring-security-oauth2/src/main/java/org/springframework/security/oauth2/provider/JdbcClientDetailsService.java

If you need additional references I'm here.

and thanks!

--
 
 
 

[Christian Kaltepoth]

Hi Ittai,

yes, you understood it correctly. The create() method would set some kind of unique ID for the parameter that can be used later to get the value out of the FeatureState. There is also an optional label and a regular expression that must match for the value. This allows a limited form of validation when the user enters a value in the admin UI.

I'm not sure if the method names will be exactly like that, but basically this example shows the general concept.

Thanks for the references to the Spring Security API. I'll have a look at how they do it. I'll come back to you as soon as I have something working. You could then give the snapshots a try if you like. :)

Christian

2012/11/4 ittai zeidman <itt...@gmail.com>

[ittai zeidman]

I'll try.

I'm in my baby steps into continuous delivery so it might take me a bit of time but I think I'll try to conjure up an experiment for this

--
 
 
 

[Christian Kaltepoth]

Hey Ittai,

I've finally published a first Alpha version of Togglz 2.0.0 which contains the feature activation strategies that we discussed. I would be happy to get any kind of feedback.

There is no official release announcement yet, but you can have a look at this mail I sent to the list to learn more about the new version:

https://groups.google.com/d/msg/togglz-users/PfYJwH03O68/YnjmdRDmv98J

Best regards

Christian

Blog: http://blog.kaltepoth.de/

Twitter: http://twitter.com/chkal

GitHub: https://github.com/chkal