togglz-administration via JMX

[Mark Doppelfeld]

Hi,

I just stumbled about this project and really like the idea of having a 'standardized' way to implement de/activating of features. Great work guys!!!

When I saw the toggle console, I wondered why you are not going into the JMX direction ???

If you think this is a feature that would be helpful, I really would like to support you in this area.

So please let me know what you think.

Best regards

Mark Doppelfeld

[Christian Kaltepoth]

Hey Mark,

nice to hear that you like Togglz. :)

Actually I also think that JMX support would be great and very useful for many users (including me). I already thought about implementing it, but I was too busy with the other changes I wanted to get into 2.0.0.Final, which I plan to release very soon.

I would be glad to get some help with the JMX support. I already added an SPI to 2.0.0-SNAPSHOT that IMHO would be very useful:

https://github.com/togglz/togglz/blob/master/core/src/main/java/org/togglz/core/spi/FeatureManagerListener.java

This SPI receives events when the FeatureManager is started and stopped. A JMX module could hook into this and publish corresponding mbeans. I think the JMX code should go into a separate module. If the user includes the module, the JMX support is automatically enabled.

I think it would make sense to provide JMX support for reading and updating the feature state which is persisted by the FeatureStateRepository implementations. To be honest, I'm not an expert on JMX. I know there are different types of mbeans. Due to the nature of the managed feature state, it would perhaps make sense to use something different than standard mbeans.

What do you think?

Christian

2013/3/29 Mark Doppelfeld <mark.do...@gmail.com>

--

Christian Kaltepoth

Blog: http://blog.kaltepoth.de/

Twitter: http://twitter.com/chkal

GitHub: https://github.com/chkal

[Christian Kaltepoth]

Sorry, forgot to send a copy to the mailing list

---------- Forwarded message ----------
From: Christian Kaltepoth <chri...@kaltepoth.de>
Date: 2013/3/30
Subject: Re: [togglz-dev] togglz-administration via JMX
To: Mark Doppelfeld <dou...@gmx.de>

Hey Mark,

I'll answer to your question inline...

I already had a look into FeatureContext. Here you are maintaining different FeatureManager for different ContextClassLoaders. My expectation for an JMX bean would be to be able to operate on ALL FeatureManagers known for the different ClassLoaders.

I think it would make sense to have separate JMX beans for each FeatureManager. They could have different names, depending on the feature enum or something like this to simplify identifying them. And it would simplify the API, because you don't have to reference the FeatureManager you want to use in each call to the JMX API. I thought of JMX object names like:

org.togglz.jmx.FeatureManagement:name=com.example.myapp.MyFeatureEnum 

Usually there is only one FeatureManager per VM. But if users deploy multiple applications to one app server, there could be multiple ones. And I think that it makes sense to have separate JMX beans in this case.

 

BTW: Why did you coma up with this approach, i.e. having different FeatureManager per ClassLoader? Is it because you want to be able to drop the toggle jar just once into an app server?

Actually the map that stores a FeatureManager per class loader is just a cache. The basic concept is that the FeatureContext uses the FeatureManagerProvider SPI to lookup the correct FeatureManager for the current execution context. There a different implementations for different deployment scenarios. But as the lookup MAY take some time, the result is cached on a per context class loader basis. Actually I already thought about removing this caching, but I think I'll leave it there for now.

But fortunately we shouldn't have to deal with FeatureContext at all for the JMX support, if we create the JMX beans using the FeatureManagerListener SPI like described above. We just would have to publish the mbean when FeatureManagerListener#start() is called and remove it in FeatureManagerListener#stop().

 

What is the target JDK for togglz? Java got some very nice new features in JMX area in JDK 1.6 +, but if togglz is targeting 1.5 + , these options are not available.

Good news. Togglz requires only JDK 1.6 :)

 

In my view the JMX beans act 'just' as facades to the different FeatureManager instances in a jvm. So in a first implementation I'ld suggest to expose this methods and their results (probably converted into a string representation

)

I agree. This should be fine for a first version.

 

FeatureManager#getFeatures 

FeatureManager#setFeatureState 

The most important methods are getFeatureState() and setFeatureState(). The method getFeatures() is also nice to have and the new getMetaData() may also be interesting.

 

The interesting part is to find the correct FeatureManager for an update operation … so that we do the update on the correct instance of a feature.

This won't be a problem if there is exactly one JMX bean for each feature manager as I suggested above.

 

What do you think?

Christian

--

Christian Kaltepoth

Blog: http://blog.kaltepoth.de/

Twitter: http://twitter.com/chkal

GitHub: https://github.com/chkal

[Mark Doppelfeld]

Hi Christian,

I try to summarize my understandings:

- we will have a JMX bean instance for each FeatureManager

this implicates we need to have a unique id on each FeatureManager (or FeatureProvider), so we can build unique MBean object names. The EnumBasedFeatureProvider ie. might be used by several FeatureManager instances. 

What do you think?

 

- JMX support is de/activated via Listener on start/stop of FeatureManager

- Target is JDK 1.6+

- JMX is done in a separate module

- first implementation will cover 

FeatureManager#getFeatures 

FeatureManager#setFeatureState

FeatureManager#setFeatureState 

other methods will be exposed via JMX as needed...

That's fine by me.

I'll have to read on using git/github. Haven't had the pleasure up to know… 

I saw you recommend to create a special 'repository' for the implementation and issue a pull request when I'm done!?

I think I'll start working on this on monday and will finish within the next week. 

Enjoy the  weekend

Mark

Am 30.03.2013 um 17:25 schrieb Christian Kaltepoth:

Hey Mark,

I'll answer to your question inline...

I already had a look into FeatureContext. Here you are maintaining different FeatureManager for different ContextClassLoaders. My expectation for an JMX bean would be to be able to operate on ALL FeatureManagers known for the different ClassLoaders.

I think it would make sense to have separate JMX beans for each FeatureManager. They could have different names, depending on the feature enum or something like this to simplify identifying them. And it would simplify the API, because you don't have to reference the FeatureManager you want to use in each call to the JMX API. I thought of JMX object names like:

org.togglz.jmx.FeatureManagement:name=com.example.myapp.MyFeatureEnum 

Usually there is only one FeatureManager per VM. But if users deploy multiple applications to one app server, there could be multiple ones. And I think that it makes sense to have separate JMX beans in this case.

 

BTW: Why did you coma up with this approach, i.e. having different FeatureManager per ClassLoader? Is it because you want to be able to drop the toggle jar just once into an app server?

Actually the map that stores a FeatureManager per class loader is just a cache. The basic concept is that the FeatureContext uses the FeatureManagerProvider SPI to lookup the correct FeatureManager for the current execution context. There a different implementations for different deployment scenarios. But as the lookup MAY take some time, the result is cached on a per context class loader basis. Actually I already thought about removing this caching, but I think I'll leave it there for now.

But fortunately we shouldn't have to deal with FeatureContext at all for the JMX support, if we create the JMX beans using the FeatureManagerListener SPI like described above. We just would have to publish the mbean when FeatureManagerListener#start() is called and remove it in FeatureManagerListener#stop().

 

What is the target JDK for togglz? Java got some very nice new features in JMX area in JDK 1.6 +, but if togglz is targeting 1.5 + , these options are not available.

Good news. Togglz requires only JDK 1.6 :)

 

In my view the JMX beans act 'just' as facades to the different FeatureManager instances in a jvm. So in a first implementation I'ld suggest to expose this methods and their results (probably converted into a string representation

)

I agree. This should be fine for a first version.

 

FeatureManager#getFeatures 

FeatureManager#setFeatureState 

The most important methods are getFeatureState() and setFeatureState(). The method getFeatures() is also nice to have and the new getMetaData() may also be interesting.

 

The interesting part is to find the correct FeatureManager for an update operation … so that we do the update on the correct instance of a feature.

This won't be a problem if there is exactly one JMX bean for each feature manager as I suggested above.

 

What do you think?

Christian

[Christian Kaltepoth]

Hey Mark,

I try to summarize my understandings:

- we will have a JMX bean instance for each FeatureManager

this implicates we need to have a unique id on each FeatureManager (or FeatureProvider), so we can build unique MBean object names. The EnumBasedFeatureProvider ie. might be used by several FeatureManager instances. 

What do you think?

Yes, exactly. I also thought about the unique identifier. I'll add a corresponding API to the FeatureManager. Something like FeatureManager#getId or something like that. I created an issue to track that:

https://github.com/togglz/togglz/issues/31

 

- JMX support is de/activated via Listener on start/stop of FeatureManager

- Target is JDK 1.6+

- JMX is done in a separate module

- first implementation will cover 

FeatureManager#getFeatures 

FeatureManager#setFeatureState

FeatureManager#setFeatureState 

other methods will be exposed via JMX as needed...

That's fine by me.

Perfect. :)

The most difficult part will be to decide how to handle the FeatureState object. In 2.0.0-SNAPSHOT the structure has changed compared to 1.1.x. The togglz.org also still described the old state model. I'll try to find some time to update the docs soon. You can have a look here if you want to learn something about the changes in 2.0.0-SNAPSHOT:

http://www.togglz.org/whats-new.html

https://groups.google.com/forum/#!msg/togglz-users/-nQ5JP-cIc4/QqohQISm2K0J

 

I'll have to read on using git/github. Haven't had the pleasure up to know… 

I saw you recommend to create a special 'repository' for the implementation and issue a pull request when I'm done!?

If you are new to Git, I recommend to read the Git book. I learned Git this way too. :)

http://git-scm.com/book

The typical GitHub way is to fork the repository you want to contribute to:

https://help.github.com/articles/fork-a-repo

You then create a branch in your clone of the repository and commit your changes there. You should pull in the latest changes from the official repository so you don't get out of date. All this is explained there.

If you are finished with you work, you can create a pull request:

https://help.github.com/articles/using-pull-requests

But you could also tell me that you are finished before you do the pull request and I'll have a look at your repository and may provide some feedback.

I hope this helps a bit. Feel free to contact me if you need any help or guidance. :)

 

I think I'll start working on this on monday and will finish within the next week. 

Sure. No need to hurry. Take your time. And ask anything that comes to your mind. :)

[Mark Doppelfeld]

Hi Christian,

currently I've no access to the stateRepository when I'm working with a FeatureManager. and I see no other way to persist the new state, than using the repository.

In my opinion it's not a good idea to have 'normal' users access the repository. 

So I see different solutions for this: 

a) when signaling the start event, we pass the repository as well

b) having a separate interface extending the existing FeatureManager interface and passing this in the start  event

  we could hide it as  protected inner interface, but to do this FeatureManager & FeatureManagerListener 

would need to reside in the same package 

c) implement a different way to access the repository

a) is quick and somewhat dirty

b) is clean but somewhat complicated

c) how to ensure no one is misusing this way?

What do you think?

BTW: I'm using Netbeans for working on togglz… what are the code formatting options I should use?

Mark

[Christian Kaltepoth]

I'm on my phone right now. So just a quick reply. You can use setFeatureState and getFeatureState of FeatureManager for now. The manager delegates these to the repository. But I think it also makes sense that the manager exposes the repository.

I'll reply in detail tomorrow.

Christian

[Mark Doppelfeld]

Must have been blinded or my new born child consumes more than the expected energy :-)

Mark

[Christian Kaltepoth]

Hey Mark,

now I have some more time for answering. :)

Yeah, I really think it could make sense to expose the StateRepository using the FeatureManager. But on the other hand the important methods (get/setFeatureState) are already there in the manager. I don't know yet. But I think you can go with the two methods provided by the manager directly. Although it is not 100% the same as invoking the methods on the repository directly.

Regarding the code formatting. I'm using a code format file for Eclipse checked into the source repository. But I guess this won't help very much if you are working with Netbeans. The formatting is based on the JBoss Community guidelines. So perhaps this will help:

https://community.jboss.org/wiki/ImportFormattingRules#Netbeans

Christian

2013/4/1 Mark Doppelfeld <dou...@gmx.de>

[Mark Doppelfeld]

Hi Christian,

while writing some Tests I stumbled upon an uninitialized FeatureContext. 

While browsing through the code base, I found TogglzBootstrap and FeatureManagerBootstraper. Both basically create an FeatureManager and are not related to the FeatureContext. To be honest: I'm confused!

Probably the behavior I'm facing in my test is no what was intended? The problem is that FeatureContext is used to lookup FeatureManager whilst already having had access to a FeatureManager. See stack trace at the end.

What I'm personally missing is a simple way of having different FeatureManager instances for different Junit Tests without the need to use the ServiceLoader. What do you think of having the  FeatureManagerBootstrapper have a third way to lookup a configuration via a static TogglzConfig  TogglzConfigurationHolder.getConfiguration() (or something like this) which would be set via TogglzConfigurationHolder.setConfiguration(TogglzConfig config) 

Or did I again miss something completely???

Mark

testGetFeatureState(org.togglz.jmx.JmxTest)  Time elapsed: 0.125 sec  <<< ERROR!

java.lang.IllegalStateException: Could not find the FeatureManager. For web applications please verify that the TogglzFilter starts up correctly. In other deployment scenarios you will typically have to implement a FeatureManagerProvider as described in the 'Advanced Configuration' chapter of the documentation.

at org.togglz.core.context.FeatureContext.getFeatureManager(FeatureContext.java:95)

at org.togglz.core.util.UntypedFeature.getTypedFeature(UntypedFeature.java:39)

at org.togglz.core.util.UntypedFeature.name(UntypedFeature.java:27)

at org.togglz.core.manager.EnumBasedFeatureProvider.getMetaData(EnumBasedFeatureProvider.java:36)

at org.togglz.core.manager.DefaultFeatureManager.getMetaData(DefaultFeatureManager.java:47)

at org.togglz.core.manager.DefaultFeatureManager.getFeatureState(DefaultFeatureManager.java:88)

at org.togglz.jmx.JmxFeatureManagerAdapter.getFetaureState(JmxFeatureManagerAdapter.java:77)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:167)

at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:96)

at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:33)

at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)

at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)

at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:264)

at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)

at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:762)

at com.sun.jmx.mbeanserver.MXBeanProxy$InvokeHandler.invoke(MXBeanProxy.java:134)

at com.sun.jmx.mbeanserver.MXBeanProxy.invoke(MXBeanProxy.java:148)

at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:248)

at com.sun.proxy.$Proxy7.getFetaureState(Unknown Source)

at org.togglz.jmx.JmxTest.testGetFeatureState(JmxTest.java:81)

[Christian Kaltepoth]

Hey Mark,

thanks for pointing me to this. Let me explain some of the ideas.

From the user perspective the FeatureContext is the preferred way to lookup the FeatureManager. The FeatureContext uses the FeatureManagerProvider SPI to find the correct FeatureManager for the application. The manager can be obtained from anywhere. In large app servers it could be looked up from JNDI because it should be shared between different applications. 

For simple web applications there is typically a local FeatureManager which is created during application startup. This is done using FeatureManagerBootstrapper. This classes uses either TogglzBootstrap or TogglzConfig implementations provided by the end user to build and/or configure the FeatureManager. The newly created FeatureManager is then stored in a place where a FeatureManagerProvider will find it.

Back to your problem. The exception you are getting is caused by the fact that you are using UntypedFeature. To be honest, I was thinking about removing the class. Especially because it currently uses the FeatureContext even when calling UntypedFeature#name() which can lead to problems in unit tests just like in your case. 

But I just pushed an alternative implementation named NamedFeature which you can use instead. This one should work fine I think.

https://github.com/togglz/togglz/blob/master/core/src/main/java/org/togglz/core/util/NamedFeature.java

You will have to pull the latest changes I did into your repository clone to get the class. But that's simple. Have a look here:

https://help.github.com/articles/fork-a-repo

Especially "Configure Remotes" and "Pull in upstream changes".

Let me know if this work.

And thanks for pointing me to this. :)

Christian

2013/4/4 Mark Doppelfeld <dou...@gmx.de>

[Mark Doppelfeld]

Hi Christian,

what still confuses me its the following point:

Why not have the FeatureContext use FeatureManagerBootstrapper to do the lookup, or even put functionality of both classes together in one? Then there would be just one mechanism of initializing Togglz for the various usage scenarios?

Just my 2 ct.

NamedFeature also uses the FeatureContext, so I doubt it will solve my problem. But I'll try this.

Mark

[Mark Doppelfeld]

Ok, FeatureContext is only used in isActive()… so this aproach works for my uses case.

Nevertheless I'm interested in your opinion on the bootstrapping process.

Mark

[Christian Kaltepoth]

Hey Mark,

I think there is a huge difference between "getting" the FeatureManager at some specific point in time and "boostrapping" one.

The FeatureContext just has to lookup the FeatureManager that should be used in the current execution context. So if you call FeatureContext.getFeatureManager().isActive(whateverer) you just want to get the "correct" feature manager to use. For the FeatureContext it is not relevant at all where the manager comes from. It could be looked up from JNDI or it could be a proxy that delegates all the calls to it some FeatureManager running in another VM via RMI. All this is currently possible by implementing a custom FeatureManagerProvider. In these cases there is no "bootstrapping of a feature manager" at all because the application doesn't control the lifecycle of the manager it uses.

In many (but not all) cases, the application will have a feature manager for itself. In this case Togglz has to do some kind of bootstrapping which is implemented in FeatureManagerBootstrapper. So bootstrapping a FeatureManager for an application is actually an optional step although it is done in most environments. Another important thing is that it is not enough to start up the manager. It is also required to handle the shutdown. Especially to free up resource like ThreadLocals. If Togglz would lazily create a FeatureManager in FeatureContext, it would be impossible to perform a clean shutdown which will lead to memory leaks and all the ugly stuff.

I'm happy to hear that NamedFeature works for you. I'm really thinking about deprecating UntypedFeature. I'll have to check where it is used inside of Togglz and if it can be replaced with NamedFeature in all cases.

Have a nice weekend. :)

Christian

  

2013/4/5 Mark Doppelfeld <dou...@gmx.de>