Fabric for bundle config?

565 views
Skip to first unread message

Michael K

unread,
Jul 16, 2012, 2:17:31 PM7/16/12
to fusef...@googlegroups.com
hi,

Is there a way to configure blueprint bundle via Fabric?
I'd like to have ability to push bundle config properties through Fabric, store it in zk.
When my bundle starts it should get config props from Fabric / zk.
When props are changed in zk them my bundle has to reload with updated props.

Can Fabric help for this matter? And how?
Anything else can configure OSGi bundle from zk?

thanks.

Ioannis Canellos

unread,
Jul 17, 2012, 5:22:30 AM7/17/12
to fusef...@googlegroups.com
Yes, Fabric can help you on this matter.

In blueprint you can use placeholders that will get resolved using a configuration PID. Any change for that PID in the configuration admin will either update the values of placeholders or even reload the whole blueprint context (that configurable).

Fabric allows you to manage distributed configuration. So you can specify the configuration as part of the profile, and Fabric will make sure that this configuration will be assigned to all containers running under that profile.

In practice:

In blueprint you can use property placeholders that will draw values from "yet.another.pid" like this:

    <cm:property-placeholder id="proxy" persistent-id="yet.another.pid" placeholder-prefix="${" placeholder-suffix="}" update-strategy="reload">
        <cm:default-properties>
            <cm:property name="port" value="9090"/>
        </cm:default-properties>
    </cm:property-placeholder>

    <bean id="myBean" class="yet.another.bean">
        <property name="servicePort" value="${port}"/>
    </bean>

Now assuming that this blueprint runs on containers that have assinged the "myprofile", you can change the configuration of "yet.another.pid" as follows:

> fabric:profile-edit --pid yet.another.pid/port=9091 myprofile

This command will initally store the new configuration port in zookeeper, than fabric will apply that confgiuration to all containers running the "myprofile" profile. Blueprint will detect the change and will reload the whole context. If you just want to update the values without reloading the whole context you can use update-strategy="update".

I hope this helps.

--
Ioannis Canellos

Twitter: iocanel


Michael K

unread,
Jul 17, 2012, 3:06:27 PM7/17/12
to fusef...@googlegroups.com
Hello Ioannis,

thank you for the reply.
I see that I'll have
etc/yet.another.pid.cfg

When FuseESB (SMX) starts what will push property value to my blueprint bundle? Fabric or PID loader? How ConfigAdmin knows from where to loads values?
I see there is ZooKeeper that stores and there is etc/yet.another.pid.cfg

Without fabric ConfigAdmin will loads from etc/yet.another.pid.cfg based on <cm> configuration
How it works with fabric in place?

What will happen when from beginning a bundle has etc/yet.another.pid.cfg with some values?
Will fabric push those values to ZooKeeper?

What will happen when User directly modify etc/yet.another.pid.cfg ? Will fabric update ZooKeeper and other PIDs on other ESB instances?

Mike

Ioannis Canellos

unread,
Jul 17, 2012, 5:07:46 PM7/17/12
to fusef...@googlegroups.com
When FuseESB (SMX) starts what will push property value to my blueprint bundle? Fabric or PID loader? How ConfigAdmin knows from where to loads values?

ConfigAdmin, doesn't know that. Blueprint knows to retrieve the value of the placeholder from configurationAdmin. And it knows that because you tell it to do so in the following element:
<cm:property-placeholder id="proxy" persistent-id="yet.another.pid" placeholder-prefix="${" placeholder-suffix="}" update-strategy="reload">
 
Without fabric ConfigAdmin will loads from etc/yet.another.pid.cfg based on <cm> configuration

How it works with fabric in place?

To be more accurate, the blueprint will retrieve the values from the yet.another.pid from the configruation admin. Where the configuration is persisted is an other story.

What fabric does in this scenario? Fabric "bridges" zookeeper with the configuration admin service. So you can specify how containers should be configured in the zookeeper and fabric will translate/bridge that to configuration pids. So you can store the content of the pid in zookeeper and have fabric bridge that to the configuration admin service of the containers.

What I proposed you to do is to have the whole configuration inside the profile. Fabric will make sure that the configuration admin service of each container assigned that profile will see the updated values.


What will happen when from beginning a bundle has etc/yet.another.pid.cfg with some values?
Will fabric push those values to ZooKeeper?

No, fabric will never push configuration information to zookeepr, it will only pull. It will push runtime information like ssh and jmx urls, but nothing related to configuration or provisioning.
The whole concept is that in fabric you define you configuration, and provisioning setup in one place and fabric makes sure that all containers comply with it.
 
What will happen when User directly modify etc/yet.another.pid.cfg ? Will fabric update ZooKeeper and other PIDs on other ESB instances?

No, Fabric will not push the change to other containers. In fact, if the fabric registry contains configuration data for that pid, it will overwrite the change made by the user, with the data found in zookeeper registry.

You might find this useful: http://fuse.fusesource.org/fabric/ as it describes lot of things about the fabric way of things.
 

Michael K

unread,
Jul 17, 2012, 5:45:09 PM7/17/12
to fusef...@googlegroups.com
thank you very much for such detailed reply.
Yes, went through Fuse Source documentation about fabric, no it wasn't clear for me.

Would you recommend to limit User interaction with etc/*.cfg files? Tell User to NOT modify bundle pid files at all !!!
Another thing is security - it isn't right to keep password in plain text in *.cfg and for this reason I've used <cme> that allows decrypt <ENC>ripted text
How Fabric can address this concern?
Wil ZooKeeper keep in plain text passwords for lets say EndPoint login configs? Or there is a way to keep it encrypted and how to do it that Fabric automaticaly can decrypt it to push to PID?

How secure is communication between ZK to ZK in case passwords are in plain text in ZK to avoid sniffing?
Are there any diagrams about Fabric?

Thanks.
Mike.

Ioannis Canellos

unread,
Jul 18, 2012, 4:04:10 AM7/18/12
to fusef...@googlegroups.com
Would you recommend to limit User interaction with etc/*.cfg files? Tell User to NOT modify bundle pid files at all !!!

When, you are using fabric, I would recommend to go all the way with Fabric. By that I mean install/uninstall and configure everything through fabric.

 
Another thing is security - it isn't right to keep password in plain text in *.cfg and for this reason I've used <cme> that allows decrypt <ENC>ripted text
How Fabric can address this concern?

Fabric, currently doesn't do anything special about it. But I guess that you can adopt the approach that you were using to Fabric. If you provided some more details on what exactly you were doing I might be able to tell you more.

Michael K

unread,
Jul 19, 2012, 10:06:27 PM7/19/12
to fusef...@googlegroups.com
thank you for the answer.
Is there any step by step guide to provision / push properties values by User through fabric to bundle property placeholder?
Something like:
1. run fabric:create
2. ...

When my property place holder has 2 default properties. Lets say servicePort and hostName to be able configure EndPoint.

My Use Case is to allow end customer to configure EndPoints. No, end customer will not deploy / install and etc for features and bundles, just configurations.

Thanks.

Ioannis Canellos

unread,
Jul 23, 2012, 5:54:58 AM7/23/12
to fusef...@googlegroups.com
1) Initialize fabric: fabric:create
2) Define the profile:

fabric:profile-create --parents default myprofile

3) Add the blueprint bundle to the profile:
fabric:profile-edit --bundles mvn:yet.another/bluperint-bundle/1.0 myprofile

Note that the bundle needs to contain a blueprint descriptor similiar to the one in my intial post

4) Add the initial configuration to the profile:

fabric:profile-edit --pid yet.another.pid/servicePort=9091 myprofile
fabric:profile-edit --pid yet.another.pid/serviceHost="${zk:${karaf.name}/ip}" myprofile (Here I am using variable substitution so that each container will have as serviceHost its own ip.

5) Assign the profile to one or more containers:
fabric:container-profile-change somecontainer myprofile

You can then change the configuration globaly by doing things described in step 4.

Note: This requires "managed" containers (containers that run the fabric-agent). 

Michael K

unread,
Jul 24, 2012, 3:33:07 PM7/24/12
to fusef...@googlegroups.com
Thank you again for the reply.
1. Do I have to do #3 ?
I already have my bundle pre packaged and running in FuseESB.

2. Do I have to have yet.another.pid.cfg in etc folder? without it I don't see my PID (config:list |grep pid)

3. what container should I use?
FuseESB:karaf@root> fabric:profile-create --parents default myprofile
FuseESB:karaf@root> fabric:container-list
[id]                           [version] [alive] [profiles]                     [provision status]
root*                          1.0       true    fabric, fabric-ensemble-0000-1

4. how to configure ZK to run on the same and only one FuseESB - no cluster yet
FuseESB:karaf@root> fabric:container-change-profile root myprofile
FuseESB:karaf@root> fabric:container-list
Error executing command: java.lang.IllegalStateException: Error waiting for ZooKeeper connection
FuseESB:karaf@root>

and I got those in log:

12:21:08,970 | WARN  | 0.0/0.0.0.0:2181 | NIOServerCnxnFactory             | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | Ignoring unexpected runtime
exception
java.nio.channels.CancelledKeyException
        at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)[:1.6.0_33]
        at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)[:1.6.0_33]
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at java.lang.Thread.run(Thread.java:662)[:1.6.0_33]
12:21:08,971 | INFO  | 0.0/0.0.0.0:2181 | NIOServerCnxnFactory             | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | NIOServerCnxn factory exited
 run method
12:21:08,971 | INFO  | use-061-thread-3 | ZooKeeperServer                  | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | shutting down
12:21:08,971 | INFO  | use-061-thread-3 | SessionTrackerImpl               | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | Shutting down
12:21:08,971 | INFO  | use-061-thread-3 | PrepRequestProcessor             | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | Shutting down
12:21:08,971 | INFO  | use-061-thread-3 | SyncRequestProcessor             | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | Shutting down
12:21:08,971 | INFO  | SyncThread:0     | SyncRequestProcessor             | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | SyncRequestProcessor exited!
12:21:08,972 | INFO  | use-061-thread-3 | FinalRequestProcessor            | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | shutdown of request processo
r complete
12:21:08,972 | INFO  | sid:0 cport:-1): | PrepRequestProcessor             | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | PrepRequestProcessor exited
loop!
12:21:08,982 | INFO  | NAPSHOT-thread-6 | ManagementStrategyFactory        | 138 - org.apache.camel.camel-core - 2.9.0.fuse-7-061 | JMX enabled.
12:21:08,984 | INFO  | NAPSHOT-thread-6 | BlueprintCamelContext            | 138 - org.apache.camel.camel-core - 2.9.0.fuse-7-061 | Apache Camel 2.9.0.fuse-7-061 (CamelContext:
blueprintContext) is starting
12:21:08,984 | INFO  | NAPSHOT-thread-6 | faultManagementLifecycleStrategy | 138 - org.apache.camel.camel-core - 2.9.0.fuse-7-061 | StatisticsLevel at All so enabling load perfo
rmance statistics
12:21:09,071 | WARN  | use-061-thread-3 | KarafContainerRegistration       | 44 - org.fusesource.fabric.fabric-zookeeper - 7.0.0.fuse-061 | Exception while jmx domain synchroniz
ation from event: javax.management.MBeanServerNotification[source=JMImplementation:type=MBeanServerDelegate][type=JMX.mbean.unregistered][message=]. This exception will be ignor
ed.
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /fabric/registry/containers/domains/root/org.apache.ZooKeeperService
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:868)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.linkedin.zookeeper.client.ZooKeeperImpl.delete(ZooKeeperImpl.java:88)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.linkedin.zookeeper.client.AbstractZooKeeper.delete(AbstractZooKeeper.java:110)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.linkedin.zookeeper.client.AbstractZKClient.delete(AbstractZKClient.java:256)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.fusesource.fabric.zookeeper.internal.KarafContainerRegistration.handleNotification(KarafContainerRegistration.java:348)[44:org.fusesource.fabric.fabric-zookeeper:
7.0.0.fuse-061]
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor$ListenerWrapper.handleNotification(DefaultMBeanServerInterceptor.java:1732)[:1.6.0_33]
        at javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:257)[:1.6.0_33]
        at javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:322)[:1.6.0_33]
        at javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:307)[:1.6.0_33]
...
12:21:09,071 | WARN  | read-EventThread | ZooKeeperConfigAdminBridge       | 199 - org.fusesource.fabric.fabric-configadmin - 7.0.0.fuse-061 | Exception when tracking configurat
ions. This exception will be ignored.
java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /fabric/registry/containers/config/root/ip
        at org.fusesource.fabric.configadmin.ZooKeeperConfigAdminBridge$1.getValue(ZooKeeperConfigAdminBridge.java:172)[199:org.fusesource.fabric.fabric-configadmin:7.0.0.fuse-0
61]
        at org.fusesource.fabric.zookeeper.utils.InterpolationHelper.substVars(InterpolationHelper.java:178)[44:org.fusesource.fabric.fabric-zookeeper:7.0.0.fuse-061]
        at org.fusesource.fabric.zookeeper.utils.InterpolationHelper.performSubstitution(InterpolationHelper.java:88)[44:org.fusesource.fabric.fabric-zookeeper:7.0.0.fuse-061]
        at org.fusesource.fabric.configadmin.ZooKeeperConfigAdminBridge.load(ZooKeeperConfigAdminBridge.java:166)[199:org.fusesource.fabric.fabric-configadmin:7.0.0.fuse-061]
        at org.fusesource.fabric.configadmin.ZooKeeperConfigAdminBridge.onEvents(ZooKeeperConfigAdminBridge.java:274)[199:org.fusesource.fabric.fabric-configadmin:7.0.0.fuse-061
]
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.raiseEvents(ZooKeeperTreeTracker.java:402)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.access$800(ZooKeeperTreeTracker.java:50)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker$TreeWatcher.process(ZooKeeperTreeTracker.java:114)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-0
61]
....
12:21:09,559 | WARN  | yyy.xxx.com:2181) | ClientCnxn                       | 47 - org.fusesource.fabric.fabric-linkedin-zookeeper - 7.0.0.fuse-061 | Session 0x138ba6aaee20000 fo
r server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
        at sun.nio.ch.Net.connect(Native Method)[:1.6.0_33]
        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)[:1.6.0_33]
        at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:203)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:213)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:946)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:973)[47:org.fusesource.fabric.fabric-linkedin-zookeeper:7.0.0.fuse-061]

5. when I applied myprofile to container I got my PID ;-) - both records for service.pid and fabric.zookeeper.pid

Now I have to figure out how to configure ZK.

Thanks.

Stan Lewis

unread,
Jul 24, 2012, 3:50:14 PM7/24/12
to fusef...@googlegroups.com
Ah, when you do fabric:container-change-profile you want to keep the
existing profiles that your container already has, i.e. do:

fabric:container-change-profile root fabric fabric-ensemble-0000-1 myprofile

as after you do "fabric:create" you're running a zookeeper instance in
your current container, however if you get rid of the
fabric-ensemble-0000-1 profile I guess that effectively gets rid of
your zookeeper instance :-0, so you may have to re-create your fabric.
I'll raise an issue to have maybe fabric:container-profile-add and
fabric:container-profile-remove commands instead,
container-change-profile is a bit too low-level.
Reply all
Reply to author
Forward
0 new messages