Cluster manager behaviour within OSGi environment

113 views
Skip to first unread message

Dixie Sebastian

unread,
Sep 6, 2016, 10:50:40 PM9/6/16
to vert.x
Hello,

I tried to setup the clustermanger as per the example given here http://vertx.io/docs/vertx-hazelcast/java/#_using_an_existing_hazelcast_cluster

Please note: Hazelcast when loaded on OSGi, it always uses the com.hazelcast.osgi.HazelcastOSGiService service and injects HazelcastOSGiServiceImpl.

But I saw the io.vertx.hazelcast needed to be wrapped as it is not an OSGi compliant bundle, which when done exports dependencies fine.

Now, When I execute, using the send("destination",{}) and consumer("destination",{}) It works only once (i.e the first message), followed by class not found exceptions.

Initially I suspected it could be because of the different bundle classloaders, but I have both the send and consumer within the same java function inside a verticle.

Exception: 
Sep 07, 2016 8:05:16 AM com.hazelcast.spi.impl.eventservice.impl.EventServiceImpl
SEVERE: [192.16.10.108]:5701 [dev] [3.6.3] hz._hzInstance_1_dev.event-4 caught an exception while processing task:com.hazelcast.spi.impl.eventservice.impl.LocalEventDispatcher@694ffb37
com.hazelcast.nio.serialization.HazelcastSerializationException: Problem while reading DataSerializable, namespace: 0, id: 0, class: 'io.vertx.spi.cluster.hazelcast.impl.HazelcastServerID', exception: io.vertx.spi.cluster.hazelcast.impl.HazelcastServerID
at com.hazelcast.internal.serialization.impl.DataSerializer.read(DataSerializer.java:130)
at com.hazelcast.internal.serialization.impl.DataSerializer.read(DataSerializer.java:47)
at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:46)
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:170)
at com.hazelcast.map.impl.DataAwareEntryEvent.getValue(DataAwareEntryEvent.java:90)
at io.vertx.spi.cluster.hazelcast.impl.HazelcastAsyncMultiMap.entryAdded(HazelcastAsyncMultiMap.java:128)
at com.hazelcast.multimap.impl.MultiMapEventsDispatcher.dispatch0(MultiMapEventsDispatcher.java:108)
at com.hazelcast.multimap.impl.MultiMapEventsDispatcher.dispatchEntryEventData(MultiMapEventsDispatcher.java:84)
at com.hazelcast.multimap.impl.MultiMapEventsDispatcher.dispatchEvent(MultiMapEventsDispatcher.java:55)
at com.hazelcast.multimap.impl.MultiMapService.dispatchEvent(MultiMapService.java:371)
at com.hazelcast.multimap.impl.MultiMapService.dispatchEvent(MultiMapService.java:65)
at com.hazelcast.spi.impl.eventservice.impl.LocalEventDispatcher.run(LocalEventDispatcher.java:56)
at com.hazelcast.util.executor.StripedExecutor$Worker.process(StripedExecutor.java:187)
at com.hazelcast.util.executor.StripedExecutor$Worker.run(StripedExecutor.java:171)
Caused by: java.lang.ClassNotFoundException: io.vertx.spi.cluster.hazelcast.impl.HazelcastServerID
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:137)
at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:115)
at com.hazelcast.nio.ClassLoaderUtil.newInstance(ClassLoaderUtil.java:68)
at com.hazelcast.internal.serialization.impl.DataSerializer.read(DataSerializer.java:119)
... 13 more

Kindly help me with this issue?

Since, I couldn't make much progress, I'm thinking of upgrading hazelcast to 3.7. Any use you think?

Regards,
Dixie.

Thomas SEGISMONT

unread,
Sep 7, 2016, 4:31:37 AM9/7/16
to ve...@googlegroups.com
The Hazelcast cluster manager uses a few custom data classes to improve serialization efficiency. io.vertx.spi.cluster.hazelcast.impl.HazelcastServerID is one of them and is the counterpart of io.vertx.core.net.impl.ServerID.

I'm not an OSGI expert (to say the least) so I couldn't say how to fix this with your container, but the issue is classloader responsible for Hazelcast loading does not how about Vert.x Hazelcast cluster manager classes.

Regards,
Thomas

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/463516d0-52d3-4546-983e-3f442da8867c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Message has been deleted

Thomas SEGISMONT

unread,
Sep 8, 2016, 8:51:46 AM9/8/16
to ve...@googlegroups.com
Most probably because the first message only involves classes accessible to your bundle classloader (e.g. String)

2016-09-07 10:58 GMT+02:00 Dixie Sebastian <dixi...@gmail.com>:
Thanks Thomas. Yes you are right. I even tried to reference Hazelcastservice and set the clustermanger using VertxOptions(). Still the same issue.

What I'm wondering is that how is that only the first message goes through (I mean got a reply from sent->consumer).

Regards,
Dixie.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.

Gadi Eichhorn

unread,
Feb 27, 2017, 5:29:26 AM2/27/17
to vert.x
Hi Dixie,

I have the exact same issue on Karaf . did you find out what was the issue and how to resolve it?

I have two different karaf assemblies and on one the cluster works and the other I get this error. I suspect it is related to vertx-jdbc-client or the datasource (pax-jdbc) dependencies... other than the datasource dependencies the two assemblies are the same. I also get this nasty error when karaf is stopped

Exception in thread "Thread-16" java.lang.ExceptionInInitializerError
        at org.h2.command.Command.executeQuery(Command.java:199)
        at org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:106)
        at org.h2.store.LobStorageBackend.removeAllForTable(LobStorageBackend.java:169)
        at org.h2.engine.Database.close(Database.java:1125)
        at org.h2.engine.DatabaseCloser.run(DatabaseCloser.java:64)
Caused by: java.lang.NullPointerException
        at org.apache.felix.framework.BundleRevisionImpl.getResourceLocal(BundleRevisionImpl.java:500)
        at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1546)
        at org.apache.felix.framework.BundleWiringImpl.getResourceByDelegation(BundleWiringImpl.java:1458)
        at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.getResource(BundleWiringImpl.java:2546)
        at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1307)
        at java.lang.Class.getResourceAsStream(Class.java:2223)
        at org.h2.util.Utils.loadResource(Utils.java:616)
        at org.h2.util.Utils.getResource(Utils.java:609)
        at org.h2.message.DbException.<clinit>(DbException.java:38)
        ... 5 more

Thanks.

Gadi Eichhorn

unread,
Mar 1, 2017, 7:46:34 AM3/1/17
to vert.x

see here -> http://stackoverflow.com/a/42531968/148397

the hazelcast instance doesn't see the vertx serialization class inside the other bundle so to resolve this I rolled my own hazelcast server instance and set the class loader to the vertx bundle class loader

        Config hazelcastConfig = new Config("effectus-instance")
               
.setClassLoader(HazelcastClusterManager.class.getClassLoader())


now the hazelcast config has the vertx classloader and they live happily ever after...

Jochen Mader

unread,
Mar 1, 2017, 7:59:43 AM3/1/17
to ve...@googlegroups.com
A colleague of mine (also a Karaf maintainer) created a Vert.x-Karaf shjowcase, maybe you want to check that one out:

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.

For more options, visit https://groups.google.com/d/optout.



--
Jochen Mader | Lead IT Consultant

codecentric AG | Elsenheimerstr. 55a | 80687 München | Deutschland
tel: +49 89 215486633 | fax: +49 89 215486699 | mobil: +49 152 51862390
www.codecentric.de | blog.codecentric.de | www.meettheexperts.de

Sitz der Gesellschaft: Düsseldorf | HRB 63043 | Amtsgericht Düsseldorf
Vorstand: Michael Hochgürtel . Rainer Vehns
Aufsichtsrat: Patric Fedlmeier (Vorsitzender) . Klaus Jäger . Jürgen Schütz
Reply all
Reply to author
Forward
0 new messages