Hazelblast

[peter veentjer]

Hi Guys,

the last few weeks I have been working on a project called Hazelblast.
A project that adds support to Hazelcast to use remote interfaces. And
I would look to have some feedback.

The project is started because at Cloudsoft we are looking at various
datagrids and of course Hazelcast is on the top of my list, but some
functionality was missing.

The idea is that you create an api package that is shared between
'client' (the caller) and 'server' (the callee). This api package
could contain something like:


@RemoteInterface
public interface EmployeeService {

@Partitioned
void fire(@RoutingId String id);

@ForkJoin (reducer = VoidReducer.class)
void printStatisticsOnAllNodes();

@LoadBalanced
void printStatisticsOnOneNode();
}

The methods are a bit bogus and need to work on a better example, but
I hope you get the picture.

The client can create a proxy to this service using:

ProxyProvider proxyProvider = new ProxyProvider()
EmployeeService employeeService =
proxyProvider.getProxy(EmployeeService.class)
employeeService.fire("1234")

The ProxyProvider will transform this to appropriate calls using
hazelcast.

Currently there are 3 method annotations;
* partitioned; so sending it to the right machine using one of the
arguments as routing key
* fork join: all machines get the call
* load balanced: only one of the machines gets the call

Hazelblast also takes care of running the services on all nodes. This
can be done through Spring, POJO based but it is very easy to add
support for other frameworks like Guice, should that ever be needed.

One of the reasons to have the partition call is that we can keep
functionality and data on a single machine, so we can benefit from
data locality.

There is some stuff that needs to be done before a 0.1 release, but I
plan to complete it before the end of next month.

[peter veentjer]

And I guess a link is handy:

https://github.com/pveentjer/Hazelblast

[Tim Peierls]

Peter,

I took a look at this last week, and I like the idea. 

I can't use it as it stands, however, because there's no hook for me to override the task serialization. As you probably know, Hazelcast requires either Java serialization -- which means deep serializability for object graphs sent to other nodes -- or DataSerializable for the root object. My custom DataSerializable implementation uses Jackson's ObjectMapper to serialize an object graph into Jackson's binary Smile format. The advantage is that only root object, which can even be a generic holder type, needs to extend DataSerializable. The other objects don't even need to be java.io.Serializable.

So I'm manually creating tasks that embody two of your three flavors (@Partitioned and @LoadBalanced), and I'm using a special invoke method that knows how create and execute an DistributedTask given one of my tasks. It's not as pretty, but it isn't terrible.

I'm also using Guice, so if I were to use Hazelblast I'd have to write my own ProxyProvider implementation. (I didn't get how/if that can be extended, btw.)

--tim

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To post to this group, send email to haze...@googlegroups.com.
To unsubscribe from this group, send email to hazelcast+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/hazelcast?hl=en.

[Peter Veentjer]

On Wed, Apr 25, 2012 at 6:32 PM, Tim Peierls <t...@peierls.net> wrote:
> Peter,
>
> I took a look at this last week, and I like the idea.
>
> I can't use it as it stands, however, because there's no hook for me to
> override the task serialization. As you probably know, Hazelcast requires
> either Java serialization -- which means deep serializability for object
> graphs sent to other nodes -- or DataSerializable for the root object. My
> custom DataSerializable implementation uses Jackson's ObjectMapper to
> serialize an object graph into Jackson's binary Smile format. The advantage
> is that only root object, which can even be a generic holder type, needs to
> extend DataSerializable. The other objects don't even need to be
> java.io.Serializable.

That should not be hard to fix I guess so that the ProxyProvider can make
use of different serialization mechanisms.

>
> So I'm manually creating tasks that embody two of your three flavors
> (@Partitioned and @LoadBalanced), and I'm using a special invoke method that
> knows how create and execute an DistributedTask given one of my tasks. It's
> not as pretty, but it isn't terrible.
>
> I'm also using Guice, so if I were to use Hazelblast I'd have to write my
> own ProxyProvider implementation. (I didn't get how/if that can be extended,
> btw.)

Currently I only have the different di frameworks at the server side,
spring, POJO based
and Guice could be added here as well by creating a GuicePuFactory
implementation (check
the SpringPuFactory).

But I have the impression you are not talking serverside, correct? But
you are talking about the
'client' side (the side where the proxy provider lives).

[Tim Peierls]

On Thu, Apr 26, 2012 at 12:43 AM, Peter Veentjer <alarm...@gmail.com> wrote:

> I'm also using Guice, so if I were to use Hazelblast I'd have to write my

> own ProxyProvider implementation. (I didn't get how/if that can be extended,
> btw.)

Currently I only have the different di frameworks at the server side, spring, POJO based
and Guice could be added here as well by creating a GuicePuFactory implementation (check
the SpringPuFactory).

But I have the impression you are not talking serverside, correct? But you are talking about the
'client' side (the side where the proxy provider lives).

No, I did mean the server side, and now you've told me the answer: I'd have to write a GuicePuFactory implementation and register it somehow. 

--tim

[Talip Ozturk]

Should this stuff be part of Hazelcast? Hazelcast Invocation Service:

InvocationService is = Hazelcast.getInvocationService();
EmployeeService employeeService = is.getProxy(EmployeeService.class)
employeeService.fire("1234")

we can have a configuration for how the proxies are decorated:

<invocation-service>
//optional
<proxy-factory>SpringProxyFactory|GuiceProxyFactory|DefaultProxyFactory</proxy-factory>
</invocation-service>

-talip

[peter veentjer]

He Tim,

it can be done through the PuServer, which can be embedded or be run
standalone.

The thing that needs to be passed is the puFactory if you want to run
the standalone version, e.g.

-puFactory com.hazelblast.server.spring.SpringPuFactory

If you want I can give you commit rights on Hazelblast so you can add
the Guice pu factory.

On Apr 26, 3:26 pm, Tim Peierls <t...@peierls.net> wrote:

[mongonix]

Hi Peter,

Very nice project!

I have a small remark:

 

    @Partitioned
    void fire(@RoutingId String id);

I'd suggest that you also support the case, where one of the passed parameters is implementing PartitionAware interface. In such a case, you can simply use its PartitionAware.getParitionKey() to get the partition id.

I.e. you get something like:

@Partitioned
void fire(@PartitionAwareId PartitionAwareDerived partAwareObj, other params);

I'm not sure about the @PartitionAwareId annotation. May be it is not needed at all, as you can just check the type of a parameter and see if it implements the PartitionAware interface.

What do you think?
-Leo

[Talip Ozturk]

> I'm not sure about the @PartitionAwareId annotation. May be it is not needed
> at all, as you can just check the type of a parameter and see if it
> implements the PartitionAware interface.

Nice thing about this annotation is that none of the parameters should
implement PartitionAware also I think method can have many parameters
and we shouldn't check for all of them. It is more explicit and I
personally like it.

Other comments:

I would use @PartitionKey instead of @RoutingId. It matches nicely
with our naming.

Besides, I don't think @Partition attribute is needed. If no method
attribute is defined and we can simply look for the parameter with
@PartitionKey attribute. If found, send the invocation over to that
partition, otherwise randomly pick a target node.

I don't like ProcessingUnit (PU) naming as what we are doing here is
pure MethodInvocation in cluster. So that this is more like Clustered
RMI.

-talip

[peter veentjer]

Hi mongonix,

I placed comments inline.

On Apr 26, 3:46 pm, mongonix <romix...@gmail.com> wrote:
> Hi Peter,
>
> Very nice project!

Thank you.

>
> I have a small remark:
>
> >     @Partitioned
> >     void fire(@RoutingId String id);
>
> I'd suggest that you also support the case, where one of the passed
> parameters is implementing PartitionAware interface. In such a case, you
> can simply use its PartitionAware.getParitionKey() to get the partition id.

I can do that.

It needs some cleanup anyway. Sometimes you have a composed object and
you don't want to use the hash of the object itself, but have a bit
more control.

With open spaced you can say:

void hire(@RoutingId(property = "employeeId") Employee e){....}

void fire(@RoutingId employeeId){...}

The example is bogus, but it shows that want to have different types
of objects, using the same routing. In the first example, the hash of
the employee isn't used, but the field 'employeeId'.

But the same thing can be reached by letting the Employee implement
the PartitionAware interface so that it returns the employeeId as
partition key, but you don't always have ownership of
the objects (so you can't always add an additional interface).

But I agree, there is certainly room for improvement. I'll add it as a
feature for 0.1.

>
> I.e. you get something like:
>
> @Partitioned
> void fire(@PartitionAwareId PartitionAwareDerived partAwareObj, other
> params);
>
> I'm not sure about the @PartitionAwareId annotation. May be it is not
> needed at all, as you can just check the type of a parameter and see if it
> implements the PartitionAware interface.
>
> What do you think?

In this case it can be inferred.

Personally I do like to have it explicit. If you don't add an
annotation,
you need to look at the parameter classes to figure out which argument
is
responsible for the routing.

And... code changes over time.. what if another arguments later also
starts
to implement the routing aware.. then the assumption that argument x
was
responsible, secretly gets violated because argument y becomes
responsible.

> -Leo

what do you think?

[peter veentjer]

On Apr 26, 4:15 pm, Talip Ozturk <ta...@hazelcast.com> wrote:
> > I'm not sure about the @PartitionAwareId annotation. May be it is not needed
> > at all, as you can just check the type of a parameter and see if it
> > implements the PartitionAware interface.
>
> Nice thing about this annotation is that none of the parameters should
> implement PartitionAware also I think method can have many parameters
> and we shouldn't check for all of them. It is more explicit and I
> personally like it.
>
> Other comments:
>
> I would use @PartitionKey instead of @RoutingId. It matches nicely
> with our naming.

Done :)

>
> Besides, I don't think @Partition attribute is needed. If no method
> attribute is defined and we can simply look for the parameter with
> @PartitionKey attribute. If found, send the invocation over to that
> partition, otherwise randomly pick a target node.

We can infer a lot. The question is if that is going to help write
understandable
code. I have been working on a project where from time to time it was
quite vague
what was executed where.. this is really killing. Personally I would
make routing
stuff as explicit as possible to prevent problems.

>
> I don't like ProcessingUnit (PU) naming as what we are doing here is
> pure MethodInvocation in cluster. So that this is more like Clustered
> RMI.

ProcessingUnit name comes from Gigaspaces. But I don't mind changes
it
to something else.

What do you and others have in mind?

>
> -talip

[mongonix]

Hi Peter,

See my comments inline.

On Thursday, April 26, 2012 3:32:03 PM UTC+2, peter veentjer wrote:

I buy your argument about using an explicit annotation. It is safer, indeed.
Of course, in case of only parameter it is a bit of an overkill and may be should be handled differently.

Regarding using PartitionAware:

I guess with your annotation you could easily do something like:

1) void fire(@RoutingId PartitionAwareDerived e){....}  - annotation indicates that this parameter should be used to figure out the key. It is inferred from the type of the parameter that it is PartitionAware and the key is obtained by an API call. If e would be a String (or any other basic type which can service as a key) its value would be taken as is.

2) void fire(@RoutingId(partitionAware=true) PartitionAwareDerived e){....} - here the annotation is very explicit. Uses a special annotation "flag" to indicate that the object is partitionAware

3) void fire(@RoutingId(property = "partitionKey") PartitionAwareDerived e) {} - uses your proposed syntax. No special need for the framework to know anything about PartitionAware. Treat it as any other POJO bean.

- Leo

[Talip Ozturk]

PartitionKey (~RountigId) doesn't have to PartitionAwareDerived at
all. It could be a simple object.

void fire (@PartitionKey Object mySimpleKeyObject, ...other params);

-talip

> --
> You received this message because you are subscribed to the Google Groups
> "Hazelcast" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/hazelcast/-/diRxz4V6PHAJ.

[mongonix]

Talip,

On Thursday, April 26, 2012 4:35:43 PM UTC+2, Talip Ozturk wrote:

PartitionKey (~RountigId) doesn't have to PartitionAwareDerived at
all. It could be a simple object.

void fire (@PartitionKey Object mySimpleKeyObject, ...other params);

I think there is a misunderstanding here. Please read my posts. I never said that the class of the @PartitionKey-annotated parameter HAS always to be derived from PartitionAware.

I just proposed that if a @PartitionKey-annotated parameter HAPPENs to be derived from PartitionAware, then the system should be clever enough to invoke the getPartitionKey on it to figure out which partition is to be used. I suggested it purely for a convenience reasons.

If class of the parameter is not derived from PartitionAware then it is handled in the usual way (i.e. its value or one of its properties is taken), as Peter proposed right from the beginning.

I hope this explanation makes things clear.

-Leo

[peter veentjer]

What if we do the following:

So:
1) if an argument doesn't implement PartitionAware (e.g. a string) and
no additional information is provided in the @PartitionKey annotation
(this is the new name for @RoutingId) then the whole argument is used
as the partition key.
2) if an argument does implement PartitionAware and no additional
information is provided in the @PartitionKey, then the
getPartitionKey() method is called on the argument to determine the
partition key.
3) if an argument doesn't implement PartitionAware but an additional
setting is used on the @PartitionKey annotation .e.g
void fire(@PartitionKey(property = "id") Employee e)
Then we are going to use that property (getter method or public field
for example) to determine the partition key.
4) if an argument implements PartitionAware and an property is
configured on the @PartitionKey annotation, than the property value is
used and the fact that the argument is partition aware is ignored.

Does this sound like a good approach?

I want to postpone the discussion about if we need that @PartitionKey
annotation at all.

[mongonix]

> Does this sound like a good approach?

Yes. Sounds good!

-Leo

[Peter Veentjer]

On Wed, Apr 25, 2012 at 6:32 PM, Tim Peierls <t...@peierls.net> wrote:
> Peter,
>
> I took a look at this last week, and I like the idea.
>
> I can't use it as it stands, however, because there's no hook for me to
> override the task serialization. As you probably know, Hazelcast requires
> either Java serialization -- which means deep serializability for object
> graphs sent to other nodes -- or DataSerializable for the root object.

Hi Tim,

what do you need exactly for this to work because I want to fix it asap.

What I can do is to provide different PartitionedMethodInvocation
implementations that differ only in the serialization mechanism
implemented, e.g. java Serializable, Hazelcast DataSerializable etc.

Is there any reason at all to use the java Serialization mechanism?

[Peter Veentjer]

Hi Rakesh,

the project has been halted for lack of time. But I want to update it to Hazelcast 3.0.

I hope to have serious more time for Hazelcast and Hazelcast support projects in the near future.

On Sun, May 12, 2013 at 7:06 PM, Rakesh Waghela <java...@gmail.com> wrote:

It seems development is halted for almost a year on Hazelblast. Does it mean it is mature and without significant bugs and compatible to latest hazelcast ?

--

You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.

Visit this group at http://groups.google.com/group/hazelcast?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.