Peer 2 Peer message + replica map

[ivan]

I've spent 2 days comparing cache and jms solutions. I think this one
is the simplest to start so far thanks to the nice docs and the one
jar approach.

Unfortunately I couldn't find a way to send message to a peer, not to
topic. Is it planned feature for a next release ?

The other thing I couldn't find was how to do a replicated map instead
of partitioned. I am about to use 1000 objects, but need them as close
as possible. Is there something like this ? Playing with the config
file and the backup property ?

Is there any limit of used object instances, I mean if I have
something like:

IMap map1 = Hazelcast.getMap("map1");
...
IMap map999 = Hazelcast.getMap("map999");

would that bring any performance burden ?

Probably I have more Q, but I would ask them later.

[Talip Ozturk]

> Unfortunately I couldn't find a way to send message to a peer, not to
> topic. Is it planned feature for a next release ?

it is not planned. can you tell us why you would need it? How do you
plan to handle the case where message reaches to the target peer but
the peer dies before processing it?

I would not recommend doing peer targeted things at all, even if there
is way for it. It is fragile. We should see the cluster as a single
system image where peers are loosely related.

> The other thing I couldn't find was how to do a replicated map instead
> of partitioned. I am about to use 1000 objects, but need them as close
> as possible. Is there something like this ? Playing with the config
> file and the backup property ?

I did the first version of ReplicatedMapFactory for it. There are
still things to optimize but it should give you an idea.
http://code.google.com/p/hazelcast/source/browse/trunk/hazelcast/src/main/java/com/hazelcast/impl/ReplicatedMapFactory.java

> Is there any limit of used object instances, I mean if I have
> something like:
>
> IMap map1 = Hazelcast.getMap("map1");
> ...
> IMap map999 = Hazelcast.getMap("map999");
>
> would that bring any performance burden ?

that should not be a problem as long as you are destroying the maps
after you are done with them by calling IMap.destroy()

Regards,
-talip

[ivan]

Thanks for the answers. I'll explain what a great feature would be
peer to peer. Probably you are seeing the things from problem point of
view and what would happen if this node dies or something else
happens, but from my POV nodes are 99% reliable, I have done in the
past my personal distributed system that has been running on a couple
of nodes and they never died for years. So in my case I would say they
were 100% reliable. I am not saying that we have to program in such a
optimistic way, believing nothing will happen, but sometimes we will
miss great features, that can make our live much much simpler, because
their implementation is only 99.99% reliable :-) Now let me tell you a
bit of my application.

I am building a game, where players can connect through socket to what
I call "frontend" servers. Think of it as a public gateways. So we
have 5K guys for example being connected to 5 servers. Log-in goes
there, so in every server we can associate a socket connection with a
player object, which has an id. On a number of other servers we have
games running, so we have services running. Every service has an id
also. This way, when Mr Bob makes a decision hitting a button on his
PC, the message goes to the player object on a frontend server and
then can be routed to a service by id, the frontend doesn't care where
the service is. In the header of the message we can place the id of
the player, so the service can reply to the sender. This is all
asynchronous messaging. I guess JMS is all about that.

Now that I have this idea in my head and I am looking for open source
solutions. Things are not perfect. Most of the JMS frameworks need a
message broker running, so it will be a single point of failure. On
top of that they are quite complicated to use. And I have narrowed my
choices to Hazelcast, JGroups and something called MantaRay, claiming
it doesn't need a message broker. And yes, your product needless to
say looks quite easy to follow.

I guess in implementing such a peer to peer feature, you can always
add a callback idea, so in case the receiver dies, sender will be
notified that message is undeliverable. Something like the way emails
are working. Anyway, this post has become quite long, but that's my
idea. And in case it is just 50 lines of code, utilizing the available
architecture it would be a nice feature and people will decide should
they rely on it or no.

[ivan]

If we have 2 services with ids 1000 and 2000 located somewhere in the
cluster and they want to talk to each other, then we may have
something like this.

In the receiving service:

IPeerNetwork myNetwork = Hazelcast.getPeerNetwork("myNetwork");
myNetwork.register(this.id, this); // this.id is 1000

In the sending service:

IPeerNetwork myNetwork = Hazelcast.getPeerNetwork("myNetwork");
myNetwork.sendMessage("1000", new Message(),
new FailedMessageEventListener() {
public void onFailedMessage(FailedMessageEvent event) {
Message src = event.getMessage();
// failed message code logic
}});

When I say it may needs 50 lines of code I mean that even now we can
create a workaround by having in the receiving service:
Topic topic = Hazelcast.getTopic ("1000");
topic.addMessageListener(this);

and then in the sending:
Topic topic = Hazelcast.getTopic ("1000");

But for thousands of services, then we will have thousands of topics,
which personally I don't know is it a good choice. What do you think ?
topic.publish (new Message());

[Talip Ozturk]

Ivan,

Sorry "again" for the late response. I understand your concern with
creating thousands of topic and it makes sense. Here is how I would do
it.

Instead of using Topic (or even IPeerNetwork), I would use message
Map. Your ids (1000, 2000...etc.) can be the keys in the message map.

IMap mapMessages = Hazelcast.getMap ("messages");
mapMessages.put ("1000",new Message());

and whoever interested in "1000" can add a listener for it to receive events..

mapMessages.addEntryListener ("1000", new EntryListener () {

public void entryAdded(EntryEvent event) {

}

public void entryAdded(EntryEvent event) {

}
......
}

now.. you just combined thousands of topics into a single distributed
map. + you can still use the other powerful map features like
putIfAbsent, lock etc. if needed.

make sense?

Regards,
-talip

[ivan]

Not really. It would be a nice workaround, but probably not an
effective solution. If I have 10 nodes and if node1 wants to send a
message to node7, then all 10 nodes will get the message, not only the
recipient. This will happen in case we have replicated map. What will
happen in case we have the normal map I even don't know.

[Talip Ozturk]

How about this?

public void sendMessageToSpecificMember (Member target, Message message) {
IMap mapMessages = Hazelcast.getMap ("node-specific-messages");
mapMessages.put (member,new Message());
}

Each node will start with the following listener:
public void whenMemberStarts() {
IMap mapMessages = Hazelcast.getMap ("node-specific-messages");
Member memberThis = Hazelcast.getCluster().getLocalMember();
mapMessages.addEntryListener (memberThis, new EntryListener () {

public void entryAdded(EntryEvent event) {

}

public void entryAdded(EntryEvent event) {

}
......
}
}

Now you can send a message to a specific member and only target member
will receive it. This will scale quite well, even if you have 100
member cluster.

Regards,
-talip