Is data partitioned?

[serban.balamaci]

Hello everybody.

I have some questions if you would be kind to ask:
I want to clarify how the data in a distributed map/list is
replicated. I mean let's say we have two servers, the data it's
partitioned half on one machine and half on another(I understand maybe
it's not exactly half, but is it partitioned , or there is all the
data as a copy on all the servers), if another machine joins the group
will the data be redistributed and every machine will get 1/3 of the
data or will receive a whole copy of the map/list?

2. What happens when we have say two machine, and one machine goes
down? Will some data be lost?

3. If it's distributed on every machine and not any machine holds all
the data, to query a list for objects which meet a certain criteria
what would you recommend to distribute the search so that every
machine does the search on it's subset of data. I see that for map
there is entrySet(Predicate). Is there a solution for a List?

Thank you.

[Talip Ozturk]

> I have some questions if you would be kind to ask:
> I want to clarify how the data in a distributed map/list is
> replicated. I mean let's say we have two servers, the data it's
> partitioned half on one machine and half on another(I understand maybe
> it's not exactly half, but is it partitioned , or there is all the
> data as a copy on all the servers), if another machine joins the group
> will the data be redistributed and every machine will get 1/3 of the
> data or will receive a whole copy of the map/list?

Everything is partitioned and backed up. so when the 3rd machine
joins, each machine will eventually own 1/3 of the data, and hold
backup data of other nodes if backup is configured (default
backup-count is 1).

> 2. What happens when we have say two machine, and one machine goes
> down? Will some data be lost?

We won't lose any data (that is the whole purpose of Hazelcast) as we
already have backup for all data. Since you will have 2 machine
remaining, each machine will own/manage 1/2 of the entire data and
hold backup of the other node.

For more info, please see:
http://code.google.com/docreader/#p=hazelcast&s=hazelcast&t=InternalsDistributedMap

> 3. If it's distributed on every machine and not any machine holds all
> the data, to query a list for objects which meet a certain criteria
> what would you recommend to distribute the search so that every
> machine does the search on it's subset of data. I see that for map
> there is entrySet(Predicate). Is there a solution for a List?

Unfortunately no solution for List. Only for maps.

-talip

[Tim Wisniewski]

If all nodes own backups of the data, then how can the map be bigger
than the memory of any one node? Thanks.

-Tim

On Dec 11, 12:36 pm, Talip Ozturk <ta...@hazelcast.com> wrote:
> > I have some questions if you would be kind to ask:
> > I want to clarify how the data in a distributed map/list is
> > replicated. I mean let's say we have two servers, the data it's
> > partitioned half on one machine and half on another(I understand maybe
> > it's not exactly half, but is it partitioned , or there is all the
> > data as a copy on all the servers), if another machine joins the group
> > will the data be redistributed and every machine will get 1/3 of the
> > data or will receive a whole copy of the map/list?
>
> Everything is partitioned and backed up. so when the 3rd machine
> joins, each machine will eventually own 1/3 of the data, and hold
> backup data of other nodes if backup is configured (default
> backup-count is 1).
>
> > 2. What happens when we have say two machine, and one machine goes
> > down? Will some data be lost?
>
> We won't lose any data (that is the whole purpose of Hazelcast) as we
> already have backup for all data. Since you will have 2 machine
> remaining, each machine will own/manage 1/2 of the entire data and
> hold backup of the other node.
>

> For more info, please see:http://code.google.com/docreader/#p=hazelcast&s=hazelcast&t=Internals...

[Talip Ozturk]

all nodes doesn't have the backup of all data. each node is the backup
of previous member(s) only.
so let's say we have backup count=1, 4 member cluster and 4000
entries in out distributed map.

M2 is backup of M1
M3 is backup of M2
M4 is backup of M3
M1 is backup of M4

that means each member will own about 1000 entries and 1000 backup entries.

entire cluster has 4000 entries but each member has 2000 entries.

now imagine you have 20 member cluster.

-talip

> --
>
> You received this message because you are subscribed to the Google Groups "Hazelcast" group.
> To post to this group, send email to haze...@googlegroups.com.
> To unsubscribe from this group, send email to hazelcast+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/hazelcast?hl=en.
>
>
>

[Tim Wisniewski]

thanks for the clarification!

-Tim