MapLoader.loadAll problem

[Sam.Park]

Hi

I am encountering a very weird situation for loading data from
underlying layer
I implemented MapLoader.loadAllkeys() and MapLoader.loadAll(keys) to
load data
from database. When i invoked Hazelcast.getMap("mymap") ,
MapLoader.loadAll(keys)
called peoridically like infinite loop and never return Map instance.
In my implemented oadAll(keys) there is no recursive invocation. I
didn't configure
any cluster settings in my xml. I just initialized hazelcast in my
main test application

Regards,
Park

[Talip Ozturk]

Can you post test application to reproduce it?

http://twitter.com/oztalip

> --
> You received this message because you are subscribed to the Google Groups "Hazelcast" group.
> To post to this group, send email to haze...@googlegroups.com.
> To unsubscribe from this group, send email to hazelcast+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/hazelcast?hl=en.
>
>

[Sam.Park]

Thanks for reply

I will post my code when i get back to work i am now on vacation now

the codes in loadall method is just to create Hashmap filled with data
structures from JDBC connection

and return that instance

Regards,
Park

[Sam.Park]

Hi

I found why MapLoader.loadAll(keys) called multiple times . In my
maploader.loadAllKeys() method

retunrs about 40000 size of key set . And I noticed
MapLoader.loadAll(keys) is called 40 times

with keys parameter which contains only 1000 keys. It looks hazelcast
chunks the key set which gets from

loadAllKeys() and passed with part of keys instead of all keys. I
don't get how to get over this

Regards,
Park


On 5월8일, 오후5시39분, Talip Ozturk <ta...@hazelcast.com> wrote:

[Fuad Malikov]

After 40 invocations, doesn't it loads everything and returns?

Fuad

2011/5/12 Sam.Park <bbori...@gmail.com>

--
Fuad Malikov
Managing Partner, Hazelcast
http://www.hazelcast.com
+90.538.378.9777
http://www.linkedin.com/in/malikov
http://twitter.com/fuadm

[Sam.Park]

Yes It did. But I can load all data at first invocation so i doesn't
have to be called 40 times.

Regards,
Park

On 5월12일, 오후5시10분, Fuad Malikov <f...@hazelcast.com> wrote:
> After 40 invocations, doesn't it loads everything and returns?
>
> Fuad
>

> 2011/5/12 Sam.Park <bborie0...@gmail.com>

[Fuad Malikov]

How many members do you have in cluster?

Fuad

2011/5/12 Sam.Park <bbori...@gmail.com>

--
Fuad Malikov

[Mehmet Dogan]

MapLoader.loadAll(keys) called for chunks of keys because every node loads its own data separately. This is to prevent large data migrations between nodes upon load of data from store.  

@mmdogan

2011/5/12 Sam.Park <bbori...@gmail.com>

[englis...@gmail.com]

Hello everyone. I have almost the same issue with the twice calling loadAll() method.

Even having configured HZ only as single instance without any extra members

.... INFO: Cluster created with settings {hosts=[localhost:27017], mode=SINGLE ...

I still get loadAll method called twice. To be sure there is only one member in the cluster

I checked it by the code below directly in my MapStore implementation

Set<Member> sm = hazelcastInstance.getCluster().getMembers();

for(Member m : sm){

Map<String, Object> mm = m.getAttributes();

Set<Map.Entry<String, Object>> entrySet1 = mm.entrySet();

for(Map.Entry<String, Object> mmo : entrySet1){

System.out.println("Key: " + mmo.getKey());

System.out.println("Value: " + mmo.getValue());

}

}

According to the doc:

http://docs.hazelcast.org/docs/3.6/manual/html-single/index.html

Here is the MapLoader initialization flow:

1. When getMap() is first called from any member, initialization will start depending on the value ofInitialLoadMode. If it is set to EAGER, initialization starts. If it is set to LAZY, initialization does not start but data is loaded each time a partition loading completes.
2. Hazelcast will call MapLoader.loadAllKeys() to get all your keys on one of the members.
3. That member will distribute keys to all other members in batches.
4. Each member will load values of all its owned keys by calling MapLoader.loadAll(keys).
5. Each member puts its owned entries into the map by calling IMap.putTransient(key,value).

Could you please explain such a behaviour

HZ version use is 3.5

Thanks

[Enes Akar]

Hi;

This is expected behaviour as LoadAll is called per partition. With your first touch to partition, the partition calls loadAll. 

--

You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.

Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/7af25f70-1604-4fdb-af3d-b25295be73f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.