Hazelcast 3.4 : Issues with MapLoader.loadAll() - not loading Map

[AVT]

I am very new to HC, and is checking the feasibility of using HC3.4 as a distributed (read-thru) cache for our app. I was trying with the MapLoader functionalities, and could see that load() loads data to the map as expected, but loadAll() doesnot. 

On executing the below client,  from the logs I can see that loadAll() is getting called and it returns the result. Also, client receives the result map with values(for all the 3 keys that I requested). But I don't see any entries in the TEST_MAP when checking with mancenter. On requesting for the second time, I see loadAll() getting invoked again which confirms that nothing is cached in TEST_MAP in the first try. What could be wrong? Any suggestions. Not sure if I understood HC correct.

MapLoader Code

@Override

public String load(String key) {

return key+"value";

}

@Override

public Map<String, String> loadAll(Collection<String> keys) {

Iterator<String> keyIter = keys.iterator();

Map<String, String> outputMap = new HashMap<String, String>(keys.size());

        while(keyIter.hasNext()){

        String key = keyIter.next();

        outputMap.put(key, key+"value");

}

       return outputMap;

}

@Override

public Set<String> loadAllKeys() {

      return null;

}

Client Code

IMap<String,String> objectMap = hcClientInstance.getMap("TEST_MAP");

Set<String> keySet = new HashSet<String>();

keySet.add("A");

keySet.add("B");

keySet.add("C");

Map<String, String> resultMap = objectMap.getAll(keySet);

Hazelcast.xml snippet

        <map name="TEST_MAP">

<in-memory-format>OBJECT</in-memory-format>

<backup-count>0</backup-count>

<async-backup-count>0</async-backup-count>

<time-to-live-seconds>600</time-to-live-seconds>

<max-idle-seconds>0</max-idle-seconds>

<eviction-policy>NONE</eviction-policy>

<max-size policy="PER_NODE">0</max-size>

<eviction-percentage>25</eviction-percentage>

<min-eviction-check-millis>100</min-eviction-check-millis>

<merge-policy>com.hazelcast.map.merge.PutIfAbsentMapMergePolicy</merge-policy>

<map-store enabled="true" initial-mode="LAZY">

<class-name>com.loader.TestLoader</class-name>

</map-store>

</map>

[Ali Gurbuz]

Hi

This seems like a bug, thank you for reporting.

I've opened an issue:
https://github.com/hazelcast/hazelcast/issues/4458

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/dd4f1efa-6da6-43cb-aa1b-0cf0cb416638%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Ali Gurbuz
Solutions Architect
Mahir İz Cad. No:35, Altunizade, İstanbul
a...@hazelcast.com 
+90 507 857 7815
@aligurbuz

[wen xiong]

hello

I am a freshman to Hazelcast, and i would like to use command line to learn Hazelcast, but where can i find the commandline in version hazelcast-3.5-SNAPSHOT.jar

thank you for your attention!

best wishes!

xiowgwen

[Ahmet Mircik]

Hi, 

Check out this https://github.com/hazelcast/hazelcast/blob/master/hazelcast/src/main/java/com/hazelcast/console/ConsoleApp.java

--

You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/54677eea-c1d5-4658-8efa-d789cf3d4f09%40googlegroups.com.

[AVT]

Thanks for looking into this Ali.

[mat...@kryptnostic.com]

This pretty painful. We hit this on 3.3 and upgrade to 3.4 hoping it was fixed, but to no avail. 

We are loading 1 million+ objects at startup and had switch to forcing a get in the constructor of our services, since the background loading seemed to die after loading a few thousand keys. This is also triggering serialization, which means startup time is like 30 minutes instead of 30 seconds.

-mtr

[jessie.ji...@gmail.com]

Does anyone have any update on this? I'm trying HC3.8.4 and noticed that explicitly calling IMAP.loadAll(true) doesn't load map from MapStore, the method MapStore.loadAll(true)  was not even called. Using getAll(set) doesn't seem to be an option, since we don't know all the keys. We just want to warm-up the cache at start up. Does it make sense? Or is there a different approach?

Thank you!

[Ahmet Mircik]

Hi,

What was the exact version? There is no 3.8.4. Latest release is 3.8.2. Also map-store doesn’t have MapStore.loadAll(true) method, maybe you want to mean MapStore.loadAll(Collection<K> keys) or IMap#loadAll(boolean replaceExistingValues)?

Could you please share a reproducer? Normally, IMAP.loadAll(true) should work.

--

You received this message because you are subscribed to the Google Groups "Hazelcast" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+unsubscribe@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.

Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/c79ba8bc-2713-4413-bb3c-ffebbda84158%40googlegroups.com.

​

[Jessie Lin]

Thank you for getting back. You're right. I am using 3.8.2.

 A sample program is below. 

MapStore.loadAll(Collection<K> keys) were called before loadAll(boolean replaceExistingValue)

It seems the loadAllKeys() and loadAll(Collection) are called when client.getMap(MapName) before it runs to loadAll(boolean),  even though I have the initial-mode in LAZY

public class HazelCastClient {

    public static void main( String[] args ) {

        try {

            ClientConfig clientConfig = new XmlClientConfigBuilder("hazelcast-client.xml").build();

            HazelcastInstance client = HazelcastClient.newHazelcastClient( clientConfig );

            IMap<String, SampleModel> mapSample = client.getMap("sampleMap");

            readTarget(mapSample);

        } catch (IOException e) {

            System.out.println("File Not found" + e.toString());

        }

    }

    public static void readTarget(IMap<String, SampleModel> cache) {

        if (cache.size() ==0) {

            System.out.println(new Date()+ "Target loaded start ----------------------------------------------------------");

            cache.loadAll(true);

            System.out.println(new Date()+ "Target loaded end ----------------------------------------------------------");

            System.out.println(new Date()+ "Target cache size =" + cache.size());

        }

    }

}

    <map name="sampleMap">

        <backup-count>0</backup-count>

        <async-backup-count>0</async-backup-count>

        <cache-deserialized-values>INDEX-ONLY</cache-deserialized-values>

        <map-store enabled="true" initial-mode="LAZY">

            <class-name>com.sample.SampleStore</class-name>

            <write-delay-seconds>10</write-delay-seconds>

            <write-batch-size>1000</write-batch-size>

            <write-coalescing>true</write-coalescing>

        </map-store>

        <indexes>

            <index ordered="false">col1</index>

            <index ordered="false">col2</index>

        </indexes>

    </map>

The same topic is discussed in this thread:

https://groups.google.com/forum/#!topic/hazelcast/3VQM7cT63hc

Thank you very much!

To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.

To post to this group, send email to haze...@googlegroups.com.

​

[Vassilis Bekiaris]

Hi Jessie,

when setting initial load mode to LAZY, the MapStore's initial loading is deferred to the first operation that touches the Map's partitions. That operation in your sample is cache.size(), so the condition cache.size() == 0 will be false --> cache.loadAll(true) is not even invoked, loading has already happened. You can verify this if you step through your code with a debugger.

Best,
Vassilis