put() of IMap or MultiMap is too slow

[Hajime Takase]

Hi,

Somehow put() of either IMap or MultiMap are too slow to complete while ordinal Map or ConcurrentMap finish in just a second with few millions of keys.

I use version "3.3-RC2" and it takes around 5 seconds to put() 10000 key or value in standalone mode or cluster mode.I tried with 8G or 96G,but performance was quite same.

How can I improve the performance of put()?

Here is what I did (code is in Scala).

import java.util.concurrent._
import collection.JavaConverters._
import collection.mutable.HashMap
 
import com.hazelcast.client.HazelcastClient
import com.hazelcast.core.{MultiMap,IMap}
 
//calculate the execution time 
def time[A](a: => A) = {
        val now = System.nanoTime
        val result = a
        val micros = (System.nanoTime - now) / 1000
        println("%d microseconds".format(micros))
        result
      }
 
val instance = HazelcastClient.newHazelcastClient()
 
//hazelcast IMap or MultiMap
val hm: MultiMap[String,String] = instance.getMultiMap("testForPerformance")
time(Range(0,1000000).foreach(x=>hm.put("test",x.toString)))
time(Range(0,1000000).foreach(x=>hm.put(x.toString,"test")))
println(hm.keySet.asScala.toSeq.length)
println(hm.values.size)

//scala HashMap
val sm = new HashMap[String,String] 
time(Range(0,1000000).foreach(x=>sm.put("test",x.toString)))
time(Range(0,1000000).foreach(x=>sm.put(x.toString,"test")))
println(sm.keySet.toSeq.length)
println(sm.values.size) 

//scala ConcurrentMap 
val cm = new ConcurrentHashMap[String, String]
val cmm = cm.asScala
time(Range(0,1000000).foreach(x=>cmm.put("test",x.toString)))
time(Range(0,1000000).foreach(x=>cmm.put(x.toString,"test")))
println(cmm.keySet.toSeq.length)
println(cmm.values.size) 

[Peter Veentjer]

It is pointless to compare the performance of a distributed map with an in memory one.

Although they have the same interface, the have totally different implementations and features.

So if you don't need a distributedmap; use a concurrent map or hash map

If you need a distributed map; use e.g. a Hazelcast distributed map.

One minor improvement would be to use the IMap.set method; it doesn't return the old value and therefor is faster since less serialization is involved.

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/CAHm3ZsqUiwGWFtDuQR8ENF1fsfVO7x7OKwrsrp4kvxapL4yfnw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Hajime Takase]

I definitely need the distributed map as I want to run several map/reduce for the same data.

I understand that the implementations and features are different.

So can I think that this is the performance of put() for Hazelcast IMap or MultiMap?

I tried the set() for IMap but the performance improvement was quite little.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/CAGuAWdDyE%2BhM0tfS6XCnAKLLc1eJAWGKOoLYy5gmhHfJFL4XOA%40mail.gmail.com.

[Hajime Takase]

Well, I shall accept the performance of put() as written in here:

http://blog.hazelcast.com/2014/04/08/performance-top-5-1-map-put-vs-map-set

I use tmux to confirm that put() for same distributed Map on several processes of same node won't affect the performance (4 times better than single process put()).

I will check out that whether Multi Threading can boost the performance, if not dramatically.