Concurrent Maps created with MapMaker seem to be quite fast!

[matt...@gmail.com]

I'm not sure if this is the right place to post this, but I'll give it a shot.  I've spent some of my free time writing a small benchmark for key/value pair storage implementations (maps, caches, etc.), and have found that, in the specific circumstances under which I tested them (4 threads, 2 million concurrent reads/writes), the ConcurrentMap produced by Guava seems to be faster than anything else I've found so far - slightly edging out the ConcurrentHashMap implementation found in Java SE 7 and considerably faster than the implementation of the same in Java SE 8.

I'm pretty modest about things like this, and will freely admit that there are probably holes in this test and there is probably plenty of room for improvement, but I wanted to share what I found and give those that created this Map a thumbs up!  If anyone sees any obvious room for improvement or flaws in the test, feel free to contribute to the project or share suggestions.

The source and results for everything I've tested so far are up on GitHub at: https://github.com/mkosem/testcache

Some sample results from a single set of test criteria (same conditions, same system, etc.) can be found below - with a more comprehensive set appearing in the readme for the project referenced above:

ConcurrentHashMap:

• Overall average write time: 176ns
• Overall average read time: 48ns

Guava ConcurrentHashMap:

• Overall average write time: 68ns
• Overall average read time: 45ns

Ehcache (LRU/eternal/heap only/max capacity set):

• Overall average write time: 774ns
• Overall average read time: 631ns

Guava Cache (initial/max capacity set):

• Overall average write time: 622ns
• Overall average read time: 134ns

Please note, these results detail observations under one set of circumstances on one system.  Any other circumstances on any other systems may yield dramatically different results, which may show drastically different findings.

--Matt

[matt...@gmail.com]

I added results from a 2x4-core w/HT box (2x Xeon x5520 CPUs and 16GB of DDR3-1066 CAS7 ram on Windows 7) to the readme in the project referenced above. Guava's Map remains on top with 16, 32, 64, and 128 threads on that box.

Here are some samples with 16 threads:

ConcurrentHashMap:
- Overall average write time: 104ns
- Overall average read time: 98ns

Guava ConcurrentHashMap:
- Overall average write time: 56ns
- Overall average read time: 71ns

Ehcache (LRU/eternal/heap only/max capacity set):

- Overall average write time: 317ns
- Overall average read time: 262ns

Guava Cache (initial/max capacity set):

- Overall average write time: 571ns
- Overall average read time: 132ns

And a subset of the tests on the same box, but with 32 threads:

ConcurrentHashMap:
Overall average write time: 72ns
Overall average read time: 98ns

Guava ConcurrentHashMap:
Overall average write time: 37ns
Overall average read time: 62ns

And the same tests, but with 64 threads:

ConcurrentHashMap:
Overall average write time: 57ns
Overall average read time: 95ns

Guava ConcurrentHashMap:
Overall average write time: 27ns
Overall average read time: 57ns

And the same tests, but with 128 threads:

ConcurrentHashMap:
Overall average write time: 43ns
Overall average read time: 90ns

Guava ConcurrentHashMap:
Overall average write time: 37ns
Overall average read time: 50ns

--Matt

[David Beaumont]

Thanks for posting these. It's really nice to know you're happy with Guava's performance :)

I wasn't involved in any of this, but I know a huge amount of careful effort and tuning went into these classes, so it's great that it seems to have paid off (and that our original assumptions still seem to be holding) !

Cheers,

    David

--Matt

--
--
guava-...@googlegroups.com
Project site: http://guava-libraries.googlecode.com
This group: http://groups.google.com/group/guava-discuss

This list is for general discussion.
To report an issue: http://code.google.com/p/guava-libraries/issues/entry
To get help: http://stackoverflow.com/questions/ask?tags=guava
---
You received this message because you are subscribed to the Google Groups "guava-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guava-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/guava-discuss/5b7fa998-8203-4eed-b455-db93fe32e2d9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
David Beaumont :: Îñţérñåţîöñåļîžåţîờñ Libraries :: Google
Google Switzerland GmbH., Brandschenkestrasse 110, CH-8002, Zürich - Switzerland
Tel +41 44 668 1800 :: Fax +41 44 668 1818

[Colin Decker]

I'm definitely a bit surprised to see a difference between a normal java.util.concurrent.ConcurrentHashMap and one returned by MapMaker. It looks like you're just setting the concurrency level and initial capacity on MapMaker, in which case... MapMaker will just return a normal ConcurrentHashMap!

--

--
guava-...@googlegroups.com
Project site: http://guava-libraries.googlecode.com
This group: http://groups.google.com/group/guava-discuss
 
This list is for general discussion.
To report an issue: http://code.google.com/p/guava-libraries/issues/entry
To get help: http://stackoverflow.com/questions/ask?tags=guava
---
You received this message because you are subscribed to the Google Groups "guava-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guava-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/guava-discuss/bcd06c33-c2db-4d66-bc11-0cd1e4165abb%40googlegroups.com.

[Roger Schildmeijer]

The ConcurrentHashMap returned from MapMaker will have the concurrency level set to 4 [1], the same number of threads used in the benchmark. The table is internally partitioned to try to permit the indicated number of concurrent updates without contention.

A default constructed ConcurrentHashMap will have concurrency level 16 [2]

This is why you see the map from Guava is faster.

But remember what the java doc says about the knob: "But overestimates and underestimates within an order of magnitude do not usually have much noticeable impact.”

Roger Schildmeijer

[1] http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/MapMaker.html

[2] http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentHashMap.html#ConcurrentHashMap%28int,%20float,%20int%29

To view this discussion on the web visit https://groups.google.com/d/msgid/guava-discuss/CAHJY%3Dp5-U8CkD2KN5n4E1unkJeHT8gufkGChg50CCzqNewHX0g%40mail.gmail.com.

[matt...@gmail.com]

Colin,

     I see that you're absolutely correct, shame on me for not investigating that further ahead of time.  That being the case, the only difference between the two tests that I documented is that the stock java ConcurrentHashMap is set to a Guava hard-coded fill factor of .75 rather than 1 in the tests labeled as being a Guava ConcurrentHashMap.  It seems "tuning" the map in an attempt to avoid it resizing during the test actually caused it to be somewhat slower.  It looks like I'd need to set maximumSize to use Guava's own implementation, but it is deprecated (presumably because CacheBuilder exists) and permissions discourage using it so I won't.

Roger,

     The concurrency level was set to the same value for both tests.  Colin hit the nail on the head.

--Matt

[matt...@gmail.com]

In any event Guava's caching implementation also performed quite well in my testing (which, in the case of "cache" implementations, was testing basic bounded storage functionality).  Read times were consistently ahead of the pack and writes only fell somewhat behind EHCache on the 8-core box (potentially because of its poor memory bandwidth, since this system's board has only 2 memory channels per CPU and the modules are low-spec).

And a subset of the tests on the same box, but with 32 threads:

 

Guava Cache (initial/max capacity set):

 - Overall average write time: 392ns

 - Overall average read time: 101ns

Ehcache (LRU/eternal/heap only/max capacity set):

 - Iteration 8 average write time: 244ns

 - Iteration 8 average read time: 284ns

 

And the same tests, but with 64 threads:

 

Guava Cache (initial/max capacity set):

 - Overall average write time: 268ns

 - Overall average read time: 83ns

Ehcache (LRU/eternal/heap only/max capacity set):

 - Overall average write time: 213ns

 - Overall average read time: 258ns

 

And the same tests, but with 128 threads:

 

Guava Cache (initial/max capacity set):

 - Overall average write time: 215ns

 - Overall average read time: 84ns

Ehcache (LRU/eternal/heap only/max capacity set):

 - Overall average write time: 192ns

 - Overall average read time: 239ns

--Matt