Performance of Guava's caches.

[Luke Daley]

Hi,

We’ve recently been doing some performance work and utilising Guava’s caching support.

I recently had to look into cache performance for another project and discovered reasonable evidence that Guava’s caching utilities are generally not the best choice for hot caches, particularly when you just need a bounded concurrent safe cache. Most of this assumption is based on https://github.com/ben-manes/caffeine/wiki/Benchmarks and comparison with ConcurrentHashMap in Java 8. 

I didn’t do exhaustive testing, but I did do some, and found that https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/util/concurrent/BoundedConcurrentHashMap.java (which is based on the ConcurrentHashMap impl in Java 8, backported) had higher throughput than a Guava cache, at least for LRU eviction. Caffeine is not an option for Gradle as it requires Java 8, but that BoundedConcurrentHashMap impl will work on 6+.

@Lari: it’s more or less a drop in replacement AFAICT for our recent additions of Guava cache usage. It might be worth an experiment. 

[Benjamin Manes]

If you increase the concurrencyLevel of Guava's cache then the two should be nearly identical. If Guava accepted a long-standing patch then It would be significantly faster.

I'd be interested in hearing feedback on ConcurrentLinkedHashMap, which I think would be a better alternative. Its Java 6 compatible and spurred the development of BCHM, Guava, and Ehcache rewrites. Its still significantly faster in my experience.

It would also be interesting to know what the Gradle's hit rates are, as my recent focus has been improving that area.

[Lari Hotari]

Hi Ben,

It's great to get a reply from the author. ConcurrentLinkedHashMap is used in Grails for many internal caches, since 2010 (Marco Vermeulen's commit https://github.com/grails/grails-core/commit/55cf993b introduced it to Grails).
CLHM is great. However the feature I miss from CLHM is the CacheLoader interface that Guava Cache has. Using putIfAbsent is ugly.
Asynchrously refreshing expired values is a cool feature of Guava Cache as well, however that's not used in Gradle.

I don't have a separate benchmark in Gradle that shows the evidence that Guava Cache is a bottleneck. I'd like to do some JMH benchmarks for measuring, but I haven't done that (yet).

I've been profiling a Gradle build that we have in our performance tests. The test build has about 100000 source files and 100000 test source files in 99 sub-projects. There are also project dependencies and external dependencies. ("./gradlew :performance:largeJavaSwModelProject" generates the build to subprojects/performance/build/largeJavaSwModelProject in gradle core).

In the profiling I noticed that Guava Cache was showing up in the hotspots and I tried replacing Guava Cache with ConcurrentLinkedHashMap. The surprise was that I got 5-10% better results (shorter build time). I'd have to try the measurement once more to be sure about the improvement. It makes me wonder about it because Gradle has explicit locking around caches because of in-memory caches are just decorators around persistent caches that have to be guarded by an explicit lock when they are accessed.

I've been using Java Flight Recorder to do the profiling since that seems to work best for this case. I'm also using the "-XX:+DebugNonSafepoints" flag and that seems to help in getting more useful information.
I've used these JVM options for enabling Flight recorder:
export GRADLE_OPTS="-Dorg.gradle.jvmargs='-Xmx8g -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints'"

and using these scripts and JFR profile: https://github.com/lhotari/gradle-profiling/tree/master/jfr to get the dumps.

To analyze the profiling results, I'm using JMC and besides that I'm using Brendan Gregg's Flame Graphs (https://github.com/brendangregg/FlameGraph) to visualize the stack trace samples. I've have a custom tool https://github.com/lhotari/jfr-report-tool to get the relevant data from the JFR dump to the Flame Graph tool.

There is currently another cache in release branch that also uses Guava Cache.
https://github.com/gradle/gradle/blob/f77a4b6556558c9c169039fc0f4d366d720afc6b/subprojects/core/src/main/groovy/org/gradle/api/tasks/util/internal/CachingPatternSpecFactory.java#L42-L43
The hit rates on that cache is very high for Gradle builds with a lot of files. I haven't tried to replace that with other implementations like ConcurrentLinkedHashMap. I'd rather not use CLHM because it doesn't have the CacheLoader support that Guava Cache has. Creating a benchmark with JMH might be a better way to optimize a certain class instead of trying to profile it as part of a full use case (Gradle build). I assume optimizing that isn't a top priority task currently and it might take some time before I get back to it.

-Lari

--
You received this message because you are subscribed to the Google Groups "gradle-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gradle-dev+...@googlegroups.com.
To post to this group, send email to gradl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gradle-dev/2f81e522-49af-4595-afe9-b43deb0a3ff6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Lari Hotari

lari....@gradle.com
skype: lhotari , https://twitter.com/@lhotari

tel: +1-519-374-5876

[Benjamin Manes]

Hi Lari,

When I started CLHM there were few scalable solutions had been worked out so to draw from. I favored decorators to keep features focused and iterate on refining them in isolation. If you don't want to use the putIfAbsent+FutureTask trick, then lock striping is an easy substitute for making a self populating cache. I'd rather fix Guava or promote Caffeine than backport features onto CLHM.

Guava doesn't handle hot reads very well due to contention and creating garbage. I'd guess that GC pressure is a main culprit in the performance difference. I'd increase the concurrency level (default 4) for any cache you expected to be heavily used.

Another option, which is a hack, would be to use Caffeine if JDK8+ and rely on class loading to be backwards compatible. The Guava adapters would make it easy to choose an implementation based on the JVM version. This would also let Gradle leverage v2's new eviction policy, which would improve the hit rate and let you reduce the cache size. You can try the snapshot if you'd like to play with the latest.

You're welcome to push Guava to fix the performance issue (https://github.com/google/guava/issues/2063#issuecomment-107169736). Its been known for years, but I've had no success getting changes in from outside Google.

hope that helps,

Ben

[Luke Daley]

On 30 Oct 2015, at 00:24, Lari Hotari <lari....@gradle.com> wrote:

Hi Ben,

It's great to get a reply from the author. ConcurrentLinkedHashMap is used in Grails for many internal caches, since 2010 (Marco Vermeulen's commit https://github.com/grails/grails-core/commit/55cf993b introduced it to Grails).
CLHM is great. However the feature I miss from CLHM is the CacheLoader interface that Guava Cache has. Using putIfAbsent is ugly.

Asynchrously refreshing expired values is a cool feature of Guava Cache as well, however that's not used in Gradle.

I don't have a separate benchmark in Gradle that shows the evidence that Guava Cache is a bottleneck. I'd like to do some JMH benchmarks for measuring, but I haven't done that (yet).

I've been profiling a Gradle build that we have in our performance tests. The test build has about 100000 source files and 100000 test source files in 99 sub-projects. There are also project dependencies and external dependencies. ("./gradlew :performance:largeJavaSwModelProject" generates the build to subprojects/performance/build/largeJavaSwModelProject in gradle core).

In the profiling I noticed that Guava Cache was showing up in the hotspots and I tried replacing Guava Cache with ConcurrentLinkedHashMap. The surprise was that I got 5-10% better results (shorter build time). I'd have to try the measurement once more to be sure about the improvement. It makes me wonder about it because Gradle has explicit locking around caches because of in-memory caches are just decorators around persistent caches that have to be guarded by an explicit lock when they are accessed.

I've been using Java Flight Recorder to do the profiling since that seems to work best for this case. I'm also using the "-XX:+DebugNonSafepoints" flag and that seems to help in getting more useful information.
I've used these JVM options for enabling Flight recorder:
export GRADLE_OPTS="-Dorg.gradle.jvmargs='-Xmx8g -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints'"

and using these scripts and JFR profile: https://github.com/lhotari/gradle-profiling/tree/master/jfr to get the dumps.

To analyze the profiling results, I'm using JMC and besides that I'm using Brendan Gregg's Flame Graphs (https://github.com/brendangregg/FlameGraph) to visualize the stack trace samples. I've have a custom tool https://github.com/lhotari/jfr-report-tool to get the relevant data from the JFR dump to the Flame Graph tool.

There is currently another cache in release branch that also uses Guava Cache.
https://github.com/gradle/gradle/blob/f77a4b6556558c9c169039fc0f4d366d720afc6b/subprojects/core/src/main/groovy/org/gradle/api/tasks/util/internal/CachingPatternSpecFactory.java#L42-L43
The hit rates on that cache is very high for Gradle builds with a lot of files. I haven't tried to replace that with other implementations like ConcurrentLinkedHashMap. I'd rather not use CLHM because it doesn't have the CacheLoader support that Guava Cache has.

Though that's not needed at all here. CLHM would fulfil what is needed for that cache.

To view this discussion on the web visit https://groups.google.com/d/msgid/gradle-dev/CAK_qiZcgqW-y6TW0ZT2fKdbOn5SXTCWfA06y-GNfExgcUzuXxQ%40mail.gmail.com.

[Benjamin Manes]

Since Gradle 5.0 is Java 8-based, the usages of Guava's cache can be replaced with Caffeine. There seem to be 25 files importing it - are any of these performance critical like in your original testing Luke / Lari? If so, you can switch over without much complexity. You may want to set to Caffeine.executor(Runnable::run) if callbacks (like RemovalListener) need to run on a known thread (vs ForkJoinPool). I'd also advise reviewing your ample usage of soft/weak references, since soft is fairly a poor approach in most cases.

[Lóránt Pintér]

Thanks for keeping track of this for so long! :) We've mostly decommissioned the developer list in favor of GitHub issues. Let's continue the conversation here:

https://github.com/gradle/gradle/issues/7757

To view this discussion on the web visit https://groups.google.com/d/msgid/gradle-dev/689dae24-5fcf-4399-be6b-594cc91e8d17%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Lorant