Guava cache with custom weigher

[Ashwin Jayaprakash]

Hi, I'm trying to configure a LoadingCache with large byte buffers (Netty ByteBufs) as the values. These byte buffers are loaded and processed from parts of larger compressed files. There will be many such large files so, there will be many cached buffers.

I configured the LoadingCache like this:

CacheBuilder.<Key, ByteBuf>newBuilder()
  .expireAfterAccess(expireAfterAccessSeconds, SECONDS)
  .maximumWeight(maxTotalUncompressedBytes)
  .weigher((key, byteBuf) -> byteBuf.readableBytes())

  //More config.

  .build(xxx);

I used a custom Weigher to relay the buffer's readable size/bytes as weight. I set the maximumWeight as 1GB as I wanted no more than 1GB of total cached buffer sizes.

When I loaded up a 350MB buffer it got evicted the moment I tried to load another similar sized buffer. The total accumulated size should've been 400MB + 350MB, which is still below 1GB. I dug through the code and realized that there are some calculations in LoadingCache that are not obvious to the user at all. The maximum weight gets divided across all the internal cache segments, so the threshold is set to lower than what the user thinks.

What happened was that the segment was setup internally to allow only 250MB weight. Anything larger, like the 400MB buffer would get marked for eviction almost immediately.

These were obtained from a debug session:
concurrencyLevel 4
maxWeight 1073741824
maxSegmentWeight 268435456

The relationship between concurencyLevel, segmentCount, maxWeight and maxSegmentWeight are not apparent and I'm wondering how to get around this issue. I could change the maximum weight or change the concurrency level and play with the numbers but the former is not intuitive and the latter will cause performance problems.

Any suggestions or help would be appreciated.

Thanks,

Ashwin.

[Kevin Bourrillion]

We will try to make this more clear in the Javadoc for CacheBuilder.maximumWeight. We feel bad that this has confused so many people.

--
guava-...@googlegroups.com
Project site: https://github.com/google/guava
This group: http://groups.google.com/group/guava-discuss
 
This list is for general discussion.
To report an issue: https://github.com/google/guava/issues/new
To get help: http://stackoverflow.com/questions/ask?tags=guava
---
You received this message because you are subscribed to the Google Groups "guava-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guava-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/guava-discuss/32226535-b467-4169-8da7-26da3e08214e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kevin Bourrillion | Java Librarian | Google, Inc. | kev...@google.com

[Benjamin Manes]

Unfortunately this is a limitation with a segmented cache. It works well when holding many items but poorly with only a few large ones. Changing this would be invasive and it is unlikely that the Guava team would want to do that.

The Java 8 rewrite doesn't have this problem, but is not a Google project.

https://github.com/ben-manes/caffeine

--


guava-...@googlegroups.com


Project site: https://github.com/google/guava


This group: http://groups.google.com/group/guava-discuss


 


This list is for general discussion.


To report an issue: https://github.com/google/guava/issues/new


To get help: http://stackoverflow.com/questions/ask?tags=guava


---


You received this message because you are subscribed to the Google Groups "guava-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guava-discus...@googlegroups.com.

[Ashwin Jayaprakash]

Thanks everyone.

I wish the cache had an option to not rely on the maxSegmentWeight when a custom Weigher was set. Maybe a sub-type of Weigher interface or a default Java 8 method in Weigher like:

public interface Weigher<K,V> {
   int weigh(K key, V value);

   default boolean distributeAcrossSegments() {
     return false;
   }
}

[Ben Manes]

Unfortunately that would mean there would be 1 segment, which could worsen performance as you indicated. To address it without a complete rewrite, the segments would have to coordinate to borrow slack from each other and likewise trigger other segment evictions to stay balanced. This is all because the caching logic is tightly coupled to ConcurrentHashMap, which in JDK8 was changed to use fine-grained buckets instead. To best adapt to that strategy and avoid the complexities of balancing segments, the new implementation decorates the map and uses a write buffer to maintain a single policy structure.

[Ashwin Jayaprakash]

Thanks. I'll give Caffeine a shot (maybe even a double shot :)