announcing CacheBuilder, the MapMaker replacement

[Charles Fry]

tl;dr: common.cache.CacheBuilder is better at making caches than MapMaker

There is a good chance that you’ve already heard of MapMaker. You may also be aware that it has been accumulating more and more caching functionality, to the point where it has outgrown its original role as a “collection”, ultimately meriting its own package. With release 10.0 we are introducing CacheBuilder and the corresponding Cache interface as a replacement for the caching functionality in MapMaker. The API remains virtually the same:

Before:

ConcurrentMap<Key, Graph> graphs = new MapMaker()    .weakKeys()
   .maximumSize(10000)
   .makeComputingMap(
       new Function<Key, Graph>() {
         public Graph apply(Key key) {

           try {              return createExpensiveGraph(key);

           } catch (SomeCheckedException e) {

             // now what

           }
         }
       });

After:

Cache<Key, Graph> graphs = CacheBuilder.newBuilder()    .weakKeys()
   .maximumSize(10000)
   .build(
       new CacheLoader<Key, Graph>() {
         public Graph load(Key key) throws SomeCheckedException {            return createExpensiveGraph(key);
         }
       });

The cache package defines a new CacheLoader interface which makes it easier to deal with checked exceptions which occur when loading new values into the cache, as shown above. The new Cache interface will allow us to expose caching logic on the returned instance. For example, we now expose detailed statistics about cache performance.

CacheBuilder already has a long list of upcoming features, including:

• Weighted entries
• Bulk operations
• Support for refreshing stale entries

We've already deprecated the caching functionality in MapMaker, in favor of CacheBuilder. Per Guava's deprecation policy, Beta functionality will be removed in release 11.0, and non-Beta functionality will remain for 18 months from the date the deprecation was submitted. Please bring to our attention any issues you encounter migrating code from MapMaker to CacheBuilder!

thanks,

Charles

[Charles Fry]

Oh, and I forgot to add that if you're attending the StrangeLoop conference I'll be giving a talk about the implementation details of size, time, and memory based eviction within CacheBuilder:

Concurrent Caching at Google

Monday, September 19th, 10:30-11:20am

https://thestrangeloop.com/sessions/concurrent-caching-at-google

[Philipp Wendler]

Hello,

Am 12.09.2011 19:16, schrieb Charles Fry:
> tl;dr: common.cache.CacheBuilder is better at making caches than MapMaker

[...]

> For example, we now expose detailed

> statistics<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/cache/CacheStats.html>about
> cache performance.

CacheBuilder really looks cool, especially the statistics. Thanks!

However, one thing I wonder about (which also applies to MapMaker) is
whether I have a performance penalty compared to a simple HashMap when I
am in a strictly sequential environment (not even concurrent reads)?

Of course I would set concurrency level to 1, but as this allows
concurrent reads, there probably is still some synchronization.
Does anyone know about that or do I have to try it out?

And if there is indeed a performance penalty: Would there be a chance to
have support for a non-current map/cache from MapMaker/CacheBuilder?

Greetings, Philipp

[Charles Fry]

Right now we use ReentrantLock. In the future we are considering implementing concurrencyLevel(0), but we don't have any hard numbers about the potential gain. If all you want is a Map then I'd stick with HashMap. If what you want is a cache then I'd consider CacheBuilder.

Charles

--
guava-...@googlegroups.com
Project site: http://guava-libraries.googlecode.com
This group: http://groups.google.com/group/guava-discuss

This list is for general discussion.
To report an issue: http://code.google.com/p/guava-libraries/issues/entry
To get help: http://stackoverflow.com/questions/ask (use the tag "guava")

[Mark Derricutt]

+1 - looks nice.  One thing I've kinda wanted when using MapMaker as a cache is having other means of determining an eviction policy.  Currently there's only expireAfterAccess/expireAfterWrite - where's I've occasionally come across places where I'd love to have expireAfterPredicate(Predicate<V> p).

For example - whilst not the exact area I had in mind, this is a simplistic retelling from a hazy memory - we have an interface in our system called Expirable which has a from/thru timestamp which governs the lifespan of an entity - we have a IsExpiredPredicate which is often used for filtering old items from a collection.

It would handy to have say a Cache<Key,Expirable> - a computing map could load the values from the database as needed - excluding any expired ones, but the predicate could also be used to trigger eviction from the map if they become expired when .get()'d.

Would something like this be worth adding?

[Philipp Wendler]

Hi,

Am 12.09.2011 21:28, schrieb Charles Fry:
> Right now we use ReentrantLock. In the future we are considering
> implementing concurrencyLevel(0), but we don't have any hard numbers about
> the potential gain. If all you want is a Map then I'd stick with HashMap. If
> what you want is a cache then I'd consider CacheBuilder.

Ok, so I think I'll use CacheBuilder after Guava 10 is released and see
if I can produce some numbers comparing it to a Map in my application.

Greetings, Philipp

[Kublai Khan]

Hi Charles,

Very excited about the CacheBuilder - I'm glad Guava has identified
and supported MapMaker's caching demographic.

I'm wondering if it will be possible for entries to be kept strongly
reachable up until expiry, but then just be weak rather than evicted?
In this way instances uniquely identified by their key will not risk
being introduced into an environment twice if a long-running process
keeps an expired value from being garbage collected and it is
meanwhile reloaded elsewhere. See my SO question for more explanation
on the use-case and dilemma: http://stackoverflow.com/questions/6778743/my-ideal-cache-using-guava

Thanks,
Kublai

On Sep 12, 1:16 pm, Charles Fry <f...@google.com> wrote:
> tl;dr: common.cache.CacheBuilder is better at making caches than MapMaker
>
> There is a good chance that you’ve already heard of

> MapMaker<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>.

> You may also be aware that it has been accumulating more and more caching
> functionality, to the point where it has outgrown its original role as a
> “collection”, ultimately meriting its own package. With release 10.0 we are

> introducing CacheBuilder<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>and
> the corresponding
> Cache<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>interface

> as a replacement for the caching functionality in MapMaker. The
> API remains virtually the same:
>
> Before:
>
> ConcurrentMap<Key, Graph> graphs = new MapMaker()    .weakKeys()
>    .maximumSize(10000)
>    .makeComputingMap(
>        new Function<Key, Graph>() {
>          public Graph apply(Key key) {
>
>             try {              return createExpensiveGraph(key);
>
>            } catch (SomeCheckedException e) {
>
>              // now what
>
>            }
>          }
>        });
>
> After:
>
> Cache<Key, Graph> graphs = CacheBuilder.newBuilder()    .weakKeys()
>    .maximumSize(10000)
>    .build(
>        new CacheLoader<Key, Graph>() {
>          public Graph load(Key key) throws SomeCheckedException {
>            return createExpensiveGraph(key);
>          }
>        });
>
> The cache package defines a new

> CacheLoader<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>interface

> which makes it easier to deal with checked exceptions which occur
> when loading new values into the cache, as shown above. The new Cache
> interface will allow us to expose caching logic on the returned instance.
> For example, we now expose detailed

> statistics<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>about

> cache performance.
>
> CacheBuilder already has a long list of upcoming features, including:
>

>    - Weighted entries
>    - Bulk operations
>    - Support for refreshing stale entries

[Charles Fry]

The complexity with that is that eviction would no longer be a constant time operation, which is currently the case with fixed eviction times. As such I don't see native support for this soon; though of course you could bake it into a value wrapper, and invalidate stale entries.

Charles

[Ramesh Mandaleeka]

Is there a Cache implementation off-heap, contents backed up by a file implementation in guava?

-Ramesh

--

[Charles Fry]

Is there a Cache implementation off-heap, contents backed up by a file implementation in guava?

No, but feel free to file a feature request. 

[Charles Fry]

Very excited about the CacheBuilder - I'm glad Guava has identified
and supported MapMaker's caching demographic.

I'm wondering if it will be possible for entries to be kept strongly
reachable up until expiry, but then just be weak rather than evicted?
In this way instances uniquely identified by their key will not risk
being introduced into an environment twice if a long-running process
keeps an expired value from being garbage collected and it is
meanwhile reloaded elsewhere. See my SO question for more explanation
on the use-case and dilemma: http://stackoverflow.com/questions/6778743/my-ideal-cache-using-guava

You can already combine weakKeys, weakValues, and softValues with other expiration policies. Whether or not that's what you actually want is best discussed on stackoverflow. :-)

Charles

[Ramesh Mandaleeka]

Hello, Fry,

I filed a feature request at http://code.google.com/p/guava-libraries/issues/detail?id=714

Regards,
Ramesh

[Adrian Cole]

Thanks for putting this together. We are already using this in
jclouds for caching expensive calls to EC2 for example. Seems like
the right path, and I especially like the exception design, as it will
likely remove several or more lines from stack traces from MapMaker.

A couple notes:

1. Currently, for we have images keyed on string. We'd like to
preseed the cache so that asMap().keySet() returns all available
images. Important is that we can get all images in a single command.
Seems a Map<K,V> seed() method in the CacheLoader would be nice for
things like adding in a bulk of stuff.

2. Would be handy to be able to specify a CacheLoader where load
returns Future<V>, similarly seed could return Map<K, Future<V>>. All
of our apis return futures, so this would be uber efficient and handy.

Cheers,
-Adrian
jclouds

On Sep 12, 10:16 am, Charles Fry <f...@google.com> wrote:
> tl;dr: common.cache.CacheBuilder is better at making caches than MapMaker
>
> There is a good chance that you’ve already heard of

> MapMaker<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>.

> You may also be aware that it has been accumulating more and more caching
> functionality, to the point where it has outgrown its original role as a
> “collection”, ultimately meriting its own package. With release 10.0 we are

> introducing CacheBuilder<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>and
> the corresponding

> Cache<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>interface

> as a replacement for the caching functionality in MapMaker. The
> API remains virtually the same:
>
> Before:
>
> ConcurrentMap<Key, Graph> graphs = new MapMaker()    .weakKeys()
>    .maximumSize(10000)
>    .makeComputingMap(
>        new Function<Key, Graph>() {
>          public Graph apply(Key key) {
>
>             try {              return createExpensiveGraph(key);
>
>            } catch (SomeCheckedException e) {
>
>              // now what
>
>            }
>          }
>        });
>
> After:
>
> Cache<Key, Graph> graphs = CacheBuilder.newBuilder()    .weakKeys()
>    .maximumSize(10000)
>    .build(
>        new CacheLoader<Key, Graph>() {
>          public Graph load(Key key) throws SomeCheckedException {
>            return createExpensiveGraph(key);
>          }
>        });
>
> The cache package defines a new

> CacheLoader<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>interface

> which makes it easier to deal with checked exceptions which occur
> when loading new values into the cache, as shown above. The new Cache
> interface will allow us to expose caching logic on the returned instance.
> For example, we now expose detailed

> statistics<http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/com...>about

> cache performance.
>
> CacheBuilder already has a long list of upcoming features, including:
>

>    - Weighted entries
>    - Bulk operations

>    - Support for refreshing stale entries

[Charles Fry]

Thanks for putting this together.  We are already using this in
jclouds for caching expensive calls to EC2 for example.  Seems like
the right path, and I especially like the exception design, as it will
likely remove several or more lines from stack traces from MapMaker.

Glad to hear it! 

 

A couple notes:

1. Currently, for we have images keyed on string.  We'd like to
preseed the cache so that asMap().keySet() returns all available
images.  Important is that we can get all images in a single command.
Seems a Map<K,V> seed() method in the CacheLoader would be nice for
things like adding in a bulk of stuff.

We are working on explicit support for bulk operations. Expect that in release 11.0. 

 

2. Would be handy to be able to specify a CacheLoader where load
returns Future<V>, similarly seed could return Map<K, Future<V>>.  All
of our apis return futures, so this would be uber efficient and handy.

You can already do this with Cache<K, Future<V>>. We're still trying to figure out whether there's a more native integration between Cache and Future.

Charles 

[Ted Yu]

We currently have:
    backingMap = new MapMaker().maximumSize(numBlocks - 1)
        .evictionListener(listener).makeMap();
where listener is MapEvictionListener<String, CacheablePair>

Since MapEvictionListener is deprecated, I wonder how we should implement the above along with switching to CacheBuilder.

Thanks

--