Segmented Cache strategy

[Nelson Vides]

I’ve created a new Erlang cache library, from the lessons and needs we had in MongooseIM, called segmented_cache, code available at https://github.com/esl/segmented_cache and hex package here https://hex.pm/packages/segmented_cache, and I’m looking for opinions, thoughts, kudos, or maybe even contributions :)

1. How does it work: it uses a coarse-grained strategy, that is, it keeps a set of ETS tables that are periodically rotated, and on rotation, the last table is cleared.

2. What is different from all other cache libraries I saw:

* Time-to-live: entries are inserted in buckets (ets tables) that are entirely dropped when their ttl is triggered, making ttls very coarsed-grained, and book-keeping extremely simple and performant.

* Extensible values: we can configure a callback to merge old and new values upon key insert conflict, like, if values are maps, we can use maps:merge/2.

* Concurrency: this implementation is specially tailored for massively concurrent cache lookups and inserts: many caches in Erlang read from an ets table with read_concurrency but serialise inserts through a gen_server, which, in high insert bursts, becomes a bottleneck, not considering duplicates inserts that infinitely many workers might request. Other caches serialise read and writes through a gen_server that keeps the data in a map in its state: this is not only a bottleneck but a source of garbage, as the map is copied on every mutation. Here instead, the cache uses a –bunch of– ets table with read_concurrency, write_concurrency, and decentralized_counters, to maximise concurrent throughput of all read and writes. It also stores references to the buckets, the index, and the merge strategy, using persistent_term, to maximise performance.

* Instrumentation: lookups raise telemetry events with metadata about cache hits or misses, and the duration of the lookup operation, aiding at following the RED method.

Hope somebody finds it useful, and I get some feedback on the idea :)

Nelson.

Our upcoming conferences:

ElixirConf EU: 8-10 September 2021
Code Beam SF: 4-5 November 2021

Erlang Solutions cares about your data and privacy; please find all details about the basis for communicating with you and the way we process your data in our Privacy Policy. You can update your email preferences or opt-out from receiving Marketing emails here.