looking for key/value store that can automatically expire entries

[Mike W]

I am looking for a persistent key/value store for my web application (which is written in Java).

My app generates user session data that I would like to store in the key/value store.  Each user session is roughly 1 MB of binary data, and roughly a million user sessions are created every day.  Each user session should expire from the key/value store after 24 hours.

I figure the expiration could happen in one of two ways.  One way to implement the expiration would be if the key/value store had built-in support for expiring objects after a given timeout and automatically deleted any expired objects.  The second way would be if each key/value pair also had an associated timestamp, and the system would run an hourly task that deleted all key/value pairs where the timestamp was over a given timeout.

I don't need the ability to do any queries on the system other than 1) load/store a value for a key, and 2) the above-mentioned deleting of expired key/values.  Throughput requirements: 1,000,000 new 1-MB objects created per day, 100,000 1-MB objects read per day, 100,000 1-MB objects updated (read and written back to the store) per day, and 1,000,000 objects expired/deleted per day.

I don't need the system to provide fault tolerance, crash resistance, replication or auto-scaling.  Sharding would be nice.

I definitely need a system that has good support for Java clients, is open source, has good documentation, and good community support.

What system do you recommendation?

[David Rosenstrauch]

On 01/14/2015 12:21 PM, Mike W wrote:
> I am looking for a persistent key/value store for my web application (which
> is written in Java).
>
> My app generates user session data that I would like to store in the
> key/value store. Each user session is roughly 1 MB of binary data, and
> roughly a million user sessions are created every day. Each user session
> should expire from the key/value store after 24 hours.

Redis?

http://redis.io/commands/ttl

DR

[Mike Wertheim]

My understanding of Redis version 2.4 and higher is that what's stored on disk is just a backup of what's in RAM, and that the disk storage won't grow bigger than the RAM storage.  I need 1 TB of disk storage (no problem), so I think I'd also need 1 TB of RAM storage (problem).

According to http://redis.io/topics/virtual-memory: "In the future of Redis we want to simply provide the best in-memory database (but persistent on disk as usual) ever, without considering at least for now the support for databases bigger than RAM."

--
You received this message because you are subscribed to a topic in the Google Groups "NOSQL" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nosql-discussion/kxUYnLqnUZo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nosql-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to nosql-discussion@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nosql-discussion/54B6A62E.9090204%40darose.net.

For more options, visit https://groups.google.com/d/optout.

[Adam Retter]

You could look at the TTL feature in RocksDB, there is a pull-request
which will likely be merged very soon which exposes this to it's Java
API.

You could always take the pull-request branch if you want to try it
immediately, I know it is relatively stable -
https://github.com/facebook/rocksdb/pull/458

> --
> You received this message because you are subscribed to the Google Groups
> "NOSQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nosql-discussi...@googlegroups.com.
> To post to this group, send email to nosql-di...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nosql-discussion/721a565d-aa1d-45c9-9827-6c6ddfccee6e%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk

[Emin Gün Sirer]

It's easy to do this in HyperDex [1] by attaching timestamps and issuing group_del operations [2] to delete the old entries.

[1] http://hyperdex.org

[2] http://hyperdex.org/doc/latest/JavaAPI/#sec:api:java:predicates