Interesting. I assume that you need access patterns that span the entire keyset at random for the most part?
When I was evaluating the storage options years ago, BDB was the best and my usage pattern was such that for activity in a given window, the active keyset was a mostly bounded subset of the full keyset. (Use case was e.g. shopping cart at a flash sale website - while there are millions of customers, the group that shows up any given day at sale start is much smaller and activity tends to be continuous - they show up, hang out, then go away. So the result was things fit very well in the BDB cache and performance was good. What I really liked about BDB also was that it was self-maintaining. We never had to do any ops maintenance on the cluster. We put up three clusters and they only came down when we moved colos .
I guess my question for Marc was why he didn't want to use BDB - he mentioned a love of file-based and BDB is file based...
> For me, it is due to the cliff function that BDB goes
> off of when the data set goes out of RAM -- I've been
> placing a higher value on consistent latency than raw
> The BDB issues hark back to the original Dynamo
> paper where the Amazon folks had a bunch of trouble
> tuning BDB-JE for consistent performance. When I
> measured the performance of the Voldemort BDB
> storage engine a year or 2 ago, I found the same
> I wrote the InnoDB plugin because InnoDB has better
> buffer pool management than BDB and I could configure
> it for more consistent performance. I put a presentation
> about it together here:
> Granted, it sacrifices throughput for stability. I have it on
> my list to write storage engines for the other two products
> I mentioned (LevelDB and Persistit); they are promising
> in theory but admittedly it's based on superficial analysis--
> I don't have any measurements just yet. Based on the
> way the voldemort community has atrophied in recent months,
> I'm also not sure how much time I want to invest in it
> On 11/9/12, Geir Magnusson Jr. <g...@pobox.com> wrote:
>> I don't understand the BDB resistance.... can you explain?
>> On Nov 9, 2012, at 9:15 AM, Sunny Gleason <sunny.glea...@gmail.com> wrote:
>>> For your use case, I might consider leveldb or persistit.
>>> I'd try as hard as possible not to embed voldemort in a
>>> product; but if you really need to, it's not that hard to create
>>> your own storage engine. I'd be happy to give you advice if
>>> you need it; I wrote an innodb storage engine for Voldemort
>>> and it wasn't too horrible.
>>> All the best,
>>> On 11/9/12, Marc Logemann <marc.logem...@googlemail.com> wrote:
>>>> we are checking alternatives for our current redis way of doing this. We
>>>> heavily use key value collections to store agregated statistics data. So
>>>> have heavy writes, not so heavy reads on our redis instance. We are
>>>> happy the way how redis works but since we want to implement / embed the
>>>> NoSQL server in one of our java based products, redis being C based is a
>>>> go. So we evaluated a lot of projects. Voldemort comes nearest in terms
>>>> our requirements for now. I wrote about that
>>>> May i ask some questions to this group because we really need to get
>>>> facts right:
>>>> - We certainly dont want to use BDB. Is there an alternative (file
>>>> store available for Voldemort? We love file based because of freedom wrt
>>>> - Is an embedded installation scenario with just one node ok? Scaling to
>>>> different nodes just as an option...
>>>> - what about range queries for keys?
>>>> Thanks a lot for clearification. If we would settle on Voldemort, our
>>>> team would be happy to contribute back in some form. We just need to make
>>>> sane decission how to go on and since its for two of our products, it
>>>> be a long lasting decission ;-)
>>>> CEO LOGENTIS <http://www.logentis.de> GmbH
>>>> You received this message because you are subscribed to the Google
>>>> "project-voldemort" group.
>>>> To unsubscribe from this group, send email to
>>>> Visit this group at
>>> You received this message because you are subscribed to the Google Groups
>>> "project-voldemort" group.
>>> To unsubscribe from this group, send email to
>>> Visit this group at
>> You received this message because you are subscribed to the Google Groups
>> "project-voldemort" group.
>> To unsubscribe from this group, send email to
>> Visit this group at http://groups.google.com/group/project-voldemort?hl=en.
> You received this message because you are subscribed to the Google Groups "project-voldemort" group.
> To unsubscribe from this group, send email to email@example.com.
> Visit this group at http://groups.google.com/group/project-voldemort?hl=en.