Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Message from discussion KV store w/ persistence
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Geir Magnusson Jr.  
View profile  
 More options Nov 9 2012, 9:49 am
From: "Geir Magnusson Jr." <g...@pobox.com>
Date: Fri, 9 Nov 2012 09:49:12 -0500
Local: Fri, Nov 9 2012 9:49 am
Subject: Re: [project-voldemort] KV store w/ persistence
Interesting.   I assume that you need access patterns that span the entire keyset at random for the most part?

When I was evaluating the storage options years ago, BDB was the best and my usage pattern was such that for activity in a given window, the active keyset was a mostly bounded subset of the full keyset.  (Use case was e.g. shopping cart at a flash sale website - while there are millions of customers, the group that shows up any given day at sale start is much smaller and activity tends to be continuous - they show up, hang out, then go away.  So the result was things fit very well in the BDB cache and performance was good.  What I really liked about BDB also was that it was self-maintaining.  We never had to do any ops maintenance on the cluster.  We put up three clusters and they only came down when we moved colos .

I'm interested in your plugin :)

I guess my question for Marc was why he didn't want to use BDB - he mentioned a love of file-based and BDB is file based...

Thanks

geir

On Nov 9, 2012, at 9:32 AM, Sunny Gleason <sunny.glea...@gmail.com> wrote:

> For me, it is due to the cliff function that BDB goes
> off of when the data set goes out of RAM -- I've been
> placing a higher value on consistent latency than raw
> throughput.

> The BDB issues hark back to the original Dynamo
> paper where the Amazon folks had a bunch of trouble
> tuning BDB-JE for consistent performance. When I
> measured the performance of the Voldemort BDB
> storage engine a year or 2 ago, I found the same
> issues.

> I wrote the InnoDB plugin because InnoDB has better
> buffer pool management than BDB and I could configure
> it for more consistent performance. I put a presentation
> about it together here:

> http://www.slideshare.net/sunnygleason/accelerating-nosql

> Granted, it sacrifices throughput for stability. I have it on
> my list to write storage engines for the other two products
> I mentioned (LevelDB and Persistit); they are promising
> in theory but admittedly it's based on superficial analysis--
> I don't have any measurements just yet. Based on the
> way the voldemort community has atrophied in recent months,
> I'm also not sure how much time I want to invest in it
> anymore...

> -Sunny

> On 11/9/12, Geir Magnusson Jr. <g...@pobox.com> wrote:
>> Sunny,

>> I don't understand the BDB resistance....  can you explain?

>> On Nov 9, 2012, at 9:15 AM, Sunny Gleason <sunny.glea...@gmail.com> wrote:

>>> For your use case, I might consider leveldb or persistit.

>>> https://github.com/dain/leveldb
>>> http://www.akiban.com/akiban-persistit

>>> I'd try as hard as possible not to embed voldemort in a
>>> product; but if you really need to, it's not that hard to create
>>> your own storage engine. I'd be happy to give you advice if
>>> you need it; I wrote an innodb storage engine for Voldemort
>>> and it wasn't too horrible.

>>> All the best,

>>> -Sunny

>>> On 11/9/12, Marc Logemann <marc.logem...@googlemail.com> wrote:
>>>> Hi,

>>>> we are checking alternatives for our current redis way of doing this. We
>>>> heavily use key value collections to store agregated statistics data. So
>>>> we

>>>> have heavy writes, not so heavy reads on our redis instance. We are
>>>> quite
>>>> happy the way how redis works but since we want to implement / embed the
>>>> NoSQL server in one of our java based products, redis being C based is a
>>>> no

>>>> go. So we evaluated a lot of projects. Voldemort comes nearest in terms
>>>> of
>>>> our requirements for now. I wrote about that
>>>> there<http://www.logemann.org/2012/11/nosql-kv-project-search-goes-on.html>
>>>> .

>>>> May i ask some questions to this group because we really need to get
>>>> some
>>>> facts right:

>>>> - We certainly dont want to use BDB. Is there an alternative (file
>>>> based)
>>>> store available for Voldemort? We love file based because of freedom wrt
>>>> installations.
>>>> - Is an embedded installation scenario with just one node ok? Scaling to
>>>> different nodes just as an option...
>>>> - what about range queries for keys?

>>>> Thanks a lot for clearification. If we would settle on Voldemort, our
>>>> dev
>>>> team would be happy to contribute back in some form. We just need to make
>>>> a

>>>> sane decission how to go on and since its for two of our products, it
>>>> would

>>>> be a long lasting decission ;-)

>>>> Marc
>>>> CEO LOGENTIS <http://www.logentis.de> GmbH

>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups
>>>> "project-voldemort" group.
>>>> To unsubscribe from this group, send email to
>>>> project-voldemort+unsubscribe@googlegroups.com.
>>>> Visit this group at
>>>> http://groups.google.com/group/project-voldemort?hl=en.

>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "project-voldemort" group.
>>> To unsubscribe from this group, send email to
>>> project-voldemort+unsubscribe@googlegroups.com.
>>> Visit this group at
>>> http://groups.google.com/group/project-voldemort?hl=en.

>> --
>> You received this message because you are subscribed to the Google Groups
>> "project-voldemort" group.
>> To unsubscribe from this group, send email to
>> project-voldemort+unsubscribe@googlegroups.com.
>> Visit this group at http://groups.google.com/group/project-voldemort?hl=en.

> --
> You received this message because you are subscribed to the Google Groups "project-voldemort" group.
> To unsubscribe from this group, send email to project-voldemort+unsubscribe@googlegroups.com.
> Visit this group at http://groups.google.com/group/project-voldemort?hl=en.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.