* Besides the features/enhancements mentioned in "Next play" section, what
is the one feature/enhancement you like to see in Voldemort?
Probably the main enhancement that I'm looking for is a better storage
support. BDB is great, but I have the feeling that other alternatives as
Krati or LevelDB are giving better performance.
Other nice to have would be: Maven support and better documentation
* How often would you like to see the development updates from the
Voldemort team - for features/enhancements initiated by the team at
LinkedIn? How to improve the communication between the Voldemort team at
LinkedIn and the open source community?
I really thing that depends of what new code is ready. I would like to see
bug-fix/minor releases as frequently as needed. Big releases can take
longer, lets say 6 to 9 months.
To improve the communication between the Voldemort team and the OS
community I think:
a) The V LinkedIn team should keep the transparency regarding the were the
project is heading to
b) The V LinkedIn team should have some frequently releases
c) The OS community should make its voice heard, with proposals about how
to improve the project
d) The OS community should involve itself with more coding in the project,
helping to improve the documentation and writing about it in blogs,
twitter, ....
I can answer the questions about BDB and storage in general, since I have been slogging on it for close to 4-5 months now. We did evaluate LevelDB and it did not go well. There are no proper JNI bindings available.
We are also white boarding ideas about rolling our own storage engine optimized for flash. But, the problem with these new storage engines is that they are not really battle tested. So, this is a touchy subject. We are actively evaluating other options as well, including good old mysql.
I will do a personal blog on my BDB experiences and post in the groups.
On Friday, September 7, 2012 11:43:58 PM UTC-7, ctasada wrote:
> Hi Lei,
> Answering to your questions:
> * Besides the features/enhancements mentioned in "Next play" section, what > is the one feature/enhancement you like to see in Voldemort?
> Probably the main enhancement that I'm looking for is a better storage > support. BDB is great, but I have the feeling that other alternatives as > Krati or LevelDB are giving better performance. > Other nice to have would be: Maven support and better documentation
> * How often would you like to see the development updates from the > Voldemort team - for features/enhancements initiated by the team at > LinkedIn? How to improve the communication between the Voldemort team at > LinkedIn and the open source community?
> I really thing that depends of what new code is ready. I would like to see > bug-fix/minor releases as frequently as needed. Big releases can take > longer, lets say 6 to 9 months.
> To improve the communication between the Voldemort team and the OS > community I think: > a) The V LinkedIn team should keep the transparency regarding the were the > project is heading to > b) The V LinkedIn team should have some frequently releases > c) The OS community should make its voice heard, with proposals about how > to improve the project > d) The OS community should involve itself with more coding in the project, > helping to improve the documentation and writing about it in blogs, > twitter, ....
> My 2 cents.
> On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com <javascript:>>wrote:
We had used Voldemort for 3 years and tried different storage engine,
mysql, postgre and bdb. None of them meet our requirement (access 650M
records of value 1 KB size, random read/write, on client side average 5 ms).
So, We wrote storage plug-in for Voldemort. It had deployed in production
for one and half year and handles 180M request per day.
mail.vinoth.chan...@gmail.com> wrote:
> Hi Carlos,
> I can answer the questions about BDB and storage in general, since I have
> been slogging on it for close to 4-5 months now. We did evaluate LevelDB
> and it did not go well. There are no proper JNI bindings available.
> We are also white boarding ideas about rolling our own storage engine
> optimized for flash. But, the problem with these new storage engines is
> that they are not really battle tested. So, this is a touchy subject. We
> are actively evaluating other options as well, including good old mysql.
> I will do a personal blog on my BDB experiences and post in the groups.
> Thanks
> Vinoth
> On Friday, September 7, 2012 11:43:58 PM UTC-7, ctasada wrote:
>> Hi Lei,
>> Answering to your questions:
>> * Besides the features/enhancements mentioned in "Next play" section,
>> what is the one feature/enhancement you like to see in Voldemort?
>> Probably the main enhancement that I'm looking for is a better storage
>> support. BDB is great, but I have the feeling that other alternatives as
>> Krati or LevelDB are giving better performance.
>> Other nice to have would be: Maven support and better documentation
>> * How often would you like to see the development updates from the
>> Voldemort team - for features/enhancements initiated by the team at
>> LinkedIn? How to improve the communication between the Voldemort team at
>> LinkedIn and the open source community?
>> I really thing that depends of what new code is ready. I would like to
>> see bug-fix/minor releases as frequently as needed. Big releases can take
>> longer, lets say 6 to 9 months.
>> To improve the communication between the Voldemort team and the OS
>> community I think:
>> a) The V LinkedIn team should keep the transparency regarding the were
>> the project is heading to
>> b) The V LinkedIn team should have some frequently releases
>> c) The OS community should make its voice heard, with proposals about how
>> to improve the project
>> d) The OS community should involve itself with more coding in the
>> project, helping to improve the documentation and writing about it in
>> blogs, twitter, ....
>> My 2 cents.
>> On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:
Looking forward for your comments on BDB. In the next weeks I'm going to go
through testing different engines in Voldemort to find which one adapts
better to our requirements.
Basically we need some kind of storage that can manage multiple writes (avg
100 writes/second and 1000 read/second per node). Right now BDB seems to be
enough, but the usage growth that I'm expecting in the next year may be too
much.
mail.vinoth.chan...@gmail.com> wrote:
> Hi Carlos,
> I can answer the questions about BDB and storage in general, since I have
> been slogging on it for close to 4-5 months now. We did evaluate LevelDB
> and it did not go well. There are no proper JNI bindings available.
> We are also white boarding ideas about rolling our own storage engine
> optimized for flash. But, the problem with these new storage engines is
> that they are not really battle tested. So, this is a touchy subject. We
> are actively evaluating other options as well, including good old mysql.
> I will do a personal blog on my BDB experiences and post in the groups.
> Thanks
> Vinoth
> On Friday, September 7, 2012 11:43:58 PM UTC-7, ctasada wrote:
>> Hi Lei,
>> Answering to your questions:
>> * Besides the features/enhancements mentioned in "Next play" section,
>> what is the one feature/enhancement you like to see in Voldemort?
>> Probably the main enhancement that I'm looking for is a better storage
>> support. BDB is great, but I have the feeling that other alternatives as
>> Krati or LevelDB are giving better performance.
>> Other nice to have would be: Maven support and better documentation
>> * How often would you like to see the development updates from the
>> Voldemort team - for features/enhancements initiated by the team at
>> LinkedIn? How to improve the communication between the Voldemort team at
>> LinkedIn and the open source community?
>> I really thing that depends of what new code is ready. I would like to
>> see bug-fix/minor releases as frequently as needed. Big releases can take
>> longer, lets say 6 to 9 months.
>> To improve the communication between the Voldemort team and the OS
>> community I think:
>> a) The V LinkedIn team should keep the transparency regarding the were
>> the project is heading to
>> b) The V LinkedIn team should have some frequently releases
>> c) The OS community should make its voice heard, with proposals about how
>> to improve the project
>> d) The OS community should involve itself with more coding in the
>> project, helping to improve the documentation and writing about it in
>> blogs, twitter, ....
>> My 2 cents.
>> On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:
On Sat, Sep 8, 2012 at 8:05 PM, Mickey Hsieh <micke...@gmail.com> wrote:
> Hi Vinoth,
> We had used Voldemort for 3 years and tried different storage engine,
> mysql, postgre and bdb. None of them meet our requirement (access 650M
> records of value 1 KB size, random read/write, on client side average 5 ms).
> So, We wrote storage plug-in for Voldemort. It had deployed in production
> for one and half year and handles 180M request per day.
> On Sat, Sep 8, 2012 at 10:33 AM, Vinoth Chandar <
> mail.vinoth.chan...@gmail.com> wrote:
>> Hi Carlos,
>> I can answer the questions about BDB and storage in general, since I have
>> been slogging on it for close to 4-5 months now. We did evaluate LevelDB
>> and it did not go well. There are no proper JNI bindings available.
>> We are also white boarding ideas about rolling our own storage engine
>> optimized for flash. But, the problem with these new storage engines is
>> that they are not really battle tested. So, this is a touchy subject. We
>> are actively evaluating other options as well, including good old mysql.
>> I will do a personal blog on my BDB experiences and post in the groups.
>> Thanks
>> Vinoth
>> On Friday, September 7, 2012 11:43:58 PM UTC-7, ctasada wrote:
>>> Hi Lei,
>>> Answering to your questions:
>>> * Besides the features/enhancements mentioned in "Next play" section,
>>> what is the one feature/enhancement you like to see in Voldemort?
>>> Probably the main enhancement that I'm looking for is a better storage
>>> support. BDB is great, but I have the feeling that other alternatives as
>>> Krati or LevelDB are giving better performance.
>>> Other nice to have would be: Maven support and better documentation
>>> * How often would you like to see the development updates from the
>>> Voldemort team - for features/enhancements initiated by the team at
>>> LinkedIn? How to improve the communication between the Voldemort team at
>>> LinkedIn and the open source community?
>>> I really thing that depends of what new code is ready. I would like to
>>> see bug-fix/minor releases as frequently as needed. Big releases can take
>>> longer, lets say 6 to 9 months.
>>> To improve the communication between the Voldemort team and the OS
>>> community I think:
>>> a) The V LinkedIn team should keep the transparency regarding the were
>>> the project is heading to
>>> b) The V LinkedIn team should have some frequently releases
>>> c) The OS community should make its voice heard, with proposals about
>>> how to improve the project
>>> d) The OS community should involve itself with more coding in the
>>> project, helping to improve the documentation and writing about it in
>>> blogs, twitter, ....
>>> My 2 cents.
>>> On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:
On Sat, Sep 8, 2012 at 12:50 PM, Carlos Tasada <ctas...@gmail.com> wrote:
> Hi Mickey,
> Sounds really interesting. I'll take a look!
> Maybe would be interesting to have some wiki entry pointing to the
> different available storage engines and how to configure them.
> Thanks
> On Sat, Sep 8, 2012 at 8:05 PM, Mickey Hsieh <micke...@gmail.com> wrote:
>> Hi Vinoth,
>> We had used Voldemort for 3 years and tried different storage engine,
>> mysql, postgre and bdb. None of them meet our requirement (access 650M
>> records of value 1 KB size, random read/write, on client side average 5 ms).
>> So, We wrote storage plug-in for Voldemort. It had deployed in
>> production for one and half year and handles 180M request per day.
>> On Sat, Sep 8, 2012 at 10:33 AM, Vinoth Chandar <
>> mail.vinoth.chan...@gmail.com> wrote:
>>> Hi Carlos,
>>> I can answer the questions about BDB and storage in general, since I
>>> have been slogging on it for close to 4-5 months now. We did evaluate
>>> LevelDB and it did not go well. There are no proper JNI bindings available.
>>> We are also white boarding ideas about rolling our own storage engine
>>> optimized for flash. But, the problem with these new storage engines is
>>> that they are not really battle tested. So, this is a touchy subject. We
>>> are actively evaluating other options as well, including good old mysql.
>>> I will do a personal blog on my BDB experiences and post in the groups.
>>> Thanks
>>> Vinoth
>>> On Friday, September 7, 2012 11:43:58 PM UTC-7, ctasada wrote:
>>>> Hi Lei,
>>>> Answering to your questions:
>>>> * Besides the features/enhancements mentioned in "Next play" section,
>>>> what is the one feature/enhancement you like to see in Voldemort?
>>>> Probably the main enhancement that I'm looking for is a better storage
>>>> support. BDB is great, but I have the feeling that other alternatives as
>>>> Krati or LevelDB are giving better performance.
>>>> Other nice to have would be: Maven support and better documentation
>>>> * How often would you like to see the development updates from the
>>>> Voldemort team - for features/enhancements initiated by the team at
>>>> LinkedIn? How to improve the communication between the Voldemort team at
>>>> LinkedIn and the open source community?
>>>> I really thing that depends of what new code is ready. I would like to
>>>> see bug-fix/minor releases as frequently as needed. Big releases can take
>>>> longer, lets say 6 to 9 months.
>>>> To improve the communication between the Voldemort team and the OS
>>>> community I think:
>>>> a) The V LinkedIn team should keep the transparency regarding the were
>>>> the project is heading to
>>>> b) The V LinkedIn team should have some frequently releases
>>>> c) The OS community should make its voice heard, with proposals about
>>>> how to improve the project
>>>> d) The OS community should involve itself with more coding in the
>>>> project, helping to improve the documentation and writing about it in
>>>> blogs, twitter, ....
>>>> My 2 cents.
>>>> On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:
On Saturday, September 8, 2012 11:05:53 AM UTC-7, mickey wrote:
> Hi Vinoth,
> We had used Voldemort for 3 years and tried different storage engine, > mysql, postgre and bdb. None of them meet our requirement (access 650M > records of value 1 KB size, random read/write, on client side average 5 ms). > So, We wrote storage plug-in for Voldemort. It had deployed in production > for one and half year and handles 180M request per day.
> On Sat, Sep 8, 2012 at 10:33 AM, Vinoth Chandar <mail.vino...@gmail.com<javascript:> > > wrote:
>> Hi Carlos,
>> I can answer the questions about BDB and storage in general, since I have >> been slogging on it for close to 4-5 months now. We did evaluate LevelDB >> and it did not go well. There are no proper JNI bindings available.
>> We are also white boarding ideas about rolling our own storage engine >> optimized for flash. But, the problem with these new storage engines is >> that they are not really battle tested. So, this is a touchy subject. We >> are actively evaluating other options as well, including good old mysql.
>> I will do a personal blog on my BDB experiences and post in the groups.
>> Thanks >> Vinoth
>> On Friday, September 7, 2012 11:43:58 PM UTC-7, ctasada wrote:
>>> Hi Lei,
>>> Answering to your questions:
>>> * Besides the features/enhancements mentioned in "Next play" section, >>> what is the one feature/enhancement you like to see in Voldemort?
>>> Probably the main enhancement that I'm looking for is a better storage >>> support. BDB is great, but I have the feeling that other alternatives as >>> Krati or LevelDB are giving better performance. >>> Other nice to have would be: Maven support and better documentation
>>> * How often would you like to see the development updates from the >>> Voldemort team - for features/enhancements initiated by the team at >>> LinkedIn? How to improve the communication between the Voldemort team at >>> LinkedIn and the open source community?
>>> I really thing that depends of what new code is ready. I would like to >>> see bug-fix/minor releases as frequently as needed. Big releases can take >>> longer, lets say 6 to 9 months.
>>> To improve the communication between the Voldemort team and the OS >>> community I think: >>> a) The V LinkedIn team should keep the transparency regarding the were >>> the project is heading to >>> b) The V LinkedIn team should have some frequently releases >>> c) The OS community should make its voice heard, with proposals about >>> how to improve the project >>> d) The OS community should involve itself with more coding in the >>> project, helping to improve the documentation and writing about it in >>> blogs, twitter, ....
>>> My 2 cents.
>>> On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:
On Mon, Sep 10, 2012 at 12:07 PM, Lei Gao <gao...@gmail.com> wrote:
> Hey Mickey,
> Do you have any design doc on the storage engine you guys built, if it's
> okey for you to share it w/ the open source community?
> Lei
> On Saturday, September 8, 2012 11:05:53 AM UTC-7, mickey wrote:
>> Hi Vinoth,
>> We had used Voldemort for 3 years and tried different storage engine,
>> mysql, postgre and bdb. None of them meet our requirement (access 650M
>> records of value 1 KB size, random read/write, on client side average 5 ms).
>> So, We wrote storage plug-in for Voldemort. It had deployed in
>> production for one and half year and handles 180M request per day.
>> On Sat, Sep 8, 2012 at 10:33 AM, Vinoth Chandar <mail.vino...@gmail.com**
>> > wrote:
>>> Hi Carlos,
>>> I can answer the questions about BDB and storage in general, since I
>>> have been slogging on it for close to 4-5 months now. We did evaluate
>>> LevelDB and it did not go well. There are no proper JNI bindings available.
>>> We are also white boarding ideas about rolling our own storage engine
>>> optimized for flash. But, the problem with these new storage engines is
>>> that they are not really battle tested. So, this is a touchy subject. We
>>> are actively evaluating other options as well, including good old mysql.
>>> I will do a personal blog on my BDB experiences and post in the groups.
>>> Thanks
>>> Vinoth
>>> On Friday, September 7, 2012 11:43:58 PM UTC-7, ctasada wrote:
>>>> Hi Lei,
>>>> Answering to your questions:
>>>> * Besides the features/enhancements mentioned in "Next play" section,
>>>> what is the one feature/enhancement you like to see in Voldemort?
>>>> Probably the main enhancement that I'm looking for is a better storage
>>>> support. BDB is great, but I have the feeling that other alternatives as
>>>> Krati or LevelDB are giving better performance.
>>>> Other nice to have would be: Maven support and better documentation
>>>> * How often would you like to see the development updates from the
>>>> Voldemort team - for features/enhancements initiated by the team at
>>>> LinkedIn? How to improve the communication between the Voldemort team at
>>>> LinkedIn and the open source community?
>>>> I really thing that depends of what new code is ready. I would like to
>>>> see bug-fix/minor releases as frequently as needed. Big releases can take
>>>> longer, lets say 6 to 9 months.
>>>> To improve the communication between the Voldemort team and the OS
>>>> community I think:
>>>> a) The V LinkedIn team should keep the transparency regarding the were
>>>> the project is heading to
>>>> b) The V LinkedIn team should have some frequently releases
>>>> c) The OS community should make its voice heard, with proposals about
>>>> how to improve the project
>>>> d) The OS community should involve itself with more coding in the
>>>> project, helping to improve the documentation and writing about it in
>>>> blogs, twitter, ....
>>>> My 2 cents.
>>>> On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:
1) I'd really like to see VaaS, it is an idea that I had in mind when
thinking about an alternative. Might be slower, but would be very
useful. Besides, it would mean just more smaller nodes ;-) I'd be
happy to contribute a Lift (see Liftweb.net) based REST interface if
there is interest.
2) Update frequency: 3-6 months would do for a release.
--Maarten
PS: a REST interface might abstract node implementations away,
specifically it would allow other implementations (I was thinking....
Go....) to succeed once the protocol are stable and documented. Idea?
maarten.koopm...@gmail.com> wrote:
> And to answer the questions:
> 1) I'd really like to see VaaS, it is an idea that I had in mind when
> thinking about an alternative. Might be slower, but would be very
> useful. Besides, it would mean just more smaller nodes ;-) I'd be
> happy to contribute a Lift (see Liftweb.net) based REST interface if
> there is interest.
> 2) Update frequency: 3-6 months would do for a release.
> --Maarten
> PS: a REST interface might abstract node implementations away,
> specifically it would allow other implementations (I was thinking....
> Go....) to succeed once the protocol are stable and documented. Idea?
> On Tue, Sep 11, 2012 at 1:22 PM, Maarten Koopmans
> <maarten.koopm...@gmail.com> wrote:
> > I just saw this, awesome. Kudos!
> > --Maarten
> > On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:
> >> Hello everyone,
We are considering a Thin Client option for voldemort. However, there are issues to address for a Thin Client implementation. How thin should it be? Shall we have connection pooling or connection multiplexing support? What about serialization? The only thing for certain, is that routing will be on the server side.
On Tue, Sep 11, 2012 at 11:29 AM, Lei Gao <gao...@gmail.com> wrote:
> We are considering a Thin Client option for voldemort. However, there are
> issues to address for a Thin Client implementation. How thin should it be?
> Shall we have connection pooling or connection multiplexing support? What
If HTTP is used as the underlying protocol, connection reuse and
pooling should work quite nicely (with HTTP 1.1) without extra work?
