Voldemort 0.96 is released

[Lei Gao]

Hello everyone,

Voldemort 0.96 is just released. You can find out more details on the following blog post: http://data.linkedin.com/blog/2012/09/project-voldemort-096-release

Thanks,

Lei Gao

[Carlos Tasada]

Hi Lei,

Answering to your questions:

* Besides the features/enhancements mentioned in "Next play" section, what is the one feature/enhancement you like to see in Voldemort?

Probably the main enhancement that I'm looking for is a better storage support. BDB is great, but I have the feeling that other alternatives as Krati or LevelDB are giving better performance.

Other nice to have would be: Maven support and better documentation 

* How often would you like to see the development updates from the Voldemort team - for features/enhancements initiated by the team at LinkedIn? How to improve the communication between the Voldemort team at LinkedIn and the open source community?

I really thing that depends of what new code is ready. I would like to see bug-fix/minor releases as frequently as needed. Big releases can take longer, lets say 6 to 9 months.

To improve the communication between the Voldemort team and the OS community I think:

a) The V LinkedIn team should keep the transparency regarding the were the project is heading to

b) The V LinkedIn team should have some frequently releases

c) The OS community should make its voice heard, with proposals about how to improve the project

d) The OS community should involve itself with more coding in the project, helping to improve the documentation and writing about it in blogs, twitter, ....

My 2 cents.

--
 
 

[Vinoth Chandar]

Hi Carlos,

I can answer the questions about BDB and storage in general, since I have been slogging on it for close to 4-5 months now. We did evaluate LevelDB and it did not go well. There are no proper JNI bindings available.

We are also white boarding ideas about rolling our own storage engine optimized for flash. But, the problem with these new storage engines is that they are not really battle tested.  So, this is a touchy subject. We are actively evaluating other options as well, including good old mysql.

I will do a personal blog on my BDB experiences and post in the groups.

Thanks
Vinoth

[Mickey Hsieh]

Hi Vinoth,

We had used Voldemort for 3 years and tried different storage engine, mysql, postgre and bdb. None of them meet our requirement (access 650M records of value 1 KB size, random read/write, on client side average 5 ms).

So, We wrote storage plug-in  for Voldemort. It had deployed in production for one and half year and  handles 180M request per day.

Please take a look http://code.google.com/p/cachestore.. It was written for 0.81 and I am in process of porting to 0.95

Thanks,

Mickey

  

--
 
 

[Carlos Tasada]

Hi Vinoth,

Looking forward for your comments on BDB. In the next weeks I'm going to go through testing different engines in Voldemort to find which one adapts better to our requirements. 

Basically we need some kind of storage that can manage multiple writes (avg 100 writes/second and 1000 read/second per node). Right now BDB seems to be enough, but the usage growth that I'm expecting in the next year may be too much.

Regards,

Carlos.

--
 
 

[Carlos Tasada]

Hi Mickey,

Sounds really interesting. I'll take a look!

Maybe would be interesting to have some wiki entry pointing to the different available storage engines and how to configure them.

Thanks

--
 
 

[Mickey Hsieh]

Hi Carlos,

Here is configuration.link

http://code.google.com/p/cachestore/wiki/Configuration 

The server.properties configuration  is for each node. Cluster.xml and stores.xml stay the same.

Let me know if you have any questions.

Mickey

--
 
 

[Otis Gospodnetic]

Lei - the README on Github still points to Google code at the end.

Otis

[Lei Gao]

Hey Mickey,

Do you have any design doc on the storage engine you guys built, if it's okey for you to share it w/ the open source community?

Lei

[Lei Gao]

Otis,

Thanks for letting us know. We will change it once we finish the ticket migration.

Lei

[Lei Gao]

A quick note: there was an issue with downloading voldemort-0.96.zip. The problem is fixed now.

Lei

[Mickey Hsieh]

Hi Lei,

Yes, this is link for power point presentation

http://code.google.com/p/cachestore/wiki/CacheStorePresentation

If you have any questions, More than happy to connect by Google+ or Skype.

Thanks,

Mickey

--
 
 

[Maarten Koopmans]

I just saw this, awesome. Kudos!

--Maarten

On Sat, Sep 8, 2012 at 2:27 AM, Lei Gao <gao...@gmail.com> wrote:

> --
>
>

[Maarten Koopmans]

And to answer the questions:

1) I'd really like to see VaaS, it is an idea that I had in mind when
thinking about an alternative. Might be slower, but would be very
useful. Besides, it would mean just more smaller nodes ;-) I'd be
happy to contribute a Lift (see Liftweb.net) based REST interface if
there is interest.

2) Update frequency: 3-6 months would do for a release.

--Maarten

PS: a REST interface might abstract node implementations away,
specifically it would allow other implementations (I was thinking....
Go....) to succeed once the protocol are stable and documented. Idea?

[Carlos Tasada]

+1 for the REST interface

--

[Lei Gao]

We are considering a Thin Client option for voldemort. However, there are issues to address for a Thin Client implementation. How thin should it be? Shall we have connection pooling or connection multiplexing support? What about serialization?  The only thing for certain, is that routing will be on the server side.

Lei

On Friday, September 7, 2012 5:27:28 PM UTC-7, Lei Gao wrote:

[Tatu Saloranta]

On Tue, Sep 11, 2012 at 11:29 AM, Lei Gao <gao...@gmail.com> wrote:
> We are considering a Thin Client option for voldemort. However, there are
> issues to address for a Thin Client implementation. How thin should it be?
> Shall we have connection pooling or connection multiplexing support? What

If HTTP is used as the underlying protocol, connection reuse and
pooling should work quite nicely (with HTTP 1.1) without extra work?

-+ Tatu +-