0.95?

[Maarten Koopmans]

Hi,

I've been using 0.90.1 with some patches by fellow users. I seem to
recall that 0.95 was imminent for some time, or is it? Or is V slowly
dying?

Thanks,

Maarten

[Sunny Gleason]

On 8/16/12, Maarten Koopmans <maarten....@gmail.com> wrote:
> Or is V slowly dying?

Feels like it...

-Sunny

[Geir Magnusson Jr.]

why do you say that?

> --
> You received this message because you are subscribed to the Google Groups "project-voldemort" group.
> To post to this group, send email to project-...@googlegroups.com.
> To unsubscribe from this group, send email to project-voldem...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/project-voldemort?hl=en.
>

[Sunny Gleason]

This list has trailed way off since years past -- I'm not
sure who the project lead is now since Alex left LI, maybe
I'm just out of touch and the conversation/activity is
happening elsewhere.

I love Voldemort, but Riak seems to be emerging as the
go-to Dynamo clone now based on their levelDB work
and attention to features like secondary indexes and
cross-region replication.

If Voldemort really does have the community support
and strong project leadership, I'm all for it -- I tend
to slightly prefer projects that run on the JVM anyway.

Still holding on to hope,

-Sunny

[Carlos Tasada]

I'm having the same thoughts since some months ago (when I got in charge of a project where Voldemort is used). 

The list is quite dead, the issues in the code.google.com page are not updated and there is a lack of roadmap or transparency regarding where the project is heading. The only place where I see some activity is in the GitHub page.

As Sunny, I'm starting to look somewhere else, but I would rather prefer to keep using Voldemort, where we've already invested quite a lot of time resources.

Really looking forward for some comments of the Voldemort guys :)

[Brendan Harris]

Maarten,

There is actually a lot of activity and productivity with regard to the evolution and improvement of voldemort. We have not made an official community release in a while, but I think we can expect one to come soon. However, the 0.95 code is there. We're actually now committing to 0.96.

Sunny,

The project lead is Lei Gao.

Carlos,

You're correct about the google group going dry (and the #voldemort IRC channel) and the google code issues being unmaintained. We've been spending a lot of time focusing on the project internally in LinkedIn, not on the community side. But we have recently started to devote more focus to the community. As of a couple months ago we started a weekly "opensource" meeting that involves the review of pull requests and focusing on releases on github. Unfortunately, the community ball got dropped when Alex left the team, and that's our fault. But we've been putting more time into picking up that ball again.

As for the roadmap, we do have a regularly maintained one, but again to our fault we have not maintained it on github. I will talk to the team about that and see if we can work that into the weekly opensource meeting.

Brendan

[Brendan Harris]

On Thursday, August 16, 2012 3:46:04 PM UTC-7, Brendan Harris wrote:

to come soon. However, the 0.95 code is there. We're actually now committing to 0.96.

A quick note on the 0.95 and 0.96 code: We push lots of fixes and features to github and internally we have the version bumped up, but we have not pulled all of that together into an official community release. I would like to see us roll out an official release with a lot of these fixes and features soon.

Brendan

[Carlos Tasada]

Hi Brendan,

I've been monitoring the commits in GitHubs, and pulling some fixes myself, but again, I'm not really sure about the stability of any of those version. The lack of official documentation is not helping either.

What do you thing about the next points?

- Check the issues in code.google and move the valid ones to GitHub, in this way everything is in a single place.

- Update the GitHub wiki with the changes in the 0.95 LinkedIn internal release.

- Open the results of your weekly 'open source meeting' to the community, with some kind of weekly newsletter, feedback, whatever

Regards,

Carlos.

Brendan

--

You received this message because you are subscribed to the Google Groups "project-voldemort" group.

To view this discussion on the web visit https://groups.google.com/d/msg/project-voldemort/-/OWsBLzOP8PIJ.

[Maarten Koopmans]

+1

- involve the community so you can , you know.... BUILD one

[Alex Feinberg]

I've left LinkedIn (after three years, I should add -- that's almost
eternity in Silicon Valley :-)), but the Voldemort team at LinkedIn is
now much larger than it has been during my tenure (there were months
where I was the only person working full time on V). The current
Voldemort team is superb and has a great depth of distributed systems,
database, and general programming-in-the-large experience amongst
them.

Github (pushes, pull requests, et al) will give a very different
picture of the project's liveness then the mailing list: I see
experimentation (e.g., https://github.com/voldemort/voldemort/pull/81
for LevelDB, https://github.com/voldemort/voldemort/pull/83 for some
of the work Peter Bailis did during his internship, etc...) and
improvements coming daily.

I still continue to lurk on this mailing lists and occasionally
comment on pull requests or pushes. One thing I do clearly miss about
Voldemort (especially while on-call) is how node failures (of any node
in the clustert) are a complete non-event. Several other LinkedIn
Alumni have said the same.

- af

[ctasada]

Hi guys,

13 days since last reply and no news.

I just wanted to share this article, which seems to point the same we where talking about. http://java.dzone.com/articles/nosql-job-trends-–-august-2012 Voldemort is slowly dying and I thing is up to us to change it.

Looking forward for you comments

[Francois]

This community thing is a little bit the chicken and the egg.

A good community need a commercial support, I mean a company who's
primary interest is the supported product.


As for us, we made the Voldemort choice perfectly knowing the lack of
community, but made it for the following reasons:


- We wanted, for ease of operations, to stay with a pure KV model at
the ring level. At the end of the day all we needed is a good, "pure",
simple KV system that we can tweak with our routing, storage, pki,
indexing and business extensions.

Voldemort has:
- 1: Good quality java code base, easy to understand and if necessary,
maintain. At least there is more than enough in V for our needs.
- 2: As Alex pointed out, friendly node failure behaviour
- 3: Basic dynamo-like model with few extensions that we don't need
- 4: Possibility to add a custom db engine

So a very good basis for a project where you want to shorten the
support chain. This is the key value (ah ah) of Voldemort.


Our vision is to bring APIs on top of the pure KV at a level which is
high enough to hide eventual consistency management from the app
developer ( our aim is to reduce dev work on app side by bringing
sector based APIs such as file management, internet of things, queuing
& subscriptions, user management, mail management, keyword
indexing... ), keeping it always extremely simple, so a good match
with Voldemort.


Given that we have to support Voldemort anyway ( this is critical for
us ), we have three options:
1) We maintain the Voldemort know-how just to support our own needs,
and add our tweaks. That's what we do anyway, we knew it upfront, and
I guess that's what everybody does here.
2) We invest and support - commercially - more than our own needs
(Maybe others have the same proposal)
3) We pray for LI to support us



What we need here is a discussion on those options, including LI:

- Are there any companies interested in getting commercial support, or
do they prefer self-maintenance, or does everybody want free, 24/7
support :-) ?
- Would they need app support around it ( e.g.: for building low level
apis for their business on top on eventually consistent data ) ?
- What is the long term vision of LI ? Do they prefer an externally
managed, self-sufficient community, or invest in a strong, LI managed
community, or a status quo which makes V an viable option for
companies who have the knowledge to write, or at least maintain it ?

At the end of the day, there is no free lunch.

Francois

On Sep 3, 4:19 pm, ctasada <ctas...@gmail.com> wrote:
> Hi guys,
>
> 13 days since last reply and no news.
>
> I just wanted to share this article, which seems to point the same we where
> talking

> about.http://java.dzone.com/articles/nosql-job-trends-–-august-2012 Voldemort

> is slowly dying and I thing is up to us to change it.
>

[Carlos Tasada]

Hi Francois,

I thing you explained really well the problematic.

AFAIK there is both you and Marteen with a whole product build on top of V. The company I'm working for is using V as a Key-Value storage for our internal services, and LinkedIn is doing the same. I don't known about the rest of people around here.

Right now V is part of my day-by-day work, but the lack of proper updates, a clear roadmap and a proper support service/community is starting to be a problem. I'm expecting that the service I'm maintaining will increase the amount of request around 5 times in the next months. I'm having quite some problems on the client code, that I've been fixing and pushing. 

I would be happy having some type of support and a clear future for the whole system. As a private company we cannot invest time and money in something that seems to be slowly dying.

My impression is that a better roadmap and some kind of support would help to attract people. One of the first things I do when I'm going to start using a new software/service is to check: updates frequency, open/closed issues and community size. Nowadays V doesn't have any of them.

In short I feel like I need to start looking for alternatives due to the lack of time to solve all the V issues by myself.

Regards,

Carlos.

--

[Maarten Koopmans]

+1 from me. I am giving it this month, then I'll start migrating or
writing my own.

I'd be happy to put effort in the community, but please, "open up" first.

--Maarten

> --
>
>

[Francois]

Marteen, had not much time to contact you as promised, but we should
definitely talk! We could join efforts if our needs are the same
(depends much on your operating environment, as for us if we rewrite
we go for out of the box hidden partitions, "disk aware" storage and
resync, plus some ring to host callback features for queuing). Even if
the philosophy and scope of our products are different, although some
similar aspects, I guess our core needs - ring wise - are similar.

On Sep 5, 12:27 pm, Maarten Koopmans <maarten.koopm...@gmail.com>
wrote:

> +1 from me. I am giving it this month, then I'll start migrating or
> writing my own.
>
> I'd be happy to put effort in the community, but please, "open up" first.
>

> --MaartenOn Tue, Sep 4, 2012 at 4:40 PM, Carlos Tasada <ctas...@gmail.com> wrote:
> > Hi Francois,
>
> > I thing you explained really well the problematic.
>
> > AFAIK there is both you and Marteen with a whole product build on top of V.
> > The company I'm working for is using V as a Key-Value storage for our
> > internal services, and LinkedIn is doing the same. I don't known about the
> > rest of people around here.
>
> > Right now V is part of my day-by-day work, but the lack of proper updates, a
> > clear roadmap and a proper support service/community is starting to be a
> > problem. I'm expecting that the service I'm maintaining will increase the
> > amount of request around 5 times in the next months. I'm having quite some
> > problems on the client code, that I've been fixing and pushing.
>
> > I would be happy having some type of support and a clear future for the
> > whole system. As a private company we cannot invest time and money in
> > something that seems to be slowly dying.
>
> > My impression is that a better roadmap and some kind of support would help
> > to attract people. One of the first things I do when I'm going to start
> > using a new software/service is to check: updates frequency, open/closed
> > issues and community size. Nowadays V doesn't have any of them.
>
> > In short I feel like I need to start looking for alternatives due to the
> > lack of time to solve all the V issues by myself.
>
> > Regards,
> > Carlos.
>

> > On Tue, Sep 4, 2012 at 12:10 PM, Francois <francois.vai...@ezcgroup.net>

[Lei Gao]

Hey guys,

Voldemort 0.96 is released today. Additional details can be found at http://data.linkedin.com/blog/2012/09/project-voldemort-096-release

Alex F., thank you for the continued support for Voldemort!

 Sunny, I am the project lead for Voldemort. I am more than happy to hear about you suggestions in improving Voldemort and the communication between the Voldemort team and the open source community.

Thanks,

Lei Gao

[Carlos Tasada]

Hi Lei,

That's great!!! Thanks a lot!!

Reading the blog post of the release seems that all the questions asked in this thread are covered:

- New release

- Cleaning up issues list and move them to GitHub

- Present a Roadmap

- Involve the community

I think now is basically up to us as a community to involve ourselves more in the way the project progress, and up to the LinkedIn team to keep the momentum and great work.

Thanks again Lei.

Carlos.

--
 
 

[Sunny Gleason]

Hi Lei, thank you so much for your considerate blog
post and email reply!

I've seen this model of Open Source before, where
an internal team throws releases over the wall to an
external community. A major challenge is that the
internal team's reward structure is based on purely
company-internal goals, not necessarily any external
goals such as significant contributions or adoption.

I'd love to invest more time into Voldemort -- I had a
lot of fun implementing the haildb data store. The major
things I'd be looking for in the next 12 months have
nothing to do with features:

1) make development environment much more friendly
(maven support & modules, better tests, better
components/isolation/abstraction to make it easier to
build & test storage engines and network transports/clients)
2) simpler code so that V's ability to handle failure modes
is more clear (now that a lot of the ideas are baked and
there is a lot more operational knowledge of Voldemort,
I think the code could be refactored for simpler implementation,
and in many cases, code removed from unused features)
3) path to more formal recognition for external contributors
to the project (so that external contributors can actually
(a) feel like part of the team (b) know they're not wasting
time developing a feature that won't be integrated and (c) have
an idea of when a contribution will enter a release candidate
and (d) reach a point where external contributions are not
primarily limited to just documentation and small patches)

Hope this helps! All the best -- I look forward to contributing
to Voldemort in the near future and beyond,

-Sunny

> --
>
>
>

[Francois]

Great! Thanks Lei.

Kind of things we would like to add (can propose some changes):

- Configurable hashing & routing strategy classes for 3N / 2N+1 for
disk to disk replication (allows for fast drive to drive rebuilds),
and key pattern based store configuration (e.g.: 2W/1R for immutable,
never versioned keys with no inconsistency resolving needs, 2W/2R or
2W/1R for others). Will avoid us hard patches :)

- Thinking around the 2 (or 3) N +1 pattern: the +1 backup node(s)
could be combined with a gateway to remote datacenter

- Longer term Diskmap storage (disk based chained hashmap with log
forward structure, cost/benefit cleaning and ssd based index for hot
start), but not sure its of any interest for other usages than ours
(built-in inconsistency resolver and version elimination, the front
needs to be designed for that). It still needs work for becoming
generic enough and it's only designed to reduce the memory footprint
on Multi TB datasets (btree all in memory for decent BDB performance
=> Too high server cost/GB), so opposite of your SSD works.


Francois

[Francois]

And about your request for suggestions: what would be helpful is to
know upfront if a change will be accepted or not before we start
working on it. We could have some kind of regular feature approval
sessions ( via skype ).

As an example, if we know upfront that configuration based hashing/
routing/repair classes ( today done as patches ) won't be accepted, we
won't spend time trying to propose an abstraction and will patch the
hard way after each release. :)

On Sep 8, 2:37 am, Lei Gao <gao...@gmail.com> wrote:
> Hey guys,
>

> Voldemort 0.96 is released today. Additional details can be found athttp://data.linkedin.com/blog/2012/09/project-voldemort-096-release