Re: The State of Doozer

1,640 views
Skip to first unread message

Matt Reiferson

unread,
Feb 26, 2013, 9:25:58 AM2/26/13
to doo...@googlegroups.com
We (bitly) have a fork as well that is production worthy.  We've been running it as part of a core service for over a year now.

Other than the obvious fixes needed to get it to compile it has been rock solid.

We have *not* had to pull in any of the other work going on in the rest of the more active forks.  I've only added a single new feature (SELF command) so that you can deterministically identify a node at runtime.

We do have some scripting that isn't open sourced (but could be) that adds starting and managing a cluster.

I would also be interested in giving a little TLC to the project.

On Mon, Feb 25, 2013 at 11:45 PM, Erik St. Martin <alak...@gmail.com> wrote:
We (skynet team) are also really into the idea of seeing doozer further developed, and have been in a similar boat with the abandonment of the project. We've actually been researching porting skynet over to ZooKeeper as it's more actively maintained, has some great tools surrounding it, and supports things like ephemeral nodes, so we can make sure nodes are removed when our services die, and it works pretty much exactly the same way doozer does.

Two other forks of doozer that we've pulled patches from are:


Keep me posted on how this comes along, I wouldn't mind contributing to some of the work that may need to be done.

Erik


On Monday, February 25, 2013 8:45:09 PM UTC-5, Robby Colvin wrote:
I'm hoping to implement doozer for my infrastructure and I'm sad about the state of the project. From what I can tell, the original authors of doozer have moved on to other things and that development has stopped. There are quite a few forks that have been updated, but there is still no canonical repository to my knowledge. There are quite a few commits in the network, though these ones stick out the most:


I was hoping that we could somehow round everyone up that is still interested in doozer and decide where to go from here. First, is either Keith or Blake still interested in moving forward with the project? If not, should we ask for more members to be added to the organization in order to bring steam back to the project? Or should we create a new organization to host doozer related repos? I understand that Heroku's requirements or situation have changed, but I don't see Zookeeper or DynamoDB as viable options for my situation. I still believe this project has value and would like to hear from everyone else that is using it.

--
You received this message because you are subscribed to the Google Groups "doozer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to doozer+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Brian Ketelsen

unread,
Feb 26, 2013, 9:41:17 AM2/26/13
to doo...@googlegroups.com
One of the things we really want for skynet is Ephemeral nodes - paths that disappear when they disconnect.  Eric mentioned earlier that one of the forks has this already.  This would be a great feature if we were to get a "canonical" fork.

We also have some tooling for doozer that could be open sourced pretty easily, like a CLI, it's skynet specific right now, but could be pretty easily fixed up to be generic.

Brian

Brian Ketelsen

unread,
Feb 26, 2013, 10:23:14 AM2/26/13
to doo...@googlegroups.com
How about we get everybody interested on a conference call tomorrow?  2PM Eastern?  I can send out a conference bridge number, and we can work out how to effectively manage a single canonical fork of Doozer.

Let me know if you're all interested in a call, and I will send out a conference number.

Brian

Erik St. Martin

unread,
Feb 26, 2013, 10:26:26 AM2/26/13
to doo...@googlegroups.com
https://github.com/araddon/doozerd is the fork I believe I saw with ephemeral node support. I'm not sure the state of that fork I haven't used it.

I'd definitely be willing to contribute a CLI similar to ZooKeepers for doozer if we can find a canonical place for doozer so that all these additions aren't scattered across 10 different forks.

Erik


On Tuesday, February 26, 2013 9:41:17 AM UTC-5, Brian Ketelsen wrote:

Gustavo Niemeyer

unread,
Feb 26, 2013, 10:51:45 AM2/26/13
to doo...@googlegroups.com
On Tue, Feb 26, 2013 at 12:23 PM, Brian Ketelsen <bket...@gmail.com> wrote:
> How about we get everybody interested on a conference call tomorrow? 2PM
> Eastern? I can send out a conference bridge number, and we can work out how
> to effectively manage a single canonical fork of Doozer.
>
> Let me know if you're all interested in a call, and I will send out a
> conference number.

Sounds like a good idea. Perhaps a G+ hangout if that works for everybody?

I've never used Doozer, but hoped to use it at one point, and may see
it being useful in the future. We ended up using ZooKeeper via gozk,
and then MongoDB since our needs didn't quite match the ZooKeeper
model entirely.


gustavo @ http://niemeyer.net

Keith Rarick

unread,
Feb 26, 2013, 1:37:42 PM2/26/13
to doo...@googlegroups.com
Hey folks, this all sounds fantastic. I'd love to open up the ha org
if there's interest to more people who want to keep things going.
Or whatever else helps remove friction.


Phil Whelan

unread,
Feb 26, 2013, 11:57:01 AM2/26/13
to doo...@googlegroups.com
Hi,

We, ActiveState, also have a fork of Doozerd, in which we have implemented ephemeral nodes. We use this extensively in our product, Stackato. The ephemeral nodes really help in getting an overview of processes running state throughout a cluster.

Bit more background here...

We have not put a great deal of effort in doozerd development and are limited on the resources we can assign to it. Some of the work we have done is on the Ruby client libraries, but some of that work is specific to our implementation.

We have found several limitations with Doozerd, we would like to address, but we are also considering alternatives such as ZooKeeper or some other (more light-weight than ZK) clustered key-value implementation. We depend on several features of doozerd, most notably watchers, ephemeral nodes, clustering and being light-weight.

Limitations we've found include... it degrades rapidly the more values you put in, it's slow to get a group of config values in one go (lots of tcp packets back-and-forth), becomes disabled if you post large values (3kb) requiring restart. Generally it's slow, for what it does, even with a single doozerd instance, which surprises me since it's data is all in RAM.

We are limited on development time we can commit to this, but we do have insights into using doozerd, as it's been a part of our product for a little while now. We would love to talk more.

Cheers,
Phil


On Tue, Feb 26, 2013 at 7:51 AM, Gustavo Niemeyer <gus...@niemeyer.net> wrote:

Brian Ketelsen

unread,
Feb 26, 2013, 1:55:10 PM2/26/13
to doo...@googlegroups.com
Perhaps we should wait another day before organizing a hangout (great idea for those outside USA, Gustavo!) so we can get more participants.

Brian

Brian Ketelsen

unread,
Feb 26, 2013, 2:15:07 PM2/26/13
to doo...@googlegroups.com
Or, we just ask Keith nicely to add someone from ActiveState, Bitly, Skynet, torbit, etc to the HA org and we work through our updates with pull requests and issues like any normal project.  If Keith is open to adding a few people to the org, we don't have much of a problem left to solve that can't be done with some good pull requests.

Robby Colvin

unread,
Feb 26, 2013, 2:17:29 PM2/26/13
to doo...@googlegroups.com
Wow, thank you for all the responses.

Is anyone opposed to continuing with using the ha org on Github and just having everyone push their changes upstream? Since Keith has given his blessing I think it might be better than risking another fork being abondoned later on. I also think it might be nice to have a sort of quorum so the work can be distributed easier and also keep clients and other tools in once place. Even if it was the few of us here I think it could go a long way.

Matt Reiferson

unread,
Feb 26, 2013, 2:21:59 PM2/26/13
to doo...@googlegroups.com
I would prefer it continue to stay where it is and Keith add a few of us, it's already got some momentum there and it should become the canonical repo.

I also agree with Brian that a conference call might be overkill, this mailing list + pull requests should suffice for feature proposals and working through design decisions.

Gustavo Niemeyer

unread,
Feb 26, 2013, 2:28:58 PM2/26/13
to doo...@googlegroups.com
On Tue, Feb 26, 2013 at 4:17 PM, Robby Colvin <geeta...@gmail.com> wrote:
> Wow, thank you for all the responses.
>
> Is anyone opposed to continuing with using the ha org on Github and just
> having everyone push their changes upstream? Since Keith has given his

Are all the different forks well tested and sensible to go upstream?
It'd be good to have one or two people that have a good grasp on the
internals of the project, and a slightly critical eye, to be doing a
round of integration.


gustavo @ http://niemeyer.net

Edward Muller

unread,
Feb 26, 2013, 1:56:45 PM2/26/13
to doo...@googlegroups.com
I've had passing interest in Doozer and would like to attend as well. I started a fork the other day with the intent of going through more active repos and pulling in changes that looked interesting/useful/etc. Note: I haven't actually started yet though.

Keith Rarick

unread,
Feb 26, 2013, 2:39:06 PM2/26/13
to doo...@googlegroups.com
On Tue, Feb 26, 2013 at 11:15 AM, Brian Ketelsen <bket...@gmail.com> wrote:
> Or, we just ask Keith nicely to add someone from ActiveState, Bitly, Skynet, torbit, etc to
> the HA org and we work through our updates with pull requests and issues like any normal
> project. If Keith is open to adding a few people to the org, we don't have much of a problem
> left to solve that can't be done with some good pull requests.

Those options don't have to be mutually exclusive. :)

Gustavo raises a good point that care should go into merging things in, but I'm
sure everyone here will be reasonable and not go making crazy commits without
any review. I'll go ahead and add the owners of those forks listed
above to the ha
org and the community can work out what to do collectively from there on.

I'd love to be in the conference call or google hangout if it happens.

Brian Ketelsen

unread,
Feb 26, 2013, 2:51:39 PM2/26/13
to doo...@googlegroups.com
That's awesome news, Keith.  And I am sure that with the amount of production experience we all have with Doozer, we'll all take great care not to break things. 

Let's do a Google+ hangout on Thursday, just to introduce everyone.  Seems like everyone involved who has requested involvement is within the US timezone span, so how about 2PM Eastern time on Thursday?  I'll post a URL when we get some concurrence on timing.

Keith Rarick

unread,
Feb 26, 2013, 2:53:34 PM2/26/13
to doo...@googlegroups.com
On Tue, Feb 26, 2013 at 11:39 AM, Keith Rarick <k...@xph.us> wrote:
> I'll go ahead and add the owners of those forks

Okay I'm not sure exactly what github accounts to use for some of the forks,
because several are org accounts and can't be added directly to the ha org.
Can folks reply here and identify user accounts for them?
https://github.com/musicglue/doozerd
https://github.com/soundcloud/doozerd
https://github.com/araddon/doozerd
https://github.com/bitly/doozerd

Keith Rarick

unread,
Feb 26, 2013, 2:58:11 PM2/26/13
to doo...@googlegroups.com
On Tue, Feb 26, 2013 at 11:51 AM, Brian Ketelsen <bket...@gmail.com> wrote:
> how about 2PM Eastern time on Thursday?

Ok.

Thursday is waza.heroku.com, but I'm sure I can step away for a bit.

Matt Reiferson

unread,
Feb 26, 2013, 3:00:05 PM2/26/13
to doo...@googlegroups.com
Thanks Keith,

for bitly, account is mreiferson

Robby Colvin

unread,
Feb 26, 2013, 3:02:52 PM2/26/13
to doo...@googlegroups.com
My fork doesn't have much right now, but I can volunteer to start adding some tests and help out with other things unless anyone has an objection to that. I can also beef up the Go client to be more compatible with the service. My Github username is geetarista.

Brian Ketelsen

unread,
Feb 26, 2013, 3:13:32 PM2/26/13
to doo...@googlegroups.com
For skynet, please add bketelsen and erikstmartin.

Thanks!

Erik St. Martin

unread,
Feb 26, 2013, 3:15:29 PM2/26/13
to doo...@googlegroups.com
I'm so pumped there is a community rallying behind doozer now, we've been concerned that we'd have to maintain it on our own, or to migrate skynet to something else.

Erik

Matt Reiferson

unread,
Feb 26, 2013, 3:17:16 PM2/26/13
to doo...@googlegroups.com
Also, while we have the whole group here, we should discuss some early ground rules given the fact that, at some level, we may have differing visions of where the project should go.

The priorities as I see them:

1. obvious TLC that enables easy compilation, fixes bugs, and brings the codebase up to date
2. discussion of our collective production experiences and documentation of some of the pain points that need work (this will help us agree on some sort of prioritized roadmap for feature/design changes)
3. keep the core codebase and purpose small :)

Relatedly, given the fact that there will be several cooks in the kitchen and able to merge things, we should commit to some agreement that pull requests need some sort of quorum of support (much in the spirit of doozer :).

Thoughts?

--

Brian Ketelsen

unread,
Feb 26, 2013, 3:28:32 PM2/26/13
to doo...@googlegroups.com
We were just discussing this internally.  How about we use some basic github tools to manage what gets done and how.

1.  For ideas, we submit issues.  Issues with enough (how many??) +1's get approved for work.  
2.  Work happens in forks, forks submit pull requests
3.  Pull requests get code reviewed, and require some sort of quorum/consensus for merging.  Go's LGTM plan works well.

We can create a wiki page when we've agreed to the guidelines.  We also need to agree on versioning, etc. so we don't create breaking changes.   We can create and publish a roadmap to help with this.

Perhaps this list is the best place to start airing out ideas.  When we see some solid patterns, we can move to issues on Github.  Also, I certainly hope for this to be as democratic as possible, so having a small committee will help that.  Too many chiefs kills too many projects.  As long as the big production instances of doozerd are represented, we should be good.

We're really excited to see life back in the doozer community!

Brian

Erik St. Martin

unread,
Feb 26, 2013, 3:29:04 PM2/26/13
to doo...@googlegroups.com
Agreed, I definitely think we should do pull-requests, and agree on a certain number of +1's / LGTM's before a pull-request is accepted, and possibly at the feature level. I'd definitely like to see a compiled list of things people would like to see implemented/fixed.

On Tuesday, February 26, 2013 3:17:16 PM UTC-5, snakes wrote:

Phil Whelan

unread,
Feb 26, 2013, 4:44:12 PM2/26/13
to doo...@googlegroups.com

Thanks Keith. From ActiveState, please add GitHub users srid and philwhln

Phil

--
You received this message because you are subscribed to the Google Groups "doozer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to doozer+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Thanks,
Phil

Cell : +1 (778) 233-4935

David Arnold

unread,
Jan 2, 2014, 8:56:39 PM1/2/14
to doo...@googlegroups.com
So, this whole thread was a while ago now, and there's nothing much happening on github.
What's the current state?  Is doozer dead?



d


On Tuesday, February 26, 2013 11:45:09 AM UTC+10, Robby Colvin wrote:
I'm hoping to implement doozer for my infrastructure and I'm sad about the state of the project. From what I can tell, the original authors of doozer have moved on to other things and that development has stopped. There are quite a few forks that have been updated, but there is still no canonical repository to my knowledge. There are quite a few commits in the network, though these ones stick out the most:

Reply all
Reply to author
Forward
0 new messages