Redis development model
2,431 views
Skip to first unread message
Salvatore Sanfilippo
Dec 9, 2014, 5:08:33 AM12/9/14
to Redis DB
Hello,
usually I prefer to hack on code instead of writing about the project
itself, however I can count people that provided an important
contribution to the Redis code base in the fingers of my hand, and
spare fingers remain.
One of those contributors is Matt Stancliff, so I can't (and I don't
want) ignore him when he disagrees about how the project is developed.
A few days ago he tweeted the following:
"@antirez many reasons listed here here also apply to Redis:
http://www.infoworld.com/article/2855057/application-development/why-iojs-decided-to-fork-nodejs.html"
The article is about the Node.js fork. This reply is not only to Matt,
but also a long due clarification needed after the London meetup,
where similar issues were raised.
The fact that there are unhappy people with the Redis development
process does not mean I'm going to change everything. Actually I'm
going to change a few things, but for the part I will not change, at
least I want to clarify *why*.
So this email is a mix of proposal to change things, and of reasons
why many things are not going to change. As such is split into
sections.
The benevolent dictator model
===
There are two main ways to run an open source project: one is with a
group of people that decide what enters and what not, called the "core
team". Two examples: the recent Node.js fork, and the Tcl core team.
Another model is when a benevolent dictator acts as a gatekeeper for
what can enter and what not. Both models have strengths and weakness,
and there are large projects using both with success.
The core team is nice because it looks like a more democratic process,
however the benevolent dictator model has the advantage that the code
has some more coherent vision about what to put inside and what not,
there is less risk of "project as a sum of feature".
Also a lot of this depends on the story of the project. Core teams
often are a result of the original project leader to step back (this
happened both in the Tcl and Node.js case), or, the project entering a
larger software institution of some kind.
However I'm pretty sure of one thing: when the core team is not formed
by the people that are actually going to write the code itself, this
is a good recipe for disasters. Code is not just about functionality,
is also about overall design, implementation, bug fixing, and
responsibility. When the benevolent dictator reviews a patch and let
it enter the project, there is a transfer of accountability from the
original committer to the project leader: she or he will stay late at
night to fix bugs in the contributed code if there is some critical
stuff.
In the core team is different but not so much if the core team is
composed of contributors *constantly working to the project*. If a
patch is voted by the majority as "ok", there are at least 50%+1 that
will be willing to fix problems.
Now, if you ask me, I like better the benevolent dictator way because
*some coherent vision* is better than no coherent vision, and it does
not happen every day that you get a core team that really share its
vision. It will be hard to make extreme changes with a core team, so
the project will run into a conservative mode (for example, remember
when I dropped plans for disk stores?). At the same time, it is also
hard to have an high barrier to entry, since things can get accepted
just because they look useful.
But, let's assume for a moment I'm in love with the core team setup.
Who is going to be the core team of Redis? It is barely viable when
there are 3 continuous committers, but actually practical when there
are 5. In the current setup it's just Matt and me, and Matt is a
recent core contributor. Technically he may join a startup in a few
months and I'm back into solo-mode, but regretting that I changed the
view of the project. To have a core team that votes composed of
non-core contributors is not acceptable IMHO because of
accountability. If there are other people that say "Let's add this", I
want to see them to fix stuff when shit happens, otherwise it's
unfair.
There is another problem: how much big Redis should be? Sometimes I'll
not accept features that look like legitimate additions only for two
reasons: *taste* and limit the project size. Expires on single
elements of data structures? No way. Loadable modules? Not going to
make the project better in my opinion. Disk store? Great, but we don't
have resources to cover well this use case and requires too big
changes.
At the same time when I release stuff under a BSD license I - really -
mean - it. Fork it, have fun, create a new team, beat me at my own
game. It's five years I write Redis, if you create an alternative,
viable development for it that suits better the needs of most users, I
can focus on new things without regretting it a single time.
TLDR of this section: don't get offended if you'll see me acting as a
filter for what can enter or not in the core, for the coding style,
quality, and so forth, or if, even for features I believe are perfect
for Redis, I want to read and understand the code before merging. I'll
be accountable for your code in a few weeks if you run away.
PRs and Issues reply time
===
Ok you can say, all right with the development model, barrier to entry
and all this stuff. But what about PRs and issues that are trivial
improvements and get merged after many months?
You are right, and here all I have to say is "mea culpa". When a PR
violates the project vision or code quality or whatever, I should
reply ASAP with a short note and close it. Or when it can get merged,
it should go inside the code ASAP otherwise it is a waste
contribution. The problem is that merging PRs or discarding them is
Super Time Consuming. When you have many high quality things to merge
you can switch to a mode where your job is mostly merging things. With
Redis I'm currently the main contributor of code, so this is not
possible. However I need to find a more balanced approach. Even if
I'll write small replies, I need to consistently reply ASAP to all the
issues / PRs, even just with what I need from the original contributor
to move forward faster, or closing the issue/PR ASAP if it's not in
target.
Otherwise there are several risks: don't merge useful stuff,
discourage possible contributors to contribute more, create a pile of
not handled issues / PRs where a few gems will not be easily
uncovered.
I'll change my work flow to do this. However I'm sure the risk is,
you'll see less code from me, since this will require me to spend
hours for week. After 6 months I can re-evaluate this, and follow up
here in the maling list to say, ok it worked, or, Redis was better
when I did not cared too much. We'll see.
Development model reloaded
===
Another issue is with the development model. There are a set of
features that are interesting and almost ready to merge where the
blocking part is stability concerns. Or sometimes I myself avoid to do
too big changes to the core because I don't really want to make the
unstable branch too much different from others in order to cherry pick
between branches. This happens since I ran the project as a "back
port" business, basically.
Let's be a bit more explicit: Usually you have different branches with
different levels of stability. In this model when a branch looks solid
enough and a set of features / fixes were added, and the branch looks
solid enough, it's moved from stable to testing. Later it will be
marked as stable. In this model you only port fixes to multiple
branches. Testing code is no longer a candidate for back ports. This
isolates different trees into different levels of stability
requirements. It's great to hack a lot into the current development
tree without fear.
This model makes a lot of sense when there are multiple contributors.
When there are one or two contributors, you mostly know the state of
each branch precisely, so I ended using a totally different model,
where instead most of the useful features are eventually back ported
into the current stable branch, and released with patch level
versions. This requires more care writing the code, but everybody
benefits from the new development after weeks. However as I see more
contributors lately trying to provide some help with the Redis core,
this model is starting to be lacking. More isolation between branches
may be useful.
Another problem is big features, like Redis Cluster. It blocked a new
3.0 release for a lot of time. Now we are finally ready for a stable
release of Redis Cluster, however, there was an error on how I handled
that. The right way was to release 3.0 ASAP with Cluster and mark the
feature as experimental. Most big features can be switched on or off.
Or when they can't, they receive a lot of testing since they are part
of the fundamental working of Redis (example, this would happen
replacing dict.c with some different hash table implementation).
So what about having a different development model with four total
active branches?
1) unstable: develop and merge freely.
2) testing: points of unstable that are feature-freezed since they
make sense for a release. They only get bug fixes.
3) rc: Release candidates, those become the new stable Redis. Every
time an RC is released as stable, the current testing is merged into
rc. Only gets bugs fixes.
4) stable: the current stable release. Only gets bug fixes.
So basically the only development branch is unstable.
In a model like that, we would also move the Github default branch to
"rc", in order to give some exposure to the code before it gets
released as stable.
However the problem is, who is going to test the "testing" released?
Users that are in need for specific features ASAP? We should probably
try to retain high standards in the code quality anyway, even if we no
longer backport stuff aggressively, in order to make sure that people
using non stable branches have a decent experience. This is the key to
get some testing, otherwise we are just playing moving code from a
branch to another without anything actually happening.
Moreover, developers that submitted patches should be ready to do QA
activities in their part of contributed code against testing...
TLDR: we could switch to a new development model with multiple
branches, but it only makes sense if actual testing happens at the
different stages, otherwise is useless.
Ok, I hope this helps. Feedbacks are welcome.
Cheers,
Salvatore
--
Salvatore 'antirez' Sanfilippo
open source developer - GoPivotal
http://invece.org
"Fear makes the wolf bigger than he is."
— German proverb
usually I prefer to hack on code instead of writing about the project
itself, however I can count people that provided an important
contribution to the Redis code base in the fingers of my hand, and
spare fingers remain.
One of those contributors is Matt Stancliff, so I can't (and I don't
want) ignore him when he disagrees about how the project is developed.
A few days ago he tweeted the following:
"@antirez many reasons listed here here also apply to Redis:
http://www.infoworld.com/article/2855057/application-development/why-iojs-decided-to-fork-nodejs.html"
The article is about the Node.js fork. This reply is not only to Matt,
but also a long due clarification needed after the London meetup,
where similar issues were raised.
The fact that there are unhappy people with the Redis development
process does not mean I'm going to change everything. Actually I'm
going to change a few things, but for the part I will not change, at
least I want to clarify *why*.
So this email is a mix of proposal to change things, and of reasons
why many things are not going to change. As such is split into
sections.
The benevolent dictator model
===
There are two main ways to run an open source project: one is with a
group of people that decide what enters and what not, called the "core
team". Two examples: the recent Node.js fork, and the Tcl core team.
Another model is when a benevolent dictator acts as a gatekeeper for
what can enter and what not. Both models have strengths and weakness,
and there are large projects using both with success.
The core team is nice because it looks like a more democratic process,
however the benevolent dictator model has the advantage that the code
has some more coherent vision about what to put inside and what not,
there is less risk of "project as a sum of feature".
Also a lot of this depends on the story of the project. Core teams
often are a result of the original project leader to step back (this
happened both in the Tcl and Node.js case), or, the project entering a
larger software institution of some kind.
However I'm pretty sure of one thing: when the core team is not formed
by the people that are actually going to write the code itself, this
is a good recipe for disasters. Code is not just about functionality,
is also about overall design, implementation, bug fixing, and
responsibility. When the benevolent dictator reviews a patch and let
it enter the project, there is a transfer of accountability from the
original committer to the project leader: she or he will stay late at
night to fix bugs in the contributed code if there is some critical
stuff.
In the core team is different but not so much if the core team is
composed of contributors *constantly working to the project*. If a
patch is voted by the majority as "ok", there are at least 50%+1 that
will be willing to fix problems.
Now, if you ask me, I like better the benevolent dictator way because
*some coherent vision* is better than no coherent vision, and it does
not happen every day that you get a core team that really share its
vision. It will be hard to make extreme changes with a core team, so
the project will run into a conservative mode (for example, remember
when I dropped plans for disk stores?). At the same time, it is also
hard to have an high barrier to entry, since things can get accepted
just because they look useful.
But, let's assume for a moment I'm in love with the core team setup.
Who is going to be the core team of Redis? It is barely viable when
there are 3 continuous committers, but actually practical when there
are 5. In the current setup it's just Matt and me, and Matt is a
recent core contributor. Technically he may join a startup in a few
months and I'm back into solo-mode, but regretting that I changed the
view of the project. To have a core team that votes composed of
non-core contributors is not acceptable IMHO because of
accountability. If there are other people that say "Let's add this", I
want to see them to fix stuff when shit happens, otherwise it's
unfair.
There is another problem: how much big Redis should be? Sometimes I'll
not accept features that look like legitimate additions only for two
reasons: *taste* and limit the project size. Expires on single
elements of data structures? No way. Loadable modules? Not going to
make the project better in my opinion. Disk store? Great, but we don't
have resources to cover well this use case and requires too big
changes.
At the same time when I release stuff under a BSD license I - really -
mean - it. Fork it, have fun, create a new team, beat me at my own
game. It's five years I write Redis, if you create an alternative,
viable development for it that suits better the needs of most users, I
can focus on new things without regretting it a single time.
TLDR of this section: don't get offended if you'll see me acting as a
filter for what can enter or not in the core, for the coding style,
quality, and so forth, or if, even for features I believe are perfect
for Redis, I want to read and understand the code before merging. I'll
be accountable for your code in a few weeks if you run away.
PRs and Issues reply time
===
Ok you can say, all right with the development model, barrier to entry
and all this stuff. But what about PRs and issues that are trivial
improvements and get merged after many months?
You are right, and here all I have to say is "mea culpa". When a PR
violates the project vision or code quality or whatever, I should
reply ASAP with a short note and close it. Or when it can get merged,
it should go inside the code ASAP otherwise it is a waste
contribution. The problem is that merging PRs or discarding them is
Super Time Consuming. When you have many high quality things to merge
you can switch to a mode where your job is mostly merging things. With
Redis I'm currently the main contributor of code, so this is not
possible. However I need to find a more balanced approach. Even if
I'll write small replies, I need to consistently reply ASAP to all the
issues / PRs, even just with what I need from the original contributor
to move forward faster, or closing the issue/PR ASAP if it's not in
target.
Otherwise there are several risks: don't merge useful stuff,
discourage possible contributors to contribute more, create a pile of
not handled issues / PRs where a few gems will not be easily
uncovered.
I'll change my work flow to do this. However I'm sure the risk is,
you'll see less code from me, since this will require me to spend
hours for week. After 6 months I can re-evaluate this, and follow up
here in the maling list to say, ok it worked, or, Redis was better
when I did not cared too much. We'll see.
Development model reloaded
===
Another issue is with the development model. There are a set of
features that are interesting and almost ready to merge where the
blocking part is stability concerns. Or sometimes I myself avoid to do
too big changes to the core because I don't really want to make the
unstable branch too much different from others in order to cherry pick
between branches. This happens since I ran the project as a "back
port" business, basically.
Let's be a bit more explicit: Usually you have different branches with
different levels of stability. In this model when a branch looks solid
enough and a set of features / fixes were added, and the branch looks
solid enough, it's moved from stable to testing. Later it will be
marked as stable. In this model you only port fixes to multiple
branches. Testing code is no longer a candidate for back ports. This
isolates different trees into different levels of stability
requirements. It's great to hack a lot into the current development
tree without fear.
This model makes a lot of sense when there are multiple contributors.
When there are one or two contributors, you mostly know the state of
each branch precisely, so I ended using a totally different model,
where instead most of the useful features are eventually back ported
into the current stable branch, and released with patch level
versions. This requires more care writing the code, but everybody
benefits from the new development after weeks. However as I see more
contributors lately trying to provide some help with the Redis core,
this model is starting to be lacking. More isolation between branches
may be useful.
Another problem is big features, like Redis Cluster. It blocked a new
3.0 release for a lot of time. Now we are finally ready for a stable
release of Redis Cluster, however, there was an error on how I handled
that. The right way was to release 3.0 ASAP with Cluster and mark the
feature as experimental. Most big features can be switched on or off.
Or when they can't, they receive a lot of testing since they are part
of the fundamental working of Redis (example, this would happen
replacing dict.c with some different hash table implementation).
So what about having a different development model with four total
active branches?
1) unstable: develop and merge freely.
2) testing: points of unstable that are feature-freezed since they
make sense for a release. They only get bug fixes.
3) rc: Release candidates, those become the new stable Redis. Every
time an RC is released as stable, the current testing is merged into
rc. Only gets bugs fixes.
4) stable: the current stable release. Only gets bug fixes.
So basically the only development branch is unstable.
In a model like that, we would also move the Github default branch to
"rc", in order to give some exposure to the code before it gets
released as stable.
However the problem is, who is going to test the "testing" released?
Users that are in need for specific features ASAP? We should probably
try to retain high standards in the code quality anyway, even if we no
longer backport stuff aggressively, in order to make sure that people
using non stable branches have a decent experience. This is the key to
get some testing, otherwise we are just playing moving code from a
branch to another without anything actually happening.
Moreover, developers that submitted patches should be ready to do QA
activities in their part of contributed code against testing...
TLDR: we could switch to a new development model with multiple
branches, but it only makes sense if actual testing happens at the
different stages, otherwise is useless.
Ok, I hope this helps. Feedbacks are welcome.
Cheers,
Salvatore
--
Salvatore 'antirez' Sanfilippo
open source developer - GoPivotal
http://invece.org
"Fear makes the wolf bigger than he is."
— German proverb
Josiah Carlson
Dec 9, 2014, 1:30:34 PM12/9/14
to redi...@googlegroups.com
Replies inline.
+1 for Salvatore as benevolent dictator.
My favorite is when someone drops a broken PR for unwanted functionality, then uses "we" to describe the ongoing effort involved in getting the patch merged. That always gives me a laugh.
Code talks. Perfectly reasonable. Want responsibility and your opinion to mean something in the Redis project? Write code, fix bugs, and contribute to Redis.
This is perfectly reasonable, and it's awesome that you are going to spend a bit more time on this.
But if you are feeling bad about not spending enough time in the past, I wouldn't feel too bad. As the author/maintainer for a couple other smaller projects, easily 75% of the code submissions I get are never going to get merged for one reason or another. If I were getting PRs at the rate of Redis, I'd definitely be prioritizing my own work just for the sake of having code I don't need to work to understand and either help fix well enough to get into core, or explain (several times) why the code won't make it in.
I like the idea and practice of shipping disabled by default features that are not yet stable. It gives people something supported, something they can work with, but something that they know may not work 100%. I do it with some of my projects.
You keep building it, I'll keep telling people how to use it better.
- Josiah
Cheers,
Salvatore
--
Salvatore 'antirez' Sanfilippo
open source developer - GoPivotal
http://invece.org
"Fear makes the wolf bigger than he is."
— German proverb
--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.
Matt Stancliff
Dec 9, 2014, 2:33:11 PM12/9/14
to redi...@googlegroups.com
Hello!
> usually I prefer to hack on code instead of writing about the project
> itself, however I can count people that provided an important
> contribution to the Redis code base in the fingers of my hand, and
> spare fingers remain.
Good idea: find more contributors.
Bad idea: remove fingers until finger count matches contributor count.
> One of those contributors is Matt Stancliff, so I can't (and I don't
> want) ignore him when he disagrees about how the project is developed.
> A few days ago he tweeted the following:
>
> "@antirez many reasons listed here here also apply to Redis:
> http://www.infoworld.com/article/2855057/application-development/why-iojs-decided-to-fork-nodejs.html”
My approach to getting changes made to Redis: https://www.youtube.com/watch?v=9gSQg1i_q2g
I just like to complain a lot.
I have no right to complain. Redis has been my full time job for over a year. Without Salvatore, I’d probably be eating dirt on a beach in Monaco because nobody else wanted to hire me (though, Monaco beach would be nicer than a winter in NYC).
There’s more complaining below. Don’t take it too harshly. Things are good, but they could always be better. I work from home and don’t have a regularly scheduled outlet for venting minor frustrations, so things… build up. [curse your sudden but inevitable betrayal!]
> The benevolent dictator model
> ===
Ooops. Let’s start right there.
Currently, the Redis development process feels hostile to outside developers and outside contributions. (even my small fixes get ignored, and I’m _amazing_: https://github.com/antirez/redis/issues?q=is%3Aopen+author%3Amattsta)
Developers often “test the waters” by submitting a small patch. If the patch is acted on, they’ll try to contribute more fixes or features. If they get ignored, they go away and never come back. When patches get ignored for years, they don’t bother to get more involved with the project. So, Redis has a very small pool of full time developers to help because Redis doesn’t *want* help. Redis wants to be left alone. (yes, this is attempted to be fixed below.)
So, historically, there aren’t really “Redis developers” in the open source community because most non-bugfix contributions get ignored or rejected.
This year, I tried to get more people’s contributions added so they feel less left out with these:
- hiredis maintenance - https://github.com/redis/hiredis/pull/234 - 14 commits
- Fix all Redis comments - https://github.com/antirez/redis/pull/1902 - 29 commits
- Add all simple, correct, but ignored Redis PRs - https://github.com/antirez/redis/pull/1906 - 65 commits across 30+ authors, going back 3+ years
- Improve the Redis cmsgpack implementation - https://github.com/antirez/lua-cmsgpack/pull/30 - 30 commits
We can also see the “Redis is open source, but contribution isn’t really wanted” behavior reflected in this Redis mailing list itself.
This mailing list is, no offense, useless for actual Redis development. The last time Salvatore asked the community for feedback on Redis (https://groups.google.com/forum/#!topic/redis-db/Oazt2k7Lzz4%5B1-25-false%5D), the result was essentially “let’s not do that again.” [[That’s also because email is just horrible. The thread split off into 10 different discussion topics, and keeping up with those replies gets overwhelming quickly.]]
This mailing list does have some some great people (the not-me Matt, Dvir, Josiah, Jan, Michael (and the non-Michael Michel), etc), but we can’t actually discuss any Redis issues here. At the end of any discussion or community consensus, Dearest Leader can show up and say “thanks, but no way.”
Because of the threat of instant dismissal with no recourse, there’s not much of a reason to have new thoughts [[that doesn’t stop me from making wild and crazy things anyway]].
Running a community is a full time job. Coding is a full time job. Maintenance is a full time job. Future planning and figuring out what users want is a full time job. If one person tries to be the gateway for _everything_, the other tasks will fall behind.
When people post ideas or suggestions or _really_ technical problems with Redis on the mailing list, they sometimes get ignored or denied.
A good example is Bill’s great post about the Redis process itself from after the dev meeting in October with no replies (I blame myself for that too):
"Redis Improvement & Community Discussion"
https://groups.google.com/forum/#!topic/redis-db/cUYrsp8Q7I4
Bill is there practically begging to help, but… no go. no va. imbranato.
> Also a lot of this depends on the story of the project. Core teams
> often are a result of the original project leader to step back (this
> happened both in the Tcl and Node.js case), or, the project entering a
> larger software institution of some kind.
There’s also a middle ground where outside contributions are discussed, refined, and compromised on until they fit the project.
See things like:
- https://www.python.org/dev/peps/pep-0001/
- http://www.erlang.org/eeps/eep-0002.html
- https://github.com/erlang/otp/wiki/EEP-light
> However I'm pretty sure of one thing: when the core team is not formed
> by the people that are actually going to write the code itself, this
> is a good recipe for disasters.
“core team” means developers who write code, approve code, merge code.
> Code is not just about functionality,
> is also about overall design, implementation, bug fixing, and
> responsibility.
Nobody would argue against that.
> When the benevolent dictator reviews a patch and let
> it enter the project, there is a transfer of accountability from the
> original committer to the project leader: she or he will stay late at
> night to fix bugs in the contributed code if there is some critical
> stuff.
See, that’s the problem there—the “project leader” shouldn’t be the only person responsible for fixing all code in the entire project forever.
Look at Linux. Is Linus out there *fixing* code (or even writing code) any more? He’s just there to yell at people, merge, set policy, and make releases, but he still runs everything.
It’s arguable whether one person can do all code development and complete community management for a very popular open source project.
> In the core team is different but not so much if the core team is
> composed of contributors *constantly working to the project*. If a
> patch is voted by the majority as "ok", there are at least 50%+1 that
> will be willing to fix problems.
If a core team must vote to override other team members on issues, the core team has already failed.
> Now, if you ask me, I like better the benevolent dictator way because
> *some coherent vision* is better than no coherent vision, and it does
> not happen every day that you get a core team that really share its
> vision.
If we can’t find anybody else to “share the vision,” then where is the flaw? If nobody agrees with what I say is right, am I actually correct?
> At the same time, it is also
> hard to have an high barrier to entry, since things can get accepted
> just because they look useful.
That’s just about trusting people to do the right thing. :)
> But, let's assume for a moment I'm in love with the core team setup.
> Who is going to be the core team of Redis? It is barely viable when
> there are 3 continuous committers, but actually practical when there
> are 5.
Current Redis contributors are just a small pool of crazy people (in a good way) who enjoy the work even when they get ignored for a while. More developers could show up, but it requires a community cultivation and openness.
Sometimes contributing to Redis feels like being a 5 year old jumping around saying “look at me! look over here! look what I can do!” just to get the attention of the authority figure who can shine the light of merging and feedback upon you.
> If there are other people that say "Let's add this", I
> want to see them to fix stuff when shit happens, otherwise it's
> unfair.
Core contributors/team members are responsible for everything else too (or their own specializations), not just adding their own pet projects. :)
Specialization is a good point too (thanks, me, for bringing it up). Right now, antirez is responsible for: creating new features (allowing new commands, making improvements to existing infrastructure), fixing existing features (data structures, networking, event loop, config parsing, dependency management/upgrades, …), making releases, merging contributions, creating custom distributed systems, creating custom file formats, writing documentation, merging documentation, running the website, testing everything….
At any given time, only one thing is being worked on, but other things may also need improvements too.
The best I can do right now is try to pre-approve everything and try to guess what antirez wants for patches/improvements/code quality review.
> There is another problem: how much big Redis should be?
bigger than a bread box, smaller than a 747.
> At the same time when I release stuff under a BSD license I - really -
> mean - it. Fork it, have fun, create a new team, beat me at my own
> game. It's five years I write Redis, if you create an alternative,
> viable development for it that suits better the needs of most users, I
> can focus on new things without regretting it a single time.
Noted! I’ll release my big merge branch with large changes so people can at least use them.
> TLDR of this section: don't get offended if you'll see me acting as a
> filter for what can enter or not in the core, for the coding style,
> quality, and so forth, or if, even for features I believe are perfect
> for Redis, I want to read and understand the code before merging.
If the filter is also the main developer, the community will always fall behind.
When I’m creating new code things, sometimes I ignore the world for weeks at a time to make code progress. That’s where having a trusted, distributed team helps.
> I’ll be accountable for your code in a few weeks if you run away.
That’s the difference between just a community pull request and an actual core contributor. One just walks away, the other stays.
> Redis I'm currently the main contributor of code, so this is not possible.
Well, you’re the main code contributor, release maintainer, feature planner, maintenance do-er, and a dozen other things. :)
If I’m driving on a long trip and keep saying “I’m the main driver, nobody else can drive,” I can’t feel bad about not having any help while driving.
> Otherwise there are several risks: don't merge useful stuff,
> discourage possible contributors to contribute more, create a pile of
> not handled issues / PRs where a few gems will not be easily
> uncovered.
Very true. People won’t contribute big fixes and improvements if their small fixes get ignored. Contributors want their (useful) work to be added, not sit around for years untouched.
> When a PR violates the project vision or code quality or whatever, I should
> reply ASAP with a short note and close it.
Note the use of terms there: “project vision,” “code quality,” “whatever.” — Those are personal issues that can’t be delegated. If the project is 100% What-antirez-Wants, then, provably, other people _can’t_ help too much.
> I'll change my work flow to do this. However I'm sure the risk is,
> you'll see less code from me, since this will require me to spend
> hours for week. After 6 months I can re-evaluate this, and follow up
> here in the maling list to say, ok it worked, or, Redis was better
> when I did not cared too much. We'll see.
The development model of Redis is actually: “antirez-brain-as-a-service.” That’s just an observation, not a judgement. That model has created great software so far.
But, today there are over 2 million developers who use Redis at over 100,000 companies. Does having one person be the gatekeeper for the needs of _everybody_ still work?
We can’t have outside contributors if the guideline for getting things added is “What antirez wants” because: valid fixes and contributions will be rejected if they don’t fit the unspoken "design-in-antirez-head" _and_ it essentially turns all contributors into personal assistants to Dear Leader instead of people who have their own goals and get their own rewards.
> Development model reloaded
> ===
Some people have very strong opinions about that movie.
> So what about having a different development model with four total
> active branches?
> 1) unstable: develop and merge freely.
> 2) testing: points of unstable that are feature-freezed since they
> make sense for a release. They only get bug fixes.
> 3) rc: Release candidates, those become the new stable Redis. Every
> time an RC is released as stable, the current testing is merged into
> rc. Only gets bugs fixes.
> 4) stable: the current stable release. Only gets bug fixes.
These are kinda solved problems. Not sure if we need to invent another solution here.
See things like:
- http://nvie.com/posts/a-successful-git-branching-model/
- http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows#Dictator-and-Lieutenants-Workflow
- https://github.com/erlang/otp/wiki/submitting-patches
We also could use a more responsive version system. Example: the currently released version doesn’t build on old Linux systems (my fault, my patch broke it), but because we don’t have “minor maintenance” version numbers, it’ll just remain broken until the next bigger release (and people will keep opening issues because they can’t build).
We shouldn’t be afraid of releasing fixes because it makes version numbers go up. :) So, something like 2.8.18.1 or -1 or anything else that seems right. (The “maint” branch concept.)
> Moreover, developers that submitted patches should be ready to do QA
> activities in their part of contributed code against testing…
We should allow automated tests on all branches, not just what gets merged into an official branch.
Bonus points if we can set up a multi-environment automated test system (opensolaris, five versions of linux, freebsd, openbsd, …).
As a contributor, I can’t be held responsible for knowing my code doesn’t compile on an 8 year old version of RedHat without the proper resources in place. :)
> TLDR: we could switch to a new development model with multiple
> branches, but it only makes sense if actual testing happens at the
> different stages, otherwise is useless.
Testing at different stages?
I think the only thing we’re missing is a “proposed updates” branch where we can merge some _maybe_ useful things together, test, get feedback, then either merge into the develop/unstable branch. Then we create release branches _from_ develop/unstable instead of running two or three release branches at the same time.
It’s great how Redis currently has a “very stable” release (2.8.x) and a more forward-looking release (unstable/3.0+), but it adds to the burden of creating releases when everything is backported instead branched from a common master (as you mentioned previously too).
Basically, try the picture and description at http://nvie.com/posts/a-successful-git-branching-model/ (then, each version also has a long-lived maintenance branch too for minor fixes without bumping larger version numbers).
Summary:
- redis: yay!
- progress: yay!
- openness: yay!
- transparency: yay!
- overworked developers: boo!
- not engaging with community: boo!
- discouraging people who want to help: boo!
- Redis reaching 3.0: yay!
-Matt
> usually I prefer to hack on code instead of writing about the project
> itself, however I can count people that provided an important
> contribution to the Redis code base in the fingers of my hand, and
> spare fingers remain.
Bad idea: remove fingers until finger count matches contributor count.
> One of those contributors is Matt Stancliff, so I can't (and I don't
> want) ignore him when he disagrees about how the project is developed.
> A few days ago he tweeted the following:
>
> "@antirez many reasons listed here here also apply to Redis:
My approach to getting changes made to Redis: https://www.youtube.com/watch?v=9gSQg1i_q2g
I just like to complain a lot.
I have no right to complain. Redis has been my full time job for over a year. Without Salvatore, I’d probably be eating dirt on a beach in Monaco because nobody else wanted to hire me (though, Monaco beach would be nicer than a winter in NYC).
There’s more complaining below. Don’t take it too harshly. Things are good, but they could always be better. I work from home and don’t have a regularly scheduled outlet for venting minor frustrations, so things… build up. [curse your sudden but inevitable betrayal!]
> The benevolent dictator model
> ===
Currently, the Redis development process feels hostile to outside developers and outside contributions. (even my small fixes get ignored, and I’m _amazing_: https://github.com/antirez/redis/issues?q=is%3Aopen+author%3Amattsta)
Developers often “test the waters” by submitting a small patch. If the patch is acted on, they’ll try to contribute more fixes or features. If they get ignored, they go away and never come back. When patches get ignored for years, they don’t bother to get more involved with the project. So, Redis has a very small pool of full time developers to help because Redis doesn’t *want* help. Redis wants to be left alone. (yes, this is attempted to be fixed below.)
So, historically, there aren’t really “Redis developers” in the open source community because most non-bugfix contributions get ignored or rejected.
This year, I tried to get more people’s contributions added so they feel less left out with these:
- hiredis maintenance - https://github.com/redis/hiredis/pull/234 - 14 commits
- Fix all Redis comments - https://github.com/antirez/redis/pull/1902 - 29 commits
- Add all simple, correct, but ignored Redis PRs - https://github.com/antirez/redis/pull/1906 - 65 commits across 30+ authors, going back 3+ years
- Improve the Redis cmsgpack implementation - https://github.com/antirez/lua-cmsgpack/pull/30 - 30 commits
We can also see the “Redis is open source, but contribution isn’t really wanted” behavior reflected in this Redis mailing list itself.
This mailing list is, no offense, useless for actual Redis development. The last time Salvatore asked the community for feedback on Redis (https://groups.google.com/forum/#!topic/redis-db/Oazt2k7Lzz4%5B1-25-false%5D), the result was essentially “let’s not do that again.” [[That’s also because email is just horrible. The thread split off into 10 different discussion topics, and keeping up with those replies gets overwhelming quickly.]]
This mailing list does have some some great people (the not-me Matt, Dvir, Josiah, Jan, Michael (and the non-Michael Michel), etc), but we can’t actually discuss any Redis issues here. At the end of any discussion or community consensus, Dearest Leader can show up and say “thanks, but no way.”
Because of the threat of instant dismissal with no recourse, there’s not much of a reason to have new thoughts [[that doesn’t stop me from making wild and crazy things anyway]].
Running a community is a full time job. Coding is a full time job. Maintenance is a full time job. Future planning and figuring out what users want is a full time job. If one person tries to be the gateway for _everything_, the other tasks will fall behind.
When people post ideas or suggestions or _really_ technical problems with Redis on the mailing list, they sometimes get ignored or denied.
A good example is Bill’s great post about the Redis process itself from after the dev meeting in October with no replies (I blame myself for that too):
"Redis Improvement & Community Discussion"
https://groups.google.com/forum/#!topic/redis-db/cUYrsp8Q7I4
Bill is there practically begging to help, but… no go. no va. imbranato.
> Also a lot of this depends on the story of the project. Core teams
> often are a result of the original project leader to step back (this
> happened both in the Tcl and Node.js case), or, the project entering a
> larger software institution of some kind.
See things like:
- https://www.python.org/dev/peps/pep-0001/
- http://www.erlang.org/eeps/eep-0002.html
- https://github.com/erlang/otp/wiki/EEP-light
> However I'm pretty sure of one thing: when the core team is not formed
> by the people that are actually going to write the code itself, this
> is a good recipe for disasters.
> Code is not just about functionality,
> is also about overall design, implementation, bug fixing, and
> responsibility.
> When the benevolent dictator reviews a patch and let
> it enter the project, there is a transfer of accountability from the
> original committer to the project leader: she or he will stay late at
> night to fix bugs in the contributed code if there is some critical
> stuff.
Look at Linux. Is Linus out there *fixing* code (or even writing code) any more? He’s just there to yell at people, merge, set policy, and make releases, but he still runs everything.
It’s arguable whether one person can do all code development and complete community management for a very popular open source project.
> In the core team is different but not so much if the core team is
> composed of contributors *constantly working to the project*. If a
> patch is voted by the majority as "ok", there are at least 50%+1 that
> will be willing to fix problems.
> Now, if you ask me, I like better the benevolent dictator way because
> *some coherent vision* is better than no coherent vision, and it does
> not happen every day that you get a core team that really share its
> vision.
> At the same time, it is also
> hard to have an high barrier to entry, since things can get accepted
> just because they look useful.
> But, let's assume for a moment I'm in love with the core team setup.
> Who is going to be the core team of Redis? It is barely viable when
> there are 3 continuous committers, but actually practical when there
> are 5.
Sometimes contributing to Redis feels like being a 5 year old jumping around saying “look at me! look over here! look what I can do!” just to get the attention of the authority figure who can shine the light of merging and feedback upon you.
> If there are other people that say "Let's add this", I
> want to see them to fix stuff when shit happens, otherwise it's
> unfair.
Specialization is a good point too (thanks, me, for bringing it up). Right now, antirez is responsible for: creating new features (allowing new commands, making improvements to existing infrastructure), fixing existing features (data structures, networking, event loop, config parsing, dependency management/upgrades, …), making releases, merging contributions, creating custom distributed systems, creating custom file formats, writing documentation, merging documentation, running the website, testing everything….
At any given time, only one thing is being worked on, but other things may also need improvements too.
The best I can do right now is try to pre-approve everything and try to guess what antirez wants for patches/improvements/code quality review.
> There is another problem: how much big Redis should be?
> At the same time when I release stuff under a BSD license I - really -
> mean - it. Fork it, have fun, create a new team, beat me at my own
> game. It's five years I write Redis, if you create an alternative,
> viable development for it that suits better the needs of most users, I
> can focus on new things without regretting it a single time.
> TLDR of this section: don't get offended if you'll see me acting as a
> filter for what can enter or not in the core, for the coding style,
> quality, and so forth, or if, even for features I believe are perfect
> for Redis, I want to read and understand the code before merging.
When I’m creating new code things, sometimes I ignore the world for weeks at a time to make code progress. That’s where having a trusted, distributed team helps.
> I’ll be accountable for your code in a few weeks if you run away.
> Redis I'm currently the main contributor of code, so this is not possible.
If I’m driving on a long trip and keep saying “I’m the main driver, nobody else can drive,” I can’t feel bad about not having any help while driving.
> Otherwise there are several risks: don't merge useful stuff,
> discourage possible contributors to contribute more, create a pile of
> not handled issues / PRs where a few gems will not be easily
> uncovered.
> When a PR violates the project vision or code quality or whatever, I should
> reply ASAP with a short note and close it.
> I'll change my work flow to do this. However I'm sure the risk is,
> you'll see less code from me, since this will require me to spend
> hours for week. After 6 months I can re-evaluate this, and follow up
> here in the maling list to say, ok it worked, or, Redis was better
> when I did not cared too much. We'll see.
But, today there are over 2 million developers who use Redis at over 100,000 companies. Does having one person be the gatekeeper for the needs of _everybody_ still work?
We can’t have outside contributors if the guideline for getting things added is “What antirez wants” because: valid fixes and contributions will be rejected if they don’t fit the unspoken "design-in-antirez-head" _and_ it essentially turns all contributors into personal assistants to Dear Leader instead of people who have their own goals and get their own rewards.
> Development model reloaded
> ===
Some people have very strong opinions about that movie.
> So what about having a different development model with four total
> active branches?
> 1) unstable: develop and merge freely.
> 2) testing: points of unstable that are feature-freezed since they
> make sense for a release. They only get bug fixes.
> 3) rc: Release candidates, those become the new stable Redis. Every
> time an RC is released as stable, the current testing is merged into
> rc. Only gets bugs fixes.
> 4) stable: the current stable release. Only gets bug fixes.
See things like:
- http://nvie.com/posts/a-successful-git-branching-model/
- http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows#Dictator-and-Lieutenants-Workflow
- https://github.com/erlang/otp/wiki/submitting-patches
We also could use a more responsive version system. Example: the currently released version doesn’t build on old Linux systems (my fault, my patch broke it), but because we don’t have “minor maintenance” version numbers, it’ll just remain broken until the next bigger release (and people will keep opening issues because they can’t build).
We shouldn’t be afraid of releasing fixes because it makes version numbers go up. :) So, something like 2.8.18.1 or -1 or anything else that seems right. (The “maint” branch concept.)
> Moreover, developers that submitted patches should be ready to do QA
We should allow automated tests on all branches, not just what gets merged into an official branch.
Bonus points if we can set up a multi-environment automated test system (opensolaris, five versions of linux, freebsd, openbsd, …).
As a contributor, I can’t be held responsible for knowing my code doesn’t compile on an 8 year old version of RedHat without the proper resources in place. :)
> TLDR: we could switch to a new development model with multiple
> branches, but it only makes sense if actual testing happens at the
> different stages, otherwise is useless.
I think the only thing we’re missing is a “proposed updates” branch where we can merge some _maybe_ useful things together, test, get feedback, then either merge into the develop/unstable branch. Then we create release branches _from_ develop/unstable instead of running two or three release branches at the same time.
It’s great how Redis currently has a “very stable” release (2.8.x) and a more forward-looking release (unstable/3.0+), but it adds to the burden of creating releases when everything is backported instead branched from a common master (as you mentioned previously too).
Basically, try the picture and description at http://nvie.com/posts/a-successful-git-branching-model/ (then, each version also has a long-lived maintenance branch too for minor fixes without bumping larger version numbers).
Summary:
- redis: yay!
- progress: yay!
- openness: yay!
- transparency: yay!
- overworked developers: boo!
- not engaging with community: boo!
- discouraging people who want to help: boo!
- Redis reaching 3.0: yay!
-Matt
Salvatore Sanfilippo
Dec 9, 2014, 4:25:41 PM12/9/14
to Redis DB
On Tue, Dec 9, 2014 at 7:30 PM, Josiah Carlson <josiah....@gmail.com> wrote:
> +1 for Salvatore as benevolent dictator.
>
> My favorite is when someone drops a broken PR for unwanted functionality,
> then uses "we" to describe the ongoing effort involved in getting the patch
> merged. That always gives me a laugh.
>
> Code talks. Perfectly reasonable. Want responsibility and your opinion to
> mean something in the Redis project? Write code, fix bugs, and contribute to
> Redis.
Thanks Josiah, I forgot to say one thing in my original email, that
> +1 for Salvatore as benevolent dictator.
>
> My favorite is when someone drops a broken PR for unwanted functionality,
> then uses "we" to describe the ongoing effort involved in getting the patch
> merged. That always gives me a laugh.
>
> Code talks. Perfectly reasonable. Want responsibility and your opinion to
> mean something in the Redis project? Write code, fix bugs, and contribute to
> Redis.
your email remembers me.
Maybe it's just the term that sucks "benevolent dictator", let's talk
about "single point of responsibility", but indeed "code talks" as you
say, but also use cases, experience. So why those can't give you
right-to-vote in the current setup, before doing things, I try to
listen to what the community here believes is appropriate or not, and
I try to do this in an open-minded fashion, so I believe that the
current Redis design is also the result of many point of views here.
> Redis, I'd definitely be prioritizing my own work just for the sake of
> having code I don't need to work to understand and either help fix well
> enough to get into core, or explain (several times) why the code won't make
> it in.
emails, PRs, issues, or the ability to do some work today.
It's sad but it's like that. To do "a bit of issues", "a bit of code"
requires an higher degree of discipline. Often all my discipline is
spent in actually doing something to bring the project forward instead
of writing Hacker News or alike.
> You keep building it, I'll keep telling people how to use it better.
Salvatore Sanfilippo
Dec 9, 2014, 5:34:54 PM12/9/14
to Redis DB
On Tue, Dec 9, 2014 at 8:33 PM, Matt Stancliff <mstan...@pivotal.io> wrote:
> I just like to complain a lot.
>
> I have no right to complain. Redis has been my full time job for over a year. Without Salvatore, I’d probably be eating dirt on a beach in Monaco because nobody else wanted to hire me (though, Monaco beach would be nicer than a winter in NYC).
That's not true, you contributed many things lately, so you have right
> I just like to complain a lot.
>
> I have no right to complain. Redis has been my full time job for over a year. Without Salvatore, I’d probably be eating dirt on a beach in Monaco because nobody else wanted to hire me (though, Monaco beach would be nicer than a winter in NYC).
to complain ;-)
> Currently, the Redis development process feels hostile to outside developers and outside contributions. (even my small fixes get ignored, and I’m _amazing_: https://github.com/antirez/redis/issues?q=is%3Aopen+author%3Amattsta)
>
> Developers often “test the waters” by submitting a small patch. If the patch is acted on, they’ll try to contribute more fixes or features. If they get ignored, they go away and never come back. When patches get ignored for years, they don’t bother to get more involved with the project. So, Redis has a very small pool of full time developers to help because Redis doesn’t *want* help. Redis wants to be left alone. (yes, this is attempted to be fixed below.)
>
> So, historically, there aren’t really “Redis developers” in the open source community because most non-bugfix contributions get ignored or rejected.
518
$ git log | grep Stancliff | wc -l
89
Does not look like an inexpugnable castle to me, but yes, there is a
barrier to entry that is large due to project focus, quality
requirements, and, lack of time, in some random order.
> This year, I tried to get more people’s contributions added so they feel less left out with these:
> - hiredis maintenance - https://github.com/redis/hiredis/pull/234 - 14 commits
> - Fix all Redis comments - https://github.com/antirez/redis/pull/1902 - 29 commits
> - Add all simple, correct, but ignored Redis PRs - https://github.com/antirez/redis/pull/1906 - 65 commits across 30+ authors, going back 3+ years
> - Improve the Redis cmsgpack implementation - https://github.com/antirez/lua-cmsgpack/pull/30 - 30 commits
proposed the changes in a more systematic way, performing some sanity
check. You basically provided your time in order to make it possible
to merge them.
In some way you also tricked me, basically, into re-reading many of
those commits that I skipped (and it was a good idea).
But I found a few bad things that fortunately I skipped. For example
one completely violated safety of Redis Cluster fail overs.
In the lua-cmsgpack there was this that I fixed a few days ago since
it broken build in Solaris:
https://github.com/antirez/lua-cmsgpack/commit/1852b23068efeda00d9ead61f1e721ecfb667af5
The above should not pass reviews. I think that I've just a big focus
on quality, and this requires me to read the code I merge, and when I
don't do it I regret not doing it.
> We can also see the “Redis is open source, but contribution isn’t really wanted” behavior reflected in this Redis mailing list itself.
>
> This mailing list is, no offense, useless for actual Redis development. The last time Salvatore asked the community for feedback on Redis (https://groups.google.com/forum/#!topic/redis-db/Oazt2k7Lzz4%5B1-25-false%5D), the result was essentially “let’s not do that again.” [[That’s also because email is just horrible. The thread split off into 10 different discussion topics, and keeping up with those replies gets overwhelming quickly.]]
>
> This mailing list does have some some great people (the not-me Matt, Dvir, Josiah, Jan, Michael (and the non-Michael Michel), etc), but we can’t actually discuss any Redis issues here. At the end of any discussion or community consensus, Dearest Leader can show up and say “thanks, but no way.”
1) "Linenoise updated to be more VT100 compatible" <- user complain
about linenoise incompatibility with old windows-only terminals.
2) "PFCOUNT can be used on slaves now" <- Asked in this mailing list.
I took some time to figure out better solutions, did not found
anything, implemented what proposed.
3) "ZSCAN no longer report very small scores as 0" <- User reported
and fixed. (p.s. about code reviews, I checked your patch and rewrote
it because there was a problem. To hit "merge" is not a great idea
without reading).
4) "Don't show the ASCII logo if syslog is enabled." <- User complain
that could be ignored. Took my time, and acted.
5) EXPERIMENTAL: Diskless replication <- Proposed by Redis Labs at the
London meetup, implemented by me after writing a lot of non trivial
code.
6) "Transparent Huge Pages detection and reporting in logs" reported
many times as "Redis is slow" by the community. But of course nobody
cares of spending hours to actually fix the problem.
7) Many Lua scripting enhancements: Bitops API, cjson upgrade and
tests, cmsgpack upgrade." <- Your contrb, accepted without complains.
8) "Total and instantaneous Network bandwidth tracking in INFO." <-
Implemented because somebody asked about this in Stack Overflow. I did
not ever replied to the Q, I implemented it.
Are we talking of what? Seriously.
> A good example is Bill’s great post about the Redis process itself from after the dev meeting in October with no replies (I blame myself for that too):
> "Redis Improvement & Community Discussion"
> https://groups.google.com/forum/#!topic/redis-db/cUYrsp8Q7I4
>
> Bill is there practically begging to help, but… no go. no va. imbranato.
reply unfortunately. This is not lack of interest, in this instance.
To be fair, when the email was posted, I totally missed it. I later
discovered it thanks to Michel Martens, put it in my TODO list, and
was never to find the time to reply.
Sorry Bill, the changes you propose to Sentinel are interesting for
me, and I remember you reported a pathological condition where a
sentinel is monitoring too many masters and/or when there are too many
sentinels, were you observed a delay. I think about it from time to
time, but then there is something else, and is gone from my head.
>> Also a lot of this depends on the story of the project. Core teams
>> often are a result of the original project leader to step back (this
>> happened both in the Tcl and Node.js case), or, the project entering a
>> larger software institution of some kind.
>
> There’s also a middle ground where outside contributions are discussed, refined, and compromised on until they fit the project.
> See things like:
> - https://www.python.org/dev/peps/pep-0001/
> - http://www.erlang.org/eeps/eep-0002.html
> - https://github.com/erlang/otp/wiki/EEP-light
as much as a redis-dev mailing list with is lower traffic. We can just
talk there.
However lately I tried multiple times to get feedbacks here before
implementing stuff, but got very little replies.
We can't fix a lack of resources problem by writing RFCs. We would end
writing most of them ourselves. It's a lot better if we just write
messages in the mailing list.
>> When the benevolent dictator reviews a patch and let
>> it enter the project, there is a transfer of accountability from the
>> original committer to the project leader: she or he will stay late at
>> night to fix bugs in the contributed code if there is some critical
>> stuff.
>
> See, that’s the problem there—the “project leader” shouldn’t be the only person responsible for fixing all code in the entire project forever.
>
> Look at Linux. Is Linus out there *fixing* code (or even writing code) any more? He’s just there to yell at people, merge, set policy, and make releases, but he still runs everything.
>
> It’s arguable whether one person can do all code development and complete community management for a very popular open source project.
many people involved into a project. The many people involved with
Linux, are, 90% (or more?) payed by companies.
Let's see how cool is OSS in practice: I write code for Redis because
I'm payed. You write code for Redis because you are payed, you were a
Redis user before, but Pivotal was not sponsoring your work, so no
much from you, and know we see you are capable of great contributions,
like quicklists. Pieter wrote a lot of code for Redis when he was
payed to do so, and so forth.
Most of the OSS development you see in system software like MySQL,
Linux kernel, and so forth, is form developers that pay the bill with
it.
So IMHO Redis can't be handled like Linux because it is not a big
enough project, there are not enough companies interested in
sponsoring developers, and si simple enough that people interested in
new developments can more easily fork in-house and do what they need.
So as long I write a big percentage of the code, I'm going to be able
to fix a big percentage of the code.
>> In the core team is different but not so much if the core team is
>> composed of contributors *constantly working to the project*. If a
>> patch is voted by the majority as "ok", there are at least 50%+1 that
>> will be willing to fix problems.
>
> If a core team must vote to override other team members on issues, the core team has already failed.
things inside. If there are 3/5 votes ok, there are 3 developers that
endorse a given change and will work on it.
> If we can’t find anybody else to “share the vision,” then where is the flaw? If nobody agrees with what I say is right, am I actually correct?
developers. Once you have N people working constantly to the core, you
can scale and say, what about doing this core team stuff?
So the problem is that there are no contrib. But actually it is slowly
happening now, and there are a bit more contribs and they are around
for some time now. And I don't think I'm less an asshole I used to be,
it's just that the project is more mainstream, there are payed people
working at it, and new enthusiasts (that if not payed soon or later
stop contributing, likely).
> Current Redis contributors are just a small pool of crazy people (in a good way) who enjoy the work even when they get ignored for a while. More developers could show up, but it requires a community cultivation and openness.
>
> Sometimes contributing to Redis feels like being a 5 year old jumping around saying “look at me! look over here! look what I can do!” just to get the attention of the authority figure who can shine the light of merging and feedback upon you.
I conceptually accept, are well implemented, and complete, I try to
just hit "merge".
For example your work on quicklist is pending because there is a
blocking thing about RDB format, which is otherwise a regression
compared to the past, and does not allow Redis Cluster to benefit from
quciklists enough (actually it makes migrating small lists much slower
than before).
Your work on Geocoding is pending because it needs to get a
refactoring to avoid duplicating a lot of code which is common with
sorted sets.
The two above are not ignored, but still not ready to be merged.
Sometimes it's me. For example I've still to read the JSON thing, but
I de-prioritized it because the initial feeling makes me thing it's
better to look at other patches than JSON. Maybe I'll read it and
think, wow, that's what we need, but it is unlikely. So I sent time
looking at quicklists, geocoding, IPv6 fixes, and other stuff you
contributed.
>> If there are other people that say "Let's add this", I
>> want to see them to fix stuff when shit happens, otherwise it's
>> unfair.
>
> Core contributors/team members are responsible for everything else too (or their own specializations), not just adding their own pet projects. :)
>
> Specialization is a good point too (thanks, me, for bringing it up). Right now, antirez is responsible for: creating new features (allowing new commands, making improvements to existing infrastructure), fixing existing features (data structures, networking, event loop, config parsing, dependency management/upgrades, …), making releases, merging contributions, creating custom distributed systems, creating custom file formats, writing documentation, merging documentation, running the website, testing everything….
>
> At any given time, only one thing is being worked on, but other things may also need improvements too.
>
> The best I can do right now is try to pre-approve everything and try to guess what antirez wants for patches/improvements/code quality review.
needed feature, you implemented this, I'm willing to merge.
This is parallelization. Note that you did not had to ask me if this
was going to get merged.
> If the filter is also the main developer, the community will always fall behind.
contrib-and-run-away contributors. If patches are not reviewed, is a
big problem in a few weeks / months.
> That’s the difference between just a community pull request and an actual core contributor. One just walks away, the other stays.
>
>> Redis I'm currently the main contributor of code, so this is not possible.
>
> Well, you’re the main code contributor, release maintainer, feature planner, maintenance do-er, and a dozen other things. :)
>
> If I’m driving on a long trip and keep saying “I’m the main driver, nobody else can drive,” I can’t feel bad about not having any help while driving.
replying but I don't understand what you mean anymore.
Let's be clear: do you think we have a possible, theoretically large
(3, 4, 5?), pull of Redis core developers, constant in time, that
provide quality work, so we can run the project as a distributed
effort?
When I'm quite a few weeks, nothing happens. If you remove your latest
contributions, it is very rare to see *anything* of big. So I don't
understand what we are talking about.
I'm driving because I'm mostly alone. Now you are contributing, and I
try to merge your code with some priority compared to any other
contributor, of course. But also your hacking time must be aligned
with the project needs.
Did you noted that I rarely try to put new features? Instead I
implement stuff like: latency doctor to help people fix stuff. THP
checks to avoid that people run with bad configs, and so forth.
Quicklist was a step in the right direction. But JSON, the Geocoding
API (even if that is interesting work), Loadable modules, are all:
let's implement new things.
This of course will have a much larger barrier to entry, nobody asked
to have certain features, so they may be added or not, but are
certainly not prioritized.
So my feeling is that I'm driving alone, but when you help to drive, I
think I merge your contributions, otherwise who merged your 85
commits, and is ready to merge ~2000 lines of code for quicklists?
> Very true. People won’t contribute big fixes and improvements if their small fixes get ignored. Contributors want their (useful) work to be added, not sit around for years untouched.
which is not ok IMHO.
>> When a PR violates the project vision or code quality or whatever, I should
>> reply ASAP with a short note and close it.
>
> Note the use of terms there: “project vision,” “code quality,” “whatever.” — Those are personal issues that can’t be delegated. If the project is 100% What-antirez-Wants, then, provably, other people _can’t_ help too much.
understand you don't like it, but I really believe there is value in
having a small project.
You should really release your fork with loadable modules, JSON, Geo
API as-it-is, and everything else. Maybe a different model is
possible.
However I don't want to have a project that grows into directions I
think are not good, I'm more happy with a slower growth.
> But, today there are over 2 million developers who use Redis at over 100,000 companies. Does having one person be the gatekeeper for the needs of _everybody_ still work?
Yes, can work. A bigger software that handles more use cases? Probably
not.
However I don't see the issue black or white like you. Give me more
long-term developers, and I can put them to do good work.
I've a huge list of things that are priorities... and there is an
issue open. Those stuff are obvious merge-them if implemented
correctly, and I would sure help to point in the right direction if
the first implementation is not perfect.
But you say all the people in queue to write those parts are away
doing something else because my way of handling the project.
> We can’t have outside contributors if the guideline for getting things added is “What antirez wants” because: valid fixes and contributions will be rejected if they don’t fit the unspoken "design-in-antirez-head" _and_ it essentially turns all contributors into personal assistants to Dear Leader instead of people who have their own goals and get their own rewards.
> We also could use a more responsive version system. Example: the currently released version doesn’t build on old Linux systems (my fault, my patch broke it), but because we don’t have “minor maintenance” version numbers, it’ll just remain broken until the next bigger release (and people will keep opening issues because they can’t build).
can't do that?
> We shouldn’t be afraid of releasing fixes because it makes version numbers go up. :) So, something like 2.8.18.1 or -1 or anything else that seems right. (The “maint” branch concept.)
>> Moreover, developers that submitted patches should be ready to do QA
>> activities in their part of contributed code against testing…
>
> We should allow automated tests on all branches, not just what gets merged into an official branch.
> Bonus points if we can set up a multi-environment automated test system (opensolaris, five versions of linux, freebsd, openbsd, …).
>
> As a contributor, I can’t be held responsible for knowing my code doesn’t compile on an 8 year old version of RedHat without the proper resources in place. :)
was broken in a very obvious way. The same patch could be broken in a
way that compiles, passes tests, but is still broken.
So the fix is to have more care of what one writes, fixes, merges.
More care = more trust. More trust = more contribs accepted.
This is how Pieter Noordhuis rewrote a lot of Redis internals.
Just an example:
src ➤ git blame t_zset.c | grep antirez | wc -l
1493
src ➤ git blame t_zset.c | grep Noordhuis | wc -l
1350
And I was much more strict back then than I'm now. I think I'm not
easy to work with, but if we'll get committed contributors that are
willing to provide high quality work like Pieter's provided, like your
quicklist implementation, I'll just hit "merge" without problems. Give
me JSON, Loadable modules, and other stuff I don't think are going to
make Redis better in the long term, and I'll turn again into the same
closed minded guy.
Josiah Carlson
Dec 9, 2014, 6:27:20 PM12/9/14
to redi...@googlegroups.com
On Tue, Dec 9, 2014 at 1:25 PM, Salvatore Sanfilippo <ant...@gmail.com> wrote:
On Tue, Dec 9, 2014 at 7:30 PM, Josiah Carlson <josiah....@gmail.com> wrote:
> +1 for Salvatore as benevolent dictator.
>
> My favorite is when someone drops a broken PR for unwanted functionality,
> then uses "we" to describe the ongoing effort involved in getting the patch
> merged. That always gives me a laugh.
>
> Code talks. Perfectly reasonable. Want responsibility and your opinion to
> mean something in the Redis project? Write code, fix bugs, and contribute to
> Redis.
Thanks Josiah, I forgot to say one thing in my original email, that
your email remembers me.
Maybe it's just the term that sucks "benevolent dictator", let's talk
about "single point of responsibility", but indeed "code talks" as you
say, but also use cases, experience. So why those can't give you
right-to-vote in the current setup, before doing things, I try to
listen to what the community here believes is appropriate or not, and
I try to do this in an open-minded fashion, so I believe that the
current Redis design is also the result of many point of views here.
Maybe my penchant for benevolent dictators is the result of my Python history, as Guido has managed to operate as contributor and high-level decision maker for the language for something like 22-23 years now. That works partly because people trust Guido, but it also works because there is a pretty big core membership that helps direct the Python language and libraries (like Linux/Linus as Matt pointed out).
One of the bigger challenges is deciding whose opinion matters. I choose to stay out of this mostly because it's really messy, but also *because* I'm not writing and maintaining core Redis. I have seen enough people who haven't written a single line of code tell others what should be built and how it should be built* that I've lost my appetite for it, and I try to stay out of it unless I'm specifically asked, if I think it's actively bad for Redis, or if someone asks whether you might accept the changes (then I try to channel my inner Salvatore).
* Despite having been a core dev/maintainer of Python, I am at least as guilty of bikeshedding in python-dev as anyone else, which is one of the reasons why I jokingly apologized about it during Redisconf 2012. Recognizing that I was one of those people doing what I hated, I have more or less extracted myself from the Python community for the sake of my own personal sanity.
> Redis, I'd definitely be prioritizing my own work just for the sake of
> having code I don't need to work to understand and either help fix well
> enough to get into core, or explain (several times) why the code won't make
> it in.
That's what you end feeling like, with a binary choice. It's either
emails, PRs, issues, or the ability to do some work today.
It's sad but it's like that. To do "a bit of issues", "a bit of code"
requires an higher degree of discipline. Often all my discipline is
spent in actually doing something to bring the project forward instead
of writing Hacker News or alike.
It's even tougher because you are working from home, which pushes the home/work split to have work dominate even when you are supposedly "off work". When your job is *your* open source project? I honestly can't imagine how you manage to step away in the evenings after a workday with the *months* of work still sitting in your TODO list, issue tracker, and otherwise.
If there is one thing that is nice about an office and a 9-5 gig, it's that I can (and do) walk away from it when I go home. That said, you are living the open source developer's dream :)
> You keep building it, I'll keep telling people how to use it better.
Thanks :-)
Someone paid me to write a book about how to use your software. Thank you.
- Josiah
--
Salvatore 'antirez' Sanfilippo
open source developer - GoPivotal
http://invece.org
"Fear makes the wolf bigger than he is."
— German proverb
Marc Gravell
Dec 9, 2014, 7:18:38 PM12/9/14
to redi...@googlegroups.com
> Someone paid me to write a book about how to use your software. Thank you.
And I've travelled to some fun places to talk about it (redis), and have been paid to take the time to write not one but two libraries for it (confusingly, for the same language/platform).
Frankly, and based in part on my own experiences running multiple (much more minor) open source projects - I have to observe that the reality is that: there isn't a "core team" - a "core team" does not exist when there are 1.something members (no offence for the .something, Matt). Until the core has quorum, it : is antirez++. Much as I would love to contribute directly: I'm simply not a c/c++ person currently. Although I'm actually half tempted to dust off my prior skills and go bug-hunting!
So for my money: all heil our benevolent-dictator-for-now! (with a future eye to establishing a different system when sustainable).
And Matt: you raise some excellent points about approachability etc.
Marc
Mariano Pérez Rodríguez
Dec 9, 2014, 7:35:21 PM12/9/14
to redi...@googlegroups.com
Just my 2 (longish) cents.
I've been learning about Redis and developing with it for about a year and a half now. I believe it to be a GREAT project, and are tremendously grateful to all its devs for it. I know I'm not the only one that feels that way, and I know you all probably don't hear it as often as you should IMHO. My point is that I don't feel Redis is a "small" project, it's just a project with a very passive user-base, but a very big user base nonetheless.Now, how can the Redis development model benefit from both approaches? I don't think you're gonna like this one very much, but I think Matt's modules-thing-a-ma-bob might be instrumental to it, allow me to explain. One can decompose Redis in two (very broad) categories (forgive me if my rudimentary understanding of the Redis codebase led me to oversimplify this):
- the core codebase
- the not-so-core codebase (couldn't think of anything else)
The core codebase contains the event loop, networking magic, (maybe) RDB/AOF, Sentinel, Cluster, etc.
The not-so-core codebase contains the algorithms and data structures Redis implements.
Much of the needs that may arise concerning the not-so-core codebase can (up to a point) be implemented in Lua, but not all of them, and not as optimally as one might expect. So why not separate these two? One could have Matt's modules deal with the not-so-core part (say, one could have "standard" modules "STRING", "HYPERLOGLOG", "ZSET", etc), allowing for easy entry for contributors trying to get "non-standard" modules for Tries, Matrixes, Geocoding, etc, whilst maintaining them segregated from the more "critical" parts. As time passes by and these "non-standard" modules reach maturity and meet with the Redis' team approval, they can be included in the default distribution alongside the "standard" ones. A simple config line could indicate which modules to load by default.
This could decouple the ever-increasing feature list backlog, from the more involved core development. Heck, even RDB/AOF could be "modularized" to allow for experimentation in the persistence model. Heck^2, maybe the networking subsystem could too!
I sincerely think that this could take your mind off of the more "mundane" PRs and Issues, whilst at the same time attaining atomization of the codebase, allowing for "departments" of sorts (this has its own cons BTW, but they can be worked around with clean interfaces and disciplined development) dealing with specific algorithms or data structures or modules.
I'm sorry if I can't propose a "deeper" introspection of the whole development model and limited myself to whining about an instrument to a possible end, I don't mean to sound un-constructive, but rather really really really believe it could move things forward and provide for a "controlled-openness" of the whole development process.
TL;DR: dictator is fine but lieutenants needed, I love modules, I believe you should love them too (:P).
DISCLAIMER: my only contribution to Redis is a severely dumb ANSI thingy, I'm sorry if I sounded like "you MUST do this thing although I've only submitted a 2-char PR", it wasn't my intention, just trying to help (in the only way I'm currently able to) a project I REALLY REALLY like :)
Cheers!...
...Mariano.
CharSyam
Dec 10, 2014, 1:08:29 AM12/10/14
to redi...@googlegroups.com
I vote +1 as benevolent dictator model.
Redis is adapt it. :)
Felix Gallo
Dec 10, 2014, 1:48:59 AM12/10/14
to redi...@googlegroups.com
It's clearly Salvatore's project to run how he likes, but I feel, like Matt does, that it could be more "effective" in the sense of making valuable progress forwards. Salvatore gets diverted by other work, family, real life; pull requests languish, philosophy goes unexplained, some moves are made hastily (e.g. the many and varied attempts at cluster), some are made slowly. These are all fine and understandable things that we all experience in our daily lives, but would be smoothed out by the horsepower and varied experience and expertise given a healthy pool of contributors.
"Sometimes I'll not accept features that look like legitimate additions only for two reasons: *taste* and limit the project size. Expires on single elements of data structures? No way. Loadable modules? Not going to
make the project better in my opinion."
make the project better in my opinion."
There might be great reasons why expires on single elements of data structures don't exist; or why there's no data structure nesting; or why there's no sunion. But they're not algorithmic reasons. All of those things carry support risk, but redis-db is already flooded with do-my-homework-for-me posts in broken english -- it's not clear that it could get any worse. So that goes down in my book to taste, and I would suspect a far and away majority of redis users (95%+) would find an orthogonal instruction set, nested data structures and single-element-of-hash expire incredibly tasteful and exciting, and not at all outside of the Redis spirit. Other contributors could help do this sort of apparently 'tasteless' work.
And there might be a great reason why loadable modules could cause issues -- not the least of which being community fragmentation and having to ask which modules you have loaded for every question in redis-db. But consider the flipside: a world in which people could contribute data structures and algorithms in performant C not burdened with the restrictions and hacks of the Lua implementation, but yet still also benefit from the mainline of Redis work. Fostering an ecosystem is an incredibly powerful way of adding a coefficient to the utility of your work, and shifting the burden of pull requests and esoteric features out to a crowdsourced approval process.
Anyway, some hopefully constructive thoughts. I hope they help, Salvatore.
F.
On Tue, Dec 9, 2014 at 10:08 PM, CharSyam <char...@gmail.com> wrote:
I vote +1 as benevolent dictator model.Redis is adapt it. :)
--
Salvatore Sanfilippo
Dec 10, 2014, 3:14:45 AM12/10/14
to Redis DB
Thank you for your contribs, I appreciate the replies I saw in this thread.
I can't stretch the project into a direction I dislike, but I can't
ignore that we need to fix stuff. The only thing that is not going to
change is that I need to like a feature before it to be able to enter
the core.
But that said, this is going to happen:
1) I'll check and reply or close to every past issue or pull request.
All the new issues / PRs will get a reply in a matter of days in the
worst case.
2) Redis 3.0.0 will be released as stable. Only the cluster feature
will be marked as experimental for some time (this will be cleared as
soon as there is a user base that is actually testing the
implementation in production).
3) A new "testing" branch will be created. New things will get merged
*only* into unstable. When we are ready for a release, unstable will
be merged into testing. When testing is solid, it's going to become
the new release.
4) Testing and stable releases are going to only get bug fixes and
stuff that are like bug fixes even if they are not, like warnings for
bad kernel parameters or alike.
5) I'll try to force a faster turnover for releases.
6) I'll split this mailing list into two, redis and redis-dev. All the
development stuff will happen in the other side. I'll post a new
message detailing the changes I'm going to do for big features there.
I'll reply to everybody that posts a message about new features, and
finally, we'll have CONTRIBUTING explaining that they need to post a
message there for a new feature to get accepted.
Based on the above, I'll re-evaluate the workflow after 6 months from
now. I hope to see new contributors, and also contributors willing to
do testing, review patches, add tests not just for new code but for
regressions, and so forth. My fingers are crossed. If the result is
that in 6 months all the work I was doing with Redis is not
compensated by contributors, the core is less stable, and things are
not going well compared to today, I'm likely to return back to the
current model. If things will be better instead, everybody wins.
Cheers,
Salvatore
I can't stretch the project into a direction I dislike, but I can't
ignore that we need to fix stuff. The only thing that is not going to
change is that I need to like a feature before it to be able to enter
the core.
But that said, this is going to happen:
1) I'll check and reply or close to every past issue or pull request.
All the new issues / PRs will get a reply in a matter of days in the
worst case.
2) Redis 3.0.0 will be released as stable. Only the cluster feature
will be marked as experimental for some time (this will be cleared as
soon as there is a user base that is actually testing the
implementation in production).
3) A new "testing" branch will be created. New things will get merged
*only* into unstable. When we are ready for a release, unstable will
be merged into testing. When testing is solid, it's going to become
the new release.
4) Testing and stable releases are going to only get bug fixes and
stuff that are like bug fixes even if they are not, like warnings for
bad kernel parameters or alike.
5) I'll try to force a faster turnover for releases.
6) I'll split this mailing list into two, redis and redis-dev. All the
development stuff will happen in the other side. I'll post a new
message detailing the changes I'm going to do for big features there.
I'll reply to everybody that posts a message about new features, and
finally, we'll have CONTRIBUTING explaining that they need to post a
message there for a new feature to get accepted.
Based on the above, I'll re-evaluate the workflow after 6 months from
now. I hope to see new contributors, and also contributors willing to
do testing, review patches, add tests not just for new code but for
regressions, and so forth. My fingers are crossed. If the result is
that in 6 months all the work I was doing with Redis is not
compensated by contributors, the core is less stable, and things are
not going well compared to today, I'm likely to return back to the
current model. If things will be better instead, everybody wins.
Cheers,
Salvatore
Mariano Pérez Rodríguez
Dec 10, 2014, 10:03:23 AM12/10/14
to redi...@googlegroups.com
Just one tiny suggestion regarding this:
The only thing that is not going to change is that I need to like a feature before it to be able to enter the core.
Maybe if you could find the time to comment on why you dislike a particular PR the general anxiety can be better appeased: understanding your vision for Redis is important order to avoid effort duplication and butthurtness :)
Regards.
Michel Martens
Dec 10, 2014, 10:25:39 AM12/10/14
to redi...@googlegroups.com
On Wed, Dec 10, 2014 at 3:02 PM, Mariano Pérez Rodríguez
<mariano.per...@gmail.com> wrote:
> Maybe if you could find the time to comment on why you dislike a particular
> PR the general anxiety can be better appeased: understanding your vision for
> Redis is important order to avoid effort duplication and butthurtness :)
There's a tendency lately (in OSS in general) to code a solution for
<mariano.per...@gmail.com> wrote:
> Maybe if you could find the time to comment on why you dislike a particular
> PR the general anxiety can be better appeased: understanding your vision for
> Redis is important order to avoid effort duplication and butthurtness :)
some problem and offer a pull request without ever discussing the use
case. That works great in some projects, where growth is encouraged,
but in other projects it works better to discuss the use cases first.
I imagine the frustration of having a pull request rejected is huge,
but for the maintainer it's not a very enjoyable situation either.
Even thought the preferred process is summarized in the contributing
guidelines, there are lots of pull requests with features that were
never discussed, and that's one of the reasons why other valuable
contributions get a bit lost in that pile.
Salvatore Sanfilippo
Dec 10, 2014, 10:29:25 AM12/10/14
to Redis DB
On Wed, Dec 10, 2014 at 4:02 PM, Mariano Pérez Rodríguez
<mariano.per...@gmail.com> wrote:
> Maybe if you could find the time to comment on why you dislike a particular
> PR the general anxiety can be better appeased: understanding your vision for
> Redis is important order to avoid effort duplication and butthurtness :)
Very true, I'm going to spend a lot of time in PRs / Issues /
<mariano.per...@gmail.com> wrote:
> Maybe if you could find the time to comment on why you dislike a particular
> PR the general anxiety can be better appeased: understanding your vision for
> Redis is important order to avoid effort duplication and butthurtness :)
Redis-dev mailing list.
I'll write less code, but I always lament that the project gets too
big, so for a few months is not going to be a drama :-)
Mariano Pérez Rodríguez
Dec 10, 2014, 10:43:12 AM12/10/14
to redi...@googlegroups.com
There's a tendency lately (in OSS in general) to code a solution for some problem and offer a pull request without ever discussing the use case. That works great in some projects, where growth is encouraged, but in other projects it works better to discuss the use cases first.
Yes, I agree completely, just jumping into the pool without first discussing with the user and devbase is plain rude, but once the deed is done I believe it's better to steer it into a proper discussion rather than leave the PR to rot :P
Very true, I'm going to spend a lot of time in PRs / Issues / Redis-dev mailing list.
Great to hear we'll see more of you around! :)
I'll write less code, but I always lament that the project gets too big, so for a few months is not going to be a drama :-)
Well, I believe this to be more of a process: as your vision and guidelines "sink into" the community (this is the point I think is causing the most grief: we don't quite know yet how you'd like Redis to evolve), other devs will be able to respond on the "moral" aspects more frequently and you won't need to haul the weight of public-relations-ish stuff alone.
The Baldguy
Dec 11, 2014, 2:08:19 PM12/11/14
to redi...@googlegroups.com
Fortunately for me, less so for you fine denizens of the list, I am on vacation the rest of the year so I have *plenty* of time to complain and comment, which I will start with in-line below. ;)
On Tuesday, December 9, 2014 4:34:54 PM UTC-6, Salvatore Sanfilippo wrote:
On Tue, Dec 9, 2014 at 8:33 PM, Matt Stancliff <mstan...@pivotal.io> wrote:
> I just like to complain a lot.
>
> I have no right to complain. Redis has been my full time job for over a year. Without Salvatore, I’d probably be eating dirt on a beach in Monaco because nobody else wanted to hire me (though, Monaco beach would be nicer than a winter in NYC).
That's not true, you contributed many things lately, so you have right
to complain ;-)
> Currently, the Redis development process feels hostile to outside developers and outside contributions. (even my small fixes get ignored, and I’m _amazing_: https://github.com/antirez/redis/issues?q=is%3Aopen+author%3Amattsta)
>
> Developers often “test the waters” by submitting a small patch. If the patch is acted on, they’ll try to contribute more fixes or features. If they get ignored, they go away and never come back. When patches get ignored for years, they don’t bother to get more involved with the project. So, Redis has a very small pool of full time developers to help because Redis doesn’t *want* help. Redis wants to be left alone. (yes, this is attempted to be fixed below.)
>
> So, historically, there aren’t really “Redis developers” in the open source community because most non-bugfix contributions get ignored or rejected.
$ git log | grep Noordhuis | wc -l
518
$ git log | grep Stancliff | wc -l
89
Does not look like an inexpugnable castle to me, but yes, there is a
barrier to entry that is large due to project focus, quality
requirements, and, lack of time, in some random order.
When I look at Redis, I see a project that from a technical standpoint has a low barrier to entry. The code is well written and easy to follow, is relatively isolated, and not that large in terms of quantity and mental scope.
I *think* what Matt was commenting on is the perception of how useful the list is for discussing the development of Redis. Of your list above only item 4, as I recall, was a discussion on the list. That isn't saying they weren't good things, just not a product of on-list discussion and thus not relevant to what I think Matt was referring to.
Right or wrong as long as a mailing list/forum includes as it's context the development of the software people will expect to see and contribute to a discussion of possible changes whether they be major code refactoring, feature additions, or critical bugs or gaps. I agree with Matt when he says we don't really see that here; and that it is frustrating. Without the transparency there is a sense of "meh, Salvatore will pop in after a bit and say no" which does stifle discussion to a point.
> A good example is Bill’s great post about the Redis process itself from after the dev meeting in October with no replies (I blame myself for that too):
> "Redis Improvement & Community Discussion"
> https://groups.google.com/forum/#!topic/redis-db/cUYrsp8Q7I4
>
> Bill is there practically begging to help, but… no go. no va. imbranato.
I'm ok with many Sentinel changes he proposes, but I found not time to
reply unfortunately.
Methinks you're thinking of my earlier Sentinel emails and posts on Github here, while Matt is referring to my post about the very subject of this email. ;)
Sorry Bill, the changes you propose to Sentinel are interesting for
me, and I remember you reported a pathological condition where a
sentinel is monitoring too many masters and/or when there are too many
sentinels, were you observed a delay. I think about it from time to
time, but then there is something else, and is gone from my head.
OK, so on the Sentinel subject ...personally, while I see many of these changes as very important for Sentinel users it isn't that you haven't worked on them that bothers me. I've been in this field far to long for that. What does concern/bother me is the lack of visibility, and I think this is something required in a BDFL model (yes, like Josiah I am from the Python history and actually don't have a problem with that model or term). People trust a BDFL/lead more when said person isn't like "a black box" where words go into it and eventually code comes out - code which may or may not be related to the words which went in, but rather more transparent about what is being worked on. To carry the point into this specific case:
My C knowledge is as rusty as a WWII floating supply depot (circa Linux Kernel 2.0 work). I could probably make many of those changes but I'm not because I know they'd need fixed before being suitable for merging. I'd rather not see it done twice. Now, I do have at least some C developers at my disposal which could be tasked with it. However, given the likelihood of any work done there being reviewed let alone included combined with not knowing if it was being worked on it would not be the best use of their, and my, time. The sever lag between PR and, as you say even a comment on it, is a *huge* barrier.
>> Also a lot of this depends on the story of the project. Core teams
>> often are a result of the original project leader to step back (this
>> happened both in the Tcl and Node.js case), or, the project entering a
>> larger software institution of some kind.
>
> There’s also a middle ground where outside contributions are discussed, refined, and compromised on until they fit the project.
> See things like:
> - https://www.python.org/dev/peps/pep-0001/
> - http://www.erlang.org/eeps/eep-0002.html
> - https://github.com/erlang/otp/wiki/EEP-light
I think this would not help with our small project, or at least, not
as much as a redis-dev mailing list with is lower traffic. We can just
talk there.
However lately I tried multiple times to get feedbacks here before
implementing stuff, but got very little replies.
We can't fix a lack of resources problem by writing RFCs. We would end
writing most of them ourselves. It's a lot better if we just write
messages in the mailing list.
Actually, a good RFC process does precisely solve a significant chunk of resource limitations. Consider again my scenario - I'm not going to do much C contributions because I think that would hinder my overall contributions. However, I do still have available the mindset of operations despite being more of an engineer and architect these days. Thus, RFCs allow us to use people such as myself who, while not contributing code, can contribute analysis, technical document structure, and refine and condense the discussion. A good RFC provides a common base of understanding which allows more people to think about the details as well as the bigger picture. They also serve to provide transparency of process as well as explain reasons why certain decisions were or were not made. Further, a RFC can provide an entry point for more people.
As it is now if someone wanted had the time and fundamentals to contribute code to Redis, where should they start? Where would their expertise fit in? What is the specific direction? RFCs provide that. If we had a set of (growing) RFCs for things Redis should do, the way it should, etc. then those who have expertise in doing those can see that and know their contributions are potentially desirable and would be in line with the direction of Redis. It lowers the barrier to getting started, while generally improving the quality of PRs that come your way. This is one method to "scale" your pull request time - if the PR is something that follows a well defined specification it will be easier for it to be reviewed. For example it can be reviewed for conformance to the RFC by one person, QA review by another, reviewed for code style by yet another. Note that not all of these have to, or should be, the same person.
The bottom line is that RFCs are a way people can contribute to the progress of Redis without needing to commit to the repo. That means more resources are available.
For example, I'd be happy (no, seriously, I also enjoy documenting stuff - hey! stop laughing ;) ) to write up an RFC for my (many) requests for changes to Sentinel in a single (if appropriate) RFC, following it with discussion about the ways and means for it. Once accepted it would be a good blueprint for *someone* to implement them knowing they standard a good chance of acceptance assuming the code is quality and does what it is supposed to. Even during discussion it would be good enough to possibly encourage someone to work out some proof of concept code which could inform and improve the end result.
>> When the benevolent dictator reviews a patch and let
>> it enter the project, there is a transfer of accountability from the
>> original committer to the project leader: she or he will stay late at
>> night to fix bugs in the contributed code if there is some critical
>> stuff.
>
> See, that’s the problem there—the “project leader” shouldn’t be the only person responsible for fixing all code in the entire project forever.
>
> Look at Linux. Is Linus out there *fixing* code (or even writing code) any more? He’s just there to yell at people, merge, set policy, and make releases, but he still runs everything.
>
> It’s arguable whether one person can do all code development and complete community management for a very popular open source project.
Do you think it is a comparable setup? This model scales as there are
many people involved into a project. The many people involved with
Linux, are, 90% (or more?) payed by companies.
Let's see how cool is OSS in practice: I write code for Redis because
I'm payed. You write code for Redis because you are payed, you were a
Redis user before, but Pivotal was not sponsoring your work, so no
much from you, and know we see you are capable of great contributions,
like quicklists. Pieter wrote a lot of code for Redis when he was
payed to do so, and so forth.
Most of the OSS development you see in system software like MySQL,
Linux kernel, and so forth, is form developers that pay the bill with
it.
That is only the recent history.
So IMHO Redis can't be handled like Linux because it is not a big
enough project, there are not enough companies interested in
sponsoring developers,
Based on my personal experience with the Linux kernel back before it was The New Hawtness, I must disagree. Indeed I see Redis as sitting precisely at a crucial turning point the Kernel hit many, many years ago. Linux ran into this same problem. It was salved for him, and frankly I've thought about it in Redis' context - mainly because it moved Linux forward *dramatically*. It was the development of maintained patchsets - collections of disparate patches (features, fixes, cosmetics, refactors, etc.) that were integrated into one set and maintained publicly by a single person. This practice still continues to this day, but for those who weren't around when it started it may be difficult to see how big of a deal it was.
First, it offloaded a lot of work from Linus. Eventually these became the primary source for Linux to review - they acted as a filter for patches (keep in mind Git didn't exist we are talking literal diff files here). Because they were known to integrate together and because they had a defined set of capabilities they were in essence a form of RFC for the kernel. They also bootstrapped a group of core kernel developers who became "trusted", and provided an easier means to entry for would-be members. I am of the opinion that this singular act, though not intended in the result it produced, is one of the most under-appreciated events in Linux's history.
Since the developer conference I've been thinking about it in context of Redis. Personally I think we have the tooling and non-human resources required to not need it today, but I may be wrong. I've considered, and have shared this with a few people, putting together a collection of patches into a "distribution" of Redis that people can use, test, play with just as was done with the Linux kernel. A significant part of me doesn't like the idea and I've only recently understood why. My concern is that it would basically become a fork of Redis. And yes, you encourage that and have been quite open about that being O.K., I'd rather not have a fork of Redis. Maybe that's just a bit of sentimentalism on my part, but I don't think a fork would be great for the community.
What I think would prevent that fork from happening would be:
1. An RFC style process
2. A published consistent process/policy for pull requests
3. Timely review of pull requests with discussion as needed
4. A reasonable flow of changes being merged
and si simple enough that people interested in
new developments can more easily fork in-house and do what they need.
And never see the light of day for the community.
So as long I write a big percentage of the code, I'm going to be able
to fix a big percentage of the code.
But in the "model" you described, most of the code isn't being written by you - it is being written by companies who write it and keep it in house, or by people who make it public but it never gets merged.
>> In the core team is different but not so much if the core team is
>> composed of contributors *constantly working to the project*. If a
>> patch is voted by the majority as "ok", there are at least 50%+1 that
>> will be willing to fix problems.
>
> If a core team must vote to override other team members on issues, the core team has already failed.
Not sure about this. A core team votes by majority I guess to get new
things inside. If there are 3/5 votes ok, there are 3 developers that
endorse a given change and will work on it.
Any group can define the requirements for something to "pass a vote". You could say 4/5ths are required for example. The precise numbers ar eless imprtant than a known and consistent process.
> If we can’t find anybody else to “share the vision,” then where is the flaw? If nobody agrees with what I say is right, am I actually correct?
Before to agree, there is the initial problem of having the 5 core
developers.
And before that is the problem of people knowing what the hurdles are, what can be reasonably expected to get in, the specific direction, what their changes must do, etc.. The vision comes before the code. Always.
Consider it from a non-software perspective. There are auto mechanics. You can take your car to them and they'll fix it. They don't design cars. There are "tuners" - people who take a car and make it "better" (faster, quicker, more nimble in the turns, etc.). They don't design cars, they take the car and improve it. Then there are the people who design the car. Now imagine the car designer was taking improvements made by the tuners.
The key thing in all of the above is the designers of the car exist before the rest. So far that has pretty much been you. That isn't scaling. Since we can't bolt on more brains or hours in the day for you, it can only scale by going horizontal.
Once you have N people working constantly to the core, you
can scale and say, what about doing this core team stuff?
So the problem is that there are no contrib. But actually it is slowly
happening now, and there are a bit more contribs and they are around
for some time now. And I don't think I'm less an asshole I used to be,
it's just that the project is more mainstream, there are payed people
working at it, and new enthusiasts (that if not payed soon or later
stop contributing, likely).
I know of several would would be contributors if it were not for the hurdles we've been discussing.
> Current Redis contributors are just a small pool of crazy people (in a good way) who enjoy the work even when they get ignored for a while. More developers could show up, but it requires a community cultivation and openness.
>
> Sometimes contributing to Redis feels like being a 5 year old jumping around saying “look at me! look over here! look what I can do!” just to get the attention of the authority figure who can shine the light of merging and feedback upon you.
I'm sorry I've no more time than I've. When I see contributions which
I conceptually accept, are well implemented, and complete, I try to
just hit "merge".
Which is well and good but when nobody knows what to expect or what the objective criteria are it makes your job harder, and decreases contributions.
For example your work on quicklist is pending because there is a
blocking thing about RDB format, which is otherwise a regression
compared to the past, and does not allow Redis Cluster to benefit from
quciklists enough (actually it makes migrating small lists much slower
than before).
Your work on Geocoding is pending because it needs to get a
refactoring to avoid duplicating a lot of code which is common with
sorted sets.
I've not looked at those PRs, but have you commented on them to that effect? Those seemingly small things make a *huge* difference in perception of entry barriers. Also, a good set of RFCs describing, for example, the quicklist and RDB changes would have enabled (though not guaranteed) that Matt's changes would not have prevented an RDB change, or vice versa.
The two above are not ignored, but still not ready to be merged.
Sometimes it's me. For example I've still to read the JSON thing, but
I de-prioritized it because the initial feeling makes me thing it's
better to look at other patches than JSON. Maybe I'll read it and
think, wow, that's what we need, but it is unlikely. So I sent time
looking at quicklists, geocoding, IPv6 fixes, and other stuff you
contributed.
Again RFCs do a lot of that work for you and, when combined with updates (even as simple as "pending XX") provide the transparency to give more comfort in the progress of the project. They also serve as a task organizer for you as well since you can look at things which were waiting on the task you just finished and pick it up. It helps the "ooo shiny!" aspect we have to remain (partially) under control. ;)
>> If there are other people that say "Let's add this", I
>> want to see them to fix stuff when shit happens, otherwise it's
>> unfair.
>
> Core contributors/team members are responsible for everything else too (or their own specializations), not just adding their own pet projects. :)
>
> Specialization is a good point too (thanks, me, for bringing it up). Right now, antirez is responsible for: creating new features (allowing new commands, making improvements to existing infrastructure), fixing existing features (data structures, networking, event loop, config parsing, dependency management/upgrades, …), making releases, merging contributions, creating custom distributed systems, creating custom file formats, writing documentation, merging documentation, running the website, testing everything….
>
> At any given time, only one thing is being worked on, but other things may also need improvements too.
>
> The best I can do right now is try to pre-approve everything and try to guess what antirez wants for patches/improvements/code quality review.
I can't see how this is the case. Again, quicklists: it was a clearly
needed feature, you implemented this, I'm willing to merge.
This is parallelization. Note that you did not had to ask me if this
was going to get merged.
> If the filter is also the main developer, the community will always fall behind.
What is the other viable setup? Here we are talking mostly of
contrib-and-run-away contributors. If patches are not reviewed, is a
big problem in a few weeks / months.
I think this video about how movements get started applies here: https://www.youtube.com/watch?v=lbaemWIljeQ
As I see it, right now Matt is the second guy - the first follower. While I disagree with how you describe "core contributor", the way you use it means he would be your first core contributor (yet isn't yet). Core contributors don't necessarily quite contributing because they go to a different company. They most often stop because they don't, or feel they don't, have something they can contribute or that their contributions won't be accepted.
I think you've got stuck in the idea that a core contributor must be as committed as the BDFL. Both Python and the Linux Kernel are BDFL mechanisms but the "core contributors" are not necessarily as committed as Guido or Linus. The "core contributor", as I see it, doesn't need permission to commit to the repo for bugfixes or approved features - they've demonstrated they can be trusted. Whether they do that by maintaining a branch which the BDFL pulls from when features or changes are deemed ready for inclusion (not always the same as "for production").
This removes the single-approver problem. Instead the BDFL holds the power on the direction what features will or will not be added, etc. as well as what goes into a given release (until such a time as a release manager emerged - if it ever happened). Potentially one could put prioritizing in there too but I think that is a false governor. For example, if Matt were to submit a patch that implemented pulling INFO data for nodes directly from Sentinel, it would be unwise to delay reviewing it and merging it if acceptable just because a different feature had "priority".
>> Redis I'm currently the main contributor of code, so this is not possible.
>
> Well, you’re the main code contributor, release maintainer, feature planner, maintenance do-er, and a dozen other things. :)
>
> If I’m driving on a long trip and keep saying “I’m the main driver, nobody else can drive,” I can’t feel bad about not having any help while driving.
Ok, basically, I lost you a bunch of lines before this point. I'm here
replying but I don't understand what you mean anymore.
Let's be clear: do you think we have a possible, theoretically large
(3, 4, 5?), pull of Redis core developers, constant in time, that
provide quality work, so we can run the project as a distributed
effort?
Depending on what "core contributor" *functionally* means, yes. But as we've been saying, not everything has to be done by "core contributor". Consider looking at PRs and bug reports. One shouldn't have to be a "core (code) contributor" to filter out duplicates, not-a-bug, wont-fix, etc. or to classify reports and PRs into groups (ie. "sentinel", "cluster", "replication", "documentation", , "operational", "RFC-NNNN", etc.) where you could pick a group and do a deep dive into it. One way to make your life easier when looking at PRs and issues is to look at ones that are in the same context. You have less spinning up between tasks.
When I'm quite a few weeks, nothing happens.
This happens because the project has no clear direction, and there is no reasonable estimate as to how work would be received.
If you remove your latest
contributions, it is very rare to see *anything* of big. So I don't
understand what we are talking about.
I'm driving because I'm mostly alone. Now you are contributing, and I
try to merge your code with some priority compared to any other
contributor, of course. But also your hacking time must be aligned
with the project needs.
I mentioned the priority thing earlier but "aligned with the project needs" needs addressed. What are they? Who decides them? Core contributors aren't employees of the project, and projects that have considered them as such usually run into major issues.
> Very true. People won’t contribute big fixes and improvements if their small fixes get ignored. Contributors want their (useful) work to be added, not sit around for years untouched.
I'm trying to react faster, but this does not mean to merge stuff
which is not ok IMHO.
Nobody is suggesting you merge poor code.
>> When a PR violates the project vision or code quality or whatever, I should
>> reply ASAP with a short note and close it.
>
> Note the use of terms there: “project vision,” “code quality,” “whatever.” — Those are personal issues that can’t be delegated. If the project is 100% What-antirez-Wants, then, provably, other people _can’t_ help too much.
I believe it is sensible that I approve new features, honestly. I
understand you don't like it, but I really believe there is value in
having a small project.
I think you two are talking about different things here. The terms are vague - and that is the problem. Guido bears the responsibility at the larger scale but he can only do it because things are tightly defined and a structure is visible and consistent or Python. The lack of such things makes it appear arbitrary and that, I think, is what Matt was referring to here.
You should really release your fork with loadable modules, JSON, Geo
API as-it-is, and everything else. Maybe a different model is
possible.
However I don't want to have a project that grows into directions I
think are not good, I'm more happy with a slower growth.
> But, today there are over 2 million developers who use Redis at over 100,000 companies. Does having one person be the gatekeeper for the needs of _everybody_ still work?
Depends on what you want the output to be. A small reliable software?
Yes, can work. A bigger software that handles more use cases? Probably
not.
Yet it happens a lot. It can't happen if it is one guy deciding everything right down to fixing spelling in comments or documentation, and especially if nobody knows what is going on.
However I don't see the issue black or white like you. Give me more
long-term developers, and I can put them to do good work.
Some of the best work in many successful projects comes from the "short term developer". Why should, for example, an SSL expert be pushed aside because she wants to come in and add solid, expertly written SSL support and then move on?
I've a huge list of things that are priorities... and there is an
issue open. Those stuff are obvious merge-them if implemented
correctly, and I would sure help to point in the right direction if
the first implementation is not perfect.
But you say all the people in queue to write those parts are away
doing something else because my way of handling the project.
Many, yes. I know it may sound harsh, but it is a fact. Having a list of priorities for tasks you personally want to do is great. But why reject/ignore valid contributions from people who had a different priority, implemented it, and submitted it just because you would not put your time into doing it?
> We can’t have outside contributors if the guideline for getting things added is “What antirez wants” because: valid fixes and contributions will be rejected if they don’t fit the unspoken "design-in-antirez-head" _and_ it essentially turns all contributors into personal assistants to Dear Leader instead of people who have their own goals and get their own rewards.
get-random-stuff-inside is way worse, IMHO.
That is a false dichotomy. They key thing here, Salvatore, is the part about it being unclear what people can do that will be accepted; and that contributors are not employees of the BDFL.
"Fear makes the wolf bigger than he is."
— German proverb
This is highly appropriate to this thread. ;)
Salvatore Sanfilippo
Dec 11, 2014, 6:06:24 PM12/11/14
to Redis DB
On Thu, Dec 11, 2014 at 8:08 PM, The Baldguy <ucn...@gmail.com> wrote:
> That is a false dichotomy. They key thing here, Salvatore, is the part about it being unclear what people can do that will be accepted; and that contributors are not employees of the BDFL.
Hello Bill, I'm going to reply to this single line just to move
> That is a false dichotomy. They key thing here, Salvatore, is the part about it being unclear what people can do that will be accepted; and that contributors are not employees of the BDFL.
forward, but I believe I understood the sense of your email, and it
makes sense in the essence. How to implement the fixes to the problems
you describe is not easy but I guess, we are interested in the goal
after all.
1) RFCs: Maybe we'll get them, but for now in order to speedup the
process, why a redis-dev ML can't be a good substitute after all?
Especially if we also have something in the web site where it is clear
what is the good progression for contribs. Trivial fix? Just send a PR
and we'll reply in a timely fashion. Big contrib / change? Please open
a thread in the redis-dev ML and we'll discuss it.
2) Merging PR, replying to issues fast: You are right this is a key
point. I'm committed into this, and move a lot of my time from coding
to merging. However now there is a big pile of issues accumulated. As
I scan them, I need to also try to handle the new ones. The idea is
that I continue to handle the backlog, but at the same time, I also
reply ASAP to the new ones. There will be a moment where I'll be
slowed down from the history, but it's 200 PRs, It's going to be over
in a few weeks.
3) How to turn redis-dev into something good for people? I can simply
reply to each thread in a clear way, expose my point of view, get the
feedbacks, reply to the feedbacks, and at the end say: ok, this looks
good, I would change maybe XYZ in the implementation or give some
suggestion, please do the work, if it's in line to what we discussed,
it's going to be merged. However there will be cases where I'll be
against the feature, I'll argument it, people will disagree, I'll be
still not convinced, and the feature will not enter the code. I guess
it is impossible to make people every time happy, but it is possible
to give feedbacks in the negative case explaining exactly why
something can't be done. Makes sense in your opinion>?
4) " For example, if Matt were to submit a patch that implemented
pulling INFO data for nodes directly from Sentinel, it would be unwise
to delay reviewing it and merging it if acceptable just because a
different feature had "priority".". About this, there was never really
to delay reviewing it and merging it if acceptable just because a
a "priority" reasoning here. It was like, OMG, it's full of issues/PRs
out there, I'm going to don't care and work. I understand this was not
the right approach btw. However I think that *so far* this helped
Redis, at least until a certain point. It is true that this way of
doing things don't scale and must be switched ASAP now. And it is true
that it stopped to scale some time ago at this point.
5) "It can't happen if it is one guy deciding everything right down to
fixing spelling in comments or documentation". I can avoid so much
control on the code with the new branches setup, however I can't
figure out what would be a substitute of reading the code of new
features in order to check quality before merging.
So far what changed is: we have redis-dev, I'm full time merging PRs,
we have a different development/branches model that allows for lower
barrier to entry to code that could have more bugs than needed.
I believe this could do a difference. I understand that more may be
needed in the future, but maybe it is wise to take it a bit more
incrementally, especially from the point of view of QA.
Sorry for the short email compared to yours, I hope I provided some
reply/feedback.
Thanks,
Salvatore Sanfilippo
Dec 11, 2014, 6:10:10 PM12/11/14
to Redis DB
p.s. a word about a possible fork / collection-of-patches.
I'm open about it just because I really believe in the OSS fundamental
meaning: do what you want with the code freely.
However this is my prediction of what would happen. The fork would be:
1) With more features, sometimes good features. So in general more
adapt to serve different needs.
2) Less stable.
3) Incrementally would have a bigger barrier to entry because of
declining coding standards and amount of code / complexity.
4) Probably it would get more people involved that in the long run
would fix 2+3.
If the fork does not happen what we can do is to migrate the project
to stage "4" without going into 2+3.
I'm open about it just because I really believe in the OSS fundamental
meaning: do what you want with the code freely.
However this is my prediction of what would happen. The fork would be:
1) With more features, sometimes good features. So in general more
adapt to serve different needs.
2) Less stable.
3) Incrementally would have a bigger barrier to entry because of
declining coding standards and amount of code / complexity.
4) Probably it would get more people involved that in the long run
would fix 2+3.
If the fork does not happen what we can do is to migrate the project
to stage "4" without going into 2+3.
Bill Anderson
Dec 11, 2014, 7:47:17 PM12/11/14
to redi...@googlegroups.com
> On Dec 11, 2014, at 17:09, Salvatore Sanfilippo <ant...@gmail.com> wrote:
>
> p.s. a word about a possible fork / collection-of-patches.
>
> I'm open about it just because I really believe in the OSS fundamental
> meaning: do what you want with the code freely.
> However this is my prediction of what would happen. The fork would be:
>
> 1) With more features, sometimes good features. So in general more
> adapt to serve different needs.
> 2) Less stable.
> 3) Incrementally would have a bigger barrier to entry because of
> declining coding standards and amount of code / complexity.
Given whom _I_ would be willing to work with to do such a thing I doubt these problems would be even likely. Ideally we would be trying to maintain a reasonable ability to re-integrate with Redis core, a model I've seen work well.
Maybe if I can get over my current "don't fork it" mentality it might happen. I just have to let my Vulcan side have more sway. ;)
At a minimum I'd be happy to start collecting patches and/or PR links along with some docs on what and why for each. After the new year I can set up some testing resources for such a thing as I've already got buy-in for sponsoring such efforts at work. Maybe I'll even start with a git fork to start applying patches for testing purposes. Hopefully something like that could decrease your workload.
Salvatore Sanfilippo
Dec 12, 2014, 3:12:43 AM12/12/14
to Redis DB
On Fri, Dec 12, 2014 at 1:47 AM, Bill Anderson <ucn...@gmail.com> wrote:
> I've not really seen this happen. Consider even the simple case of starting with a collection of bug fixes. By iterating quickly the bug fixes can be tested and if needs be fixed faster. I don't agree that just because it wouldn't be you as the arbiter that it would inherently have declining standards or be overly complex. A possibility to be sure, but not near a certainty.
No declining standards means that people doing this work would put as
> I've not really seen this happen. Consider even the simple case of starting with a collection of bug fixes. By iterating quickly the bug fixes can be tested and if needs be fixed faster. I don't agree that just because it wouldn't be you as the arbiter that it would inherently have declining standards or be overly complex. A possibility to be sure, but not near a certainty.
much effort as much I put when merging patches: read the patch line by
line, test it manually, and try to understand what side effects and
semantical changes would be, and finally, that person should be as
conservative as I'm at the end of the process. So, such a person,
would not be faster than me doing this. TLDR: This could happen
already, see for example the code review effort that Sun He and
Jan-Erik operated in the quicklist code.
If instead the barrier to entry of the patch set would be lower, less
strict reviews would be operated, but you mostly do testing of the
patch set to check everything appears to work, what you end with is a
viable alternative source tree that may be less stable (since you find
most bugs by having thousands of deployments or reading the source
code with care, or writing it in a way that avoid bugs beforehand).
Because that tree would have more features and fixes, people would use
it. Somebody get burnt but because it's an unofficial tree nobody will
complain. Alternatively we may get bug reports here for bugs
introduced in such a tree ;-) But in the end, those bugs people
discover by using a less stable versions of Redis could be merged into
the original PRs and improve "official Redis" in the long run.
Because of all the above, when two days ago I switched to "let's merge
stuff more liberally" mode, I changed the development model as well,
and only bugs that don't require code changes are merged into testing
/ stable. Everything else lives in the "unstable" branch, which is
still the default @ github, so gets exposure, and gets time in order
for possible bugs to be discovered. The "set of patches" that people
are willing to try, will be our "testing", but there is a clear
contract with the user already in the branch name.
TLDR: I think that:
1) You can't have good code quality without PRs code-level reviews,
and expert people that understand what a seemingly harmless line of
code will result in a week in a production environment.
2) Most bugs are avoided more effectively by writing code or reviewing
code with a lot of care (the former is more effective, otherwise the
latter takes a comparable time).
3) There is 1% of bugs that happen with non-obvious state of the
program, that are discovered only with a larger set of users testing
the product, so any setup that merges more liberally, need something
people can try in real-world-ish conditions.
4) Having more tests helps a lot, however so far I'm the only one that
focuses on extending the test suite... it's the boring part nobody
wants to do. With the exception of recent versions of Matt that went
actually the extra mile. The quicklist code is an exercise in shipping
reliable code, and it shows up.
Hopefully with a better attitude and development model I'll be able to
encourage not just new contributions, but also more reviews and
testing.
Cheers,
Salvatore Sanfilippo
Dec 12, 2014, 3:19:49 AM12/12/14
to Redis DB
Btw in your reply it sounds like that in my original message I was
saying: you can't get stable things because I'm better than you. This
is the contrary of what I think, a lot of the conservative parts of
Redis are as they are because I think programmers should be a bit more
humble and say: writing system software is hard, and I can't pretend
that without a lot of care, conservative development and/or much
testing, reviews, and such, I'm going to ship high quality code. What
puzzles me a bit about all this discussion is that nobody said, yes
Salvatore, you are an asshole and PRs are not handled forever, you
have to read every single line of what gets into, contributors are
turned down, BUT Redis runs for years in productions, is regarded as
one of the most stable C projects out there, and even if ran by a lot
of people with a big number of total instances actual stability bugs
are rare. Not because I want to hear "Hey Bravo!" but because is an
obvious tradeoff we need to capture if we want to change the model and
still ship quality software.
saying: you can't get stable things because I'm better than you. This
is the contrary of what I think, a lot of the conservative parts of
Redis are as they are because I think programmers should be a bit more
humble and say: writing system software is hard, and I can't pretend
that without a lot of care, conservative development and/or much
testing, reviews, and such, I'm going to ship high quality code. What
puzzles me a bit about all this discussion is that nobody said, yes
Salvatore, you are an asshole and PRs are not handled forever, you
have to read every single line of what gets into, contributors are
turned down, BUT Redis runs for years in productions, is regarded as
one of the most stable C projects out there, and even if ran by a lot
of people with a big number of total instances actual stability bugs
are rare. Not because I want to hear "Hey Bravo!" but because is an
obvious tradeoff we need to capture if we want to change the model and
still ship quality software.
Thomas Love
Dec 12, 2014, 5:10:59 AM12/12/14
to redi...@googlegroups.com
On Friday, 12 December 2014 10:19:49 UTC+2, Salvatore Sanfilippo wrote:
Btw in your reply it sounds like that in my original message I was
saying: you can't get stable things because I'm better than you. This
is the contrary of what I think, a lot of the conservative parts of
Redis are as they are because I think programmers should be a bit more
humble and say: writing system software is hard, and I can't pretend
that without a lot of care, conservative development and/or much
testing, reviews, and such, I'm going to ship high quality code. What
puzzles me a bit about all this discussion is that nobody said, yes
Salvatore, you are an asshole and PRs are not handled forever, you
have to read every single line of what gets into, contributors are
turned down, BUT Redis runs for years in productions, is regarded as
one of the most stable C projects out there, and even if ran by a lot
of people with a big number of total instances actual stability bugs
are rare. Not because I want to hear "Hey Bravo!" but because is an
obvious tradeoff we need to capture if we want to change the model and
still ship quality software.
We're here, but probably under-represented on this thread. I got started with Redis in large part precisely because I sensed its conservative philosophy was a good fit for me. It follows that it doesn't actually matter too much to me, now if Redis were to become a ten-million-line monster. I don't run anything approaching the latest version in production. To the extent that Redis bloats (in a very subjective way!) I would expect that gap to widen over time. But that's my problem. It's future users I think you should worry about -- those who aren't here applauding conservative approaches because they don't know Redis exists.
Thomas
Bill Anderson
Dec 12, 2014, 9:39:21 AM12/12/14
to redi...@googlegroups.com
> On Dec 12, 2014, at 02:19, Salvatore Sanfilippo <ant...@gmail.com> wrote:
>
> What
> puzzles me a bit about all this discussion is that nobody said, yes
> Salvatore, you are an asshole and PRs are not handled forever, you
> have to read every single line of what gets into, contributors are
> turned down, BUT Redis runs for years in productions, is regarded as
> one of the most stable C projects out there, and even if ran by a lot
> of people with a big number of total instances actual stability bugs
> are rare.
That said, I've always held and espoused the notion that it is the simplicity and near single-mindedness of Redis which provides its core of stability. You have stated in the past why you went evented and focused on a single thread of primary operation instead of the traditional multi-threaded behemoth and I've agreed with it entirely. Even if all you did was convert Redis to a multithreaded system you'd have much more instability.
Redis also focuses on the core key-value proposition, following the UNIX a philosophy of "do one thing, do it well". This is also one of the core causes, IMO, of the stability of Redis. I think my comments on handling auth separately reflect that.
Overall I've never seen this community be the kind to call one another names, which is a nice alternative to other, more complex software, based ones.
> Not because I want to hear "Hey Bravo!" but because is an
> obvious tradeoff we need to capture if we want to change the model and
> still ship quality software.
Salvatore Sanfilippo
Dec 12, 2014, 10:13:26 AM12/12/14
to Redis DB
On Fri, Dec 12, 2014 at 11:10 AM, Thomas Love <tom...@gmail.com> wrote:
> We're here, but probably under-represented on this thread. I got started
> with Redis in large part precisely because I sensed its conservative
> philosophy was a good fit for me. It follows that it doesn't actually matter
> too much to me, now if Redis were to become a ten-million-line monster. I
> don't run anything approaching the latest version in production. To the
> extent that Redis bloats (in a very subjective way!) I would expect that gap
> to widen over time. But that's my problem. It's future users I think you
> should worry about -- those who aren't here applauding conservative
> approaches because they don't know Redis exists.
Thanks Thomas, it will be my priority to avoid over-engineering the
> We're here, but probably under-represented on this thread. I got started
> with Redis in large part precisely because I sensed its conservative
> philosophy was a good fit for me. It follows that it doesn't actually matter
> too much to me, now if Redis were to become a ten-million-line monster. I
> don't run anything approaching the latest version in production. To the
> extent that Redis bloats (in a very subjective way!) I would expect that gap
> to widen over time. But that's my problem. It's future users I think you
> should worry about -- those who aren't here applauding conservative
> approaches because they don't know Redis exists.
project. However I've the feeling most of the requests were more about
the process than the set of features to actually merge.
What I'm seeing is that a lot of the stuff I'm merging while scanning
the past PRs are about things that I would have fixed or implemented
myself, which is encouraging. Of course there is always the ability to
say no thanks, in a clear way and explaining why. This happened
yesterday for example, with a PR about a potentially interesting mix
between Pub/Sub and blocking lists semantics. However this part of
Redis is going to be externalized to the new project, Disque.
About future users, I care more about past users ;-)
Salvatore Sanfilippo
Dec 12, 2014, 10:19:19 AM12/12/14
to Redis DB
On Fri, Dec 12, 2014 at 3:39 PM, Bill Anderson <ucn...@gmail.com> wrote:
> First I think for the most part the people who would (or do) call you an asshole are outside the community. I don't want to overstep my blinds but I'd like to think the majority of us here understand the volume and intensity of the work - and is a major factor in why we want to help.
Thanks, I was just stretching a bit the concept, but I can understand
> First I think for the most part the people who would (or do) call you an asshole are outside the community. I don't want to overstep my blinds but I'd like to think the majority of us here understand the volume and intensity of the work - and is a major factor in why we want to help.
that a very conservative POV may look like I come along as person you
better avoid to deal with.
> IMO as long as we keep the core of Redis simple, dedicated, and don't try to do become multithreaded we don't have to make such a trade.
are just additions with just "additional complexity", and there are
features with multiplicative complexity. It is important we understand
what is technically pluggable (even if Redis is monolithic) and what
is not and changes in a more drastic way the core itself. Sometimes it
is not easy to predict that, or counter intuitive. For example expires
on keys is a multiplicative complexity feature, while cluster is a
totally split layer.
Matt Stancliff
Dec 12, 2014, 11:45:25 AM12/12/14
to redi...@googlegroups.com
> On Dec 12, 2014, at 10:12, Salvatore Sanfilippo <ant...@gmail.com> wrote:
>
> Thanks Thomas, it will be my priority to avoid over-engineering the
> project. However I've the feeling most of the requests were more about
> the process than the set of features to actually merge.
As a reference, the documented process for Erlang issues:
- if your patch is a minor new feature or a bug-fix, you will get a confirmation or rejection within 1-3 working days (from someone with authority to approve/reject).
- if your patch is a _major_ new feature, allow up to one month to receive a detailed response.
(useful already-documented process and decision tree at https://github.com/erlang/otp/wiki/Submitting-patches — we *do not* need to re-invent anything for an expanded contribution processes, we can just use already-made, well-thought-out ones)
> About future users, I care more about past users ;-)
If the person controlling the project doesn’t want more growth, and the argument is “the community wants more growth,” then… i don’t know.
Redis wants to be a single-owner bodega, not Walmart or McDonalds or Amazon (though, I’m pretty sure Redis powers all three of those somehow).
I feel what the *community* wants is something for Redis like http://blog.llvm.org/2014/04/the-llvm-foundation.html. What Salvatore wants is to not give up control of his most popular project, which is completely reasonable. Redis is worth a lot and it’s not good to just give away control of everything. Chris can give away control of LLVM because a.) he’s already rich from Apple stock and b.) he has a pool of really smart, dedicated developers to work with who are distributed across universities and a dozen companies.
[minor points from previous messages]
>> So, historically, there aren’t really “Redis developers” in the open source community
> Does not look like an inexpugnable castle to me,
What I meant by “Redis developers:” if someone asks us about how the Redis development process works—how releases are scheduled, how code reviews work, how features are planned—the only answer *anybody* can give is: well, Salvatore just does it all and we trust him. That’s not much of a “community of developers.” :) Compare that with everybody in this thread who has talked about being part of other open source development communities and their process, decision making, etc.
(I also find it really amusing there are antirez-specific release scripts are in the public repo: https://github.com/antirez/redis/blob/unstable/utils/releasetools/02_upload_tarball.sh; it’s just part of “Redis == antirez")
> get-random-stuff-inside is way worse, IMHO.
As Bill mentioned earlier, a project can’t scale if one person is deciding everything from one character spelling fixes to implementing HLL all by themselves. :-\
I think a lot of the fear around “get-random-stuff” is actually fear we may “get-stuff-that-works-but-antirez-doesn’t-like.” At that point, there’s no solution to the “multiple core contributors” problem because nobody else can know the [sometimes changing and contradictory!] opinions of one person.
> Depends on what you want the output to be. A small reliable software?
> Yes, can work. A bigger software that handles more use cases? Probably
> not.
> However I don't want to have a project that grows into directions I
> think are not good, I'm more happy with a slower growth.
Same point as above. Small software = single owner = great software, but much slower development cycle, and with 2+ million users, it can’t grow to meet popular requests.
> think are not good, I'm more happy with a slower growth.
> I'm trying to react faster, but this does not mean to merge stuff which is not ok IMHO.
(quick minor concrete gripe example of my own: I added a function to format an IP address directly into an sds, but it was removed because you can also do the same thing with two function calls instead of one, so the behavior is redundant? https://github.com/antirez/redis/commit/bbf0736c4e472c5488fae6990de51417bc8a2e42 — It’s just a preference issue at that point, nothing is *actually* wrong. :) Also, bikesheds should only be painted blue because blue is obviously the best color.)
Earlier this year I wrote up an entire rubric of “actual error” versus “perceived error” because I kept getting confused if I was *actually* wrong or if someone just didn’t like what I did. See https://matt.sh/programming-errors
> Let's be clear: do you think we have a possible, theoretically large
> (3, 4, 5?), pull of Redis core developers, constant in time, that
> provide quality work, so we can run the project as a distributed
> effort?
As far as “distributed effort,” I’m proud that with just three or four people, we usually have 24-hour coverage of GitHub Issues with (knowledgeable and expert!) people responding in real time.
DRAMATIC CONCLUSION
Thanks to everyone for the constructive thread. It’s difficult for me to talk about these things because I feel like I’m “attacking” Salvatore, but that’s not what I want at all. We just have to talk about process sometimes. And, with one person in charge of everything, it will always feel like more of an “attack” because everything *is* the decision of one person — spreading around responsibility can also help diffuse the one-person-is-blocking-everything feeling as well. :)
The super core issue here is: Redis is used by 2+ million developers. Redis is Salvatore’s software. Salvatore is an AMAZING developer. Salvatore’s time is *BETTER* spent writing high performing, complex C code instead of managing issues minor “obviously correct” merge requests (spelling fixes, edge case bugs, things we can test and prove to be correct). I hate to see Salvatore spend a month writing emails and going through GitHub issues because *I* would rather have a Redis bloom filter. But, we end up arguing how to split Salvatore’s Redis Life Capacity into either 100% new code days/weeks/months or 100% email days/weeks.
It seems inefficient have one of the best-in-the-world-developers not developing. But, with no other source of authority for the project, we *also* can’t have contributions/features/fixes ignored for months or years.
I don’t have a final answer (am I the weakest link too?), but that’s what we’re working with. The only solution is trust, taking deep breaths and counting to 10 when you see code you don’t like at first glance (but isn’t actually wrong), and starting to say “it’s okay if all code isn’t the Athena to antirez’s Zeus." (look it up)
We are already making progress. So far this thread has gone from “RFC/EEP/PEP is a waste of time!” to “okay, sure, let’s do it.” Let’s keep an open mind as much as possible.
In the end, all our features and our requests will get added one way or another, even if people fork and recombine as necessary (or, i don’t know, maybe someone could create a modular system where people don’t need to ask and beg for new commands to be added… they could just use some sort of “relocatable object” format to extend Redis at runtime without any forking or patching).
After all, this is INTERNET. We don’t need permission to create new things (http://withouttheirpermission.com/). We create what we create and the world responds appropriately.
-Matt
Michel Martens
Dec 12, 2014, 2:18:28 PM12/12/14
to redi...@googlegroups.com
On Fri, Dec 12, 2014 at 4:45 PM, Matt Stancliff <mstan...@pivotal.io> wrote:
> That’s the main issue right there. The “wider community” wants features, growth, and more users. But, Redis is mostly “free code given to the world by Salvatore” in terms of features, planning, direction, etc.
>
> If the person controlling the project doesn’t want more growth, and the argument is “the community wants more growth,” then… i don’t know.
I don't really know what the wider community wants, but my guess is
> That’s the main issue right there. The “wider community” wants features, growth, and more users. But, Redis is mostly “free code given to the world by Salvatore” in terms of features, planning, direction, etc.
>
> If the person controlling the project doesn’t want more growth, and the argument is “the community wants more growth,” then… i don’t know.
that the majority of users don't want more features. In part it could
be a matter of taste: I like the conservative approach and for my use
case, Redis has been feature complete for a long time. My guess is
that most users rely on Redis for what it provides without asking for
more features.
But even if it were the case that the majority wants growth and new
features, I would still object to that because of my taste for small,
stable and feature complete tools. I'm not saying there's no room for
improvement, but to me, improvement rarely comes in the form of
features added, and has more to do with finding better ways to do
something, solving a problem with less code, fixing bugs, etc.
>> I'm trying to react faster, but this does not mean to merge stuff which is not ok IMHO.
>
> “not ok” is subjective and, just keep drilling on the same point, _we_ can’t help the project if the guidelines for the project are “please only do what antirez wants.” :)
>
> (quick minor concrete gripe example of my own: I added a function to format an IP address directly into an sds, but it was removed because you can also do the same thing with two function calls instead of one, so the behavior is redundant? https://github.com/antirez/redis/commit/bbf0736c4e472c5488fae6990de51417bc8a2e42 — It’s just a preference issue at that point, nothing is *actually* wrong. :) Also, bikesheds should only be painted blue because blue is obviously the best color.)
for a library like sds.c. From that perspective, even though it's a
subtlety, it's fair to say it should be removed. It is also impossible
to cover all cases with guidelines, so there will always be room for
scenarios like that where you feel you couldn't possible read the
designer's mind. But in this example, after learning how separation of
concerns is important for the project, the error probably won't happen
again. All this to say you shouldn't think about this as an error of
type D or E in your classification, because the reason can be as an
objective criterion. And of course, I hope you won't get mad when
encountering those cases, because what you do is very valuable.
> Earlier this year I wrote up an entire rubric of “actual error” versus “perceived error” because I kept getting confused if I was *actually* wrong or if someone just didn’t like what I did. See https://matt.sh/programming-errors
>
>> Let's be clear: do you think we have a possible, theoretically large
>> (3, 4, 5?), pull of Redis core developers, constant in time, that
>> provide quality work, so we can run the project as a distributed
>> effort?
>
> The lack of core developers is a self-reinforcing failure, right? We don’t have a large pool of “core developers” because nobody can contribute successfully unless they follow many unwritten guidelines. If people don’t follow the unwritten guidelines, it’s annoying to merge their code because it requires a lot of manual clean-up (so their code gets ignored/delayed/rejected). Because nobody knows all the guidelines, and their code gets delayed or rejected (due to not-following-procedure issues), nobody has enough experience to be a long-term meaningful contributor. Because we don’t have long-term contributors, we don’t have a pool of people to pick from... we don’t have people to pick from because we don’t have contributors….
that the contributing guidelines are ignored, even if they are
extremely brief. The first step, rarely followed: "Drop a message to
the Redis Google Group with a proposal of semantics/API".
> DRAMATIC CONCLUSION
>
> Thanks to everyone for the constructive thread. It’s difficult for me to talk about these things because I feel like I’m “attacking” Salvatore, but that’s not what I want at all. We just have to talk about process sometimes. And, with one person in charge of everything, it will always feel like more of an “attack” because everything *is* the decision of one person — spreading around responsibility can also help diffuse the one-person-is-blocking-everything feeling as well. :)
doesn't look like an attack to you :-)
This thread was very valuable and the result so far has been great,
with the new mailing list, the work on the pull requests and the new
proposals we've got so far.
Salvatore Sanfilippo
Dec 19, 2014, 5:03:49 AM12/19/14
to Redis DB
On Fri, Dec 12, 2014 at 8:17 PM, Michel Martens <sov...@gmail.com> wrote:
> This thread was very valuable and the result so far has been great,
> with the new mailing list, the work on the pull requests and the new
> proposals we've got so far.
Hello, thanks Matt, Michel, for writing your final thoughts. I feel a
> This thread was very valuable and the result so far has been great,
> with the new mailing list, the work on the pull requests and the new
> proposals we've got so far.
bit battery-drained from the POV of this thread, so I'm not going to
reply point by point (sorry, I read your messages with great
interest).
So, finally, what changes as a result of the community feedbacks?
1) Mailing lists. We have redis-dev and is already working apparently.
That's cool.
2) A new way to submit new big changes, Redis Change Proposals:
https://github.com/redis/redis-rcp
3) More care about PRs / Issues. My current schedule is to use two
full days every week the handle PRs / Issues.
4) A new development model. We have now a three branches split: stable
(called 2.8, 3.0, ...), testing, unstable. At least 4 times every year
we release testing as stable, and merge unstable into testing, and the
cycle continues. No longer backporting of things which are not fixes
between branches, so that we can merge new stuff that may break Redis
without issues.
The 4 times / year schedule is the *worst* case. I hope we can do much
better than that, ideally a new stable release every 2-3 months.
FYI, I'm now following this split in my schedule. Monday and Wednesday
I do only new developments, since I want to write still code... :-)
Tuesday and Thursday I only do PRs/Issues handling. Friday I split
between developments and writing documentation.
However whatever is the day, I'll try to give feedbacks ASAP at least
about new issues whenever possible. The same about posts into
redis-dev. Don't expect me to be responsive here btw, that's a needed
sacrifice.
Cheers,
Reply all
Reply to author
Forward
0 new messages