Policy for disputed PRs: discussion

[David Roe]

Hi all,

I'm writing about an issue that I think is causing substantial harm to the Sage community: the only current mechanism we have for resolving a disagreement is to call a vote on this email list.  There are certainly times where this is an appropriate response, and I think it's still reasonable for policy disagreements or major decisions, but I would like to create an alternative resolution process that doesn't require emailing a list with 2578 members.

The need for such a process is highlighted by several recent disputes; see #36694 and #35403 for example, but there are others.  The particular case I would like to address is where there is a general consensus, but one person (or a few people) disagree.

Here is a proposed policy, which I am happy to revise:

If there are at least twice as many developers in favor of a change as there are opposed (which may include the author of a PR), then any developer may set the PR to positive review and those opposed should not set it back, as long as both of the following conditions are satisfied:

* it has been at least one week since an initial objection was raised,

* all of the participants being counted in favor have commented on the PR since the initial objection.

Of course, consensus is preferable, and this policy would not relieve us all of the responsibility to make persuasive arguments in favor of our positions.  But I think we need a mechanism of this kind when consensus can't be reached.  Also note that an objector is welcome to attempt to bring others into the discussion on their side if they remain firmly opposed.

I hope that others can suggest improvements to the idea above, and remind everyone to keep the discussion positive and civil.  I plan to call a vote on whatever proposal comes out of our discussion.

David

[Matthias Koeppe]

Thanks, David, for opening this (overdue) discussion with your thoughtful post.

I would like to put it in a larger context. I'm sure most here would agree that we want our project to be trustworthy for current and future users, to be welcoming to new users and developers, and to maintain a kind, productive, and caring atmosphere within our community.

I would welcome amendments to our Reviewing Code (https://doc.sagemath.org/html/en/developer/review.html) to align it with these goals. Describing a standard conflict resolution mechanism along the lines that David proposed is certainly a necessary improvement. But more than that is needed; the principles of our Code of Conduct (https://github.com/sagemath/sage/blob/develop/CODE_OF_CONDUCT.md) provide some guidance.

Matthias

[Kwankyu Lee]

I agree that we need a policy for disputed PRs.

Such a policy should not operate in a way to condemn those raising objections to the PR since we want to keep kind, productive, caring atmosphere as Matthias put it. The policy should be clear and operate semi-automatically. So I suggest the following detail:

(1) When a PR becomes a disputed PR? A PR becomes a disputed PR when one reviewer adds "positive review" label, another reviewer removes it, and the author does not plan to work more on the PR according to the reviewer objections.

(2) How do we count approvers and disapprovers for a disputed PR: A reviewer becomes an approver (who is in favor of the PR) when he/she sets "Approve" in the github review system. A reviewer becomes a disapprover (who objects the PR) when he/she sets "Request changes" in the github review system.

Kwankyu

[John H Palmieri]

I agree that we need a policy, and I am happy with David's proposal.

A tangential follow-up to Matthias: I think that our code of conduct should be part of the distributed documentation. Should it be in the Developer's Guide? In some other existing documentation? As a standalone document?

--

John

[Kwankyu Lee]

A tangential follow-up to Matthias: I think that our code of conduct should be part of the distributed documentation. Should it be in the Developer's Guide? In some other existing documentation? As a standalone document?

Yes. I agree that it is very relevant. But to keep a single source of truth, we may just put a link to 

https://github.com/sagemath/sage/blob/develop/CODE_OF_CONDUCT.md

in the developer guide right at the front page: https://deploy-livedoc--sagemath-tobias.netlify.app/html/en/developer/

 

 

[Kwankyu Lee]

Meanwhile, Matthias and Dima spent lots of mental energy to produce a prime example showing why we need the policy:

https://github.com/sagemath/sage/pull/36726

Please come down from sun-shining deck to the murky bottom of our ship to see the danger that might drown all of us...

[Matthias Koeppe]

On Tuesday, November 28, 2023 at 1:25:17 PM UTC-8 Kwankyu Lee wrote:

Meanwhile, Matthias and Dima spent lots of mental energy to produce a prime example showing why we need the policy:

https://github.com/sagemath/sage/pull/36726

I endorse this example as one that is safe to study, without the need for a trigger warning.

[Dima Pasechnik]

well, I am trying to pull the ship into the sea of normal Python
packages, while participating in the endless bloating up of the build
system and the external packages (because everything must be built
from source, as if we toil for one of super-paranoid 3-letter
agencies)
we are drowning in.

Like, adding by hand hundreds of 1-line text files to basically
duplicate what's known to pip.

See e.g. https://github.com/sagemath/sage/pull/36777
(WIP, where I'm supposed to add about 400 such files by copy/paste
from https://repology.org/)
- totally unneeded, we just should not be shipping Jupyter and
IPython. They can be pip-installed just fine.

We could have removed the bloat of "toolchain" years ago. No major
Python package/system ships
compilers, and vendored Python. No major Python system ships anything
Jupyter (E.g. look at scipy, sympy).
Meanwhile Mattias couldn't even part with Cygwin deadwood:
https://github.com/sagemath/sage/pull/36782

I am hating this senseless process more and more. Other people who
work on the build system aren't exactly happy with the direction this
is all going to,
either.

Dima

>
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/8b07f03b-b60a-4cce-9dd9-59925bdf15c1n%40googlegroups.com.

[David Roe]

Let's try to focus on the policy proposal, rather than specific disagreements on individual PRs.

Dima, I'm sorry that you're feeling frustrated with the whole process.  It may be helpful to have additional directions about the overall strategy for Sage's build system, but that's better put off to another thread.

David

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq35iKf1EkaLkccvNp7kd9e254QD5Y4%2B1BoFROJL2R1iiQ%40mail.gmail.com.

[Dima Pasechnik]

On Tue, Nov 28, 2023 at 10:02 PM David Roe <roed...@gmail.com> wrote:
>
> Let's try to focus on the policy proposal, rather than specific disagreements on individual PRs.

The whole thing about specific disagreements on individual PRs comes
exactly from the wrong overall direction of the project.
Which replaced, hopeless in the long run, "just vendor everything"
motto with even worse and much more labour-intensive
"let's keep track of everything Sage users could possibly need and
use, on every system Sage could run".

Let's stop this control freakery, then we won't run into disagreements
born out of frustration, and there won't be any urgent need
for this proposal in the 1st place.

Dima

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAChs6_kif5KcLFTdWVXMs8acbkg6iDxCQinF6SO_81Uavkt6uQ%40mail.gmail.com.

[Matthias Koeppe]

On Saturday, November 25, 2023 at 1:24:42 AM UTC-8 Kwankyu Lee wrote:

(2) How do we count approvers and disapprovers for a disputed PR: A reviewer becomes an approver (who is in favor of the PR) when he/she sets "Approve" in the github review system. A reviewer becomes a disapprover (who objects the PR) when he/she sets "Request changes" in the github review system.

I think there needs to be a clear indication that a voting period is active (and when it closes). Perhaps we can use a PR label "s: voting" or "s: needs votes"?

 

[Kwankyu Lee]

I think there needs to be a clear indication that a voting period is active (and when it closes). Perhaps we can use a PR label "s: voting" or "s: needs votes"?

If we do not want to invent a new label, we may add "s: needs review", "s: needs work", "s:needs info" altogether to get attention.

Then the voting period starts when the three labels are added.

I suggest to end the voting when a week has passed after the last vote was casted.

[Tobia...@gmx.de]

At first I was very enthusiastic about this proposed policy, but after thinking about this for a bit I'm no longer convinced this is a good idea.

First of all, the policy sets out to solve the case "where there is a general consensus, but one person (or a few people) disagree". In my experience, this case is not a problem. All the examples mentioned so far (and the few other examples I'm aware of), have usually one positive reviewer and one negative review. This is not a general consensus. The problem is more that a general consensus cannot be reached. Another aspect of the issue is that usually only a very small group of 2 to 3 people is involved in discussing the PR, which perhaps not surprisingly then more easily results in a state where all arguments have been exchanged without finding a solution satisfying everyone.

For example, with the proposed policy, Dima and me would have outvoted Matthias in https://github.com/sagemath/sage/pull/35403. But this PR was largely improved by the discussion on the mailing list (that it is still not clear how to proceed with this PR is another sad story).

In light of this, I would like to propose to change the policy proposal to an automatic system that draws more attention to the PR, with the hope that new people bring new input and ideas, which then resolves the conflict in a natural way. The proposal is something along the following: if a PR is say a week in the "disputed state" as defined above by Kwankyu, both parties write a short statement of why they think it should or should not be merged, and this summary is then posted to the mailing list. Not to start a voting, but to raise awareness and invite other devs to join the discussion. Similar calls for PR reviews are not uncommon on the mailing list, so I don't think it would annoy subscribers too much.

Finally, I think Dima raises a very important point. Most of the discussions in these "disputed PRs" are a result of a lack of a common vision for the project and agreement on what projects to work on. It would be immensely more productive to have a general discussion e.g. about how to proceed with sage-the-distribution (replace it?, with what?, how to sunset it? reduce it? enlarge it?). As an example, I think conda is a good candidate to replace sage-the-distribution and thus naturally open PRs with changes in that direction. But if you don't agree with this general direction, it's easy to find these changes annoying. On the other hand, if there would be an agreement that conda was a nice experiment that we don't want to continue, then I'm happy to delete it completely. But instead of a general direction, we have this situation where every developer is having their own ideas and little projects that they are working on, and that are bound to step on toes of others.

[Kwankyu Lee]

In light of this, I would like to propose to change the policy proposal to an automatic system that draws more attention to the PR, with the hope that new people bring new input and ideas, which then resolves the conflict in a natural way. 

The proposal already promotes more discussion by inviting more people to the PR, which provides more convenient place for discussion than the mailing list. It intends to stop non-productive discussion in dead-lock state.

 

... instead of a general direction, we have this situation where every developer is having their own ideas and little projects that they are working on, and that are bound to step on toes of others.

I agree. But the general direction would not resolve specific disputed PRs unless a dictator interprets the general direction for them. 

[John H Palmieri]

The original proposal allows for anyone to post to sage-devel to try to raise awareness ("Also note that an objector is welcome to attempt to bring others into the discussion on their side if they remain firmly opposed"). I prefer allowing the various participants the freedom to decide whether to start a discussion in sage-devel rather than forcing this to happen, but I'm happy to hear arguments for changing it as you propose.

For some topics and disputes, it is clear that they should be discussed on sage-devel, perhaps even discussed here before getting to the point of a PR. Recalling one of William's old April 1 jokes, if we wanted to consider switching the infrastructure from Python to Lisp, that would require a discussion on sage-devel first. You mentioned https://github.com/sagemath/sage/pull/35403, and I think that's another good example. That deals with which versions of Python we support and whether to automatically reject certain versions. Python plays such a central role in Sage, but at the same time that PR is somewhat technical, so I feel that it's on the border of "clearly should be discussed first on sage-devel" vs. "can be handled in the background". Once the conflict arose, it made a lot of sense to bring it to sage-devel. A third example: Kwankyu raised https://github.com/sagemath/sage/pull/36726, and in my view this one is very minor and technical, and I don't see a case for a long discussion or vote on sage-devel. Mentioning it here to raise awareness makes sense, but that should be the extent of it; the rest of the discussion can happen on github.

To the extent that this specific PR is emblematic of a particular approach to Sage development (a flawed approach in Dima's view, if I understand right), then the whole approach should be discussed here. Probably many of these issues in Sage development have been discussed already, but it's probably time to revisit them, to see if we can reestablish a baseline level of consensus.

[William Stein]

On Thu, Nov 30, 2023 at 12:37 PM John H Palmieri <jhpalm...@gmail.com> wrote:

To the extent that this specific PR is emblematic of a particular approach to Sage development (a flawed approach in Dima's view, if I understand right), then the whole approach should be discussed here. Probably many of these issues in Sage development have been discussed already, but it's probably time to revisit them, to see if we can reestablish a baseline level of consensus.

As a person who has been involved in Sage a long time, I just want to +1 John's remark that major issues involving the direction and approach to Sage development, even if they have been discussed before, are definitely something that should be discussed again.  The optimal choices for a Sage can easily change as the world changes, and there are many relevant factors to Sage's development that are massively different now than in the past.   Examples: python is much more popular now than ever before; GPU's are vastly more powerful now than before; GitHub with its amazing free CI infrastructure exists; Conda exists; GCC isn't the only free C compiler; WebAssembly exists, ...

 

 

On Wednesday, November 29, 2023 at 7:12:31 AM UTC-8 Tobia...@gmx.de wrote:

At first I was very enthusiastic about this proposed policy, but after thinking about this for a bit I'm no longer convinced this is a good idea.

First of all, the policy sets out to solve the case "where there is a general consensus, but one person (or a few people) disagree". In my experience, this case is not a problem. All the examples mentioned so far (and the few other examples I'm aware of), have usually one positive reviewer and one negative review. This is not a general consensus. The problem is more that a general consensus cannot be reached. Another aspect of the issue is that usually only a very small group of 2 to 3 people is involved in discussing the PR, which perhaps not surprisingly then more easily results in a state where all arguments have been exchanged without finding a solution satisfying everyone.

For example, with the proposed policy, Dima and me would have outvoted Matthias in https://github.com/sagemath/sage/pull/35403. But this PR was largely improved by the discussion on the mailing list (that it is still not clear how to proceed with this PR is another sad story).

In light of this, I would like to propose to change the policy proposal to an automatic system that draws more attention to the PR, with the hope that new people bring new input and ideas, which then resolves the conflict in a natural way. The proposal is something along the following: if a PR is say a week in the "disputed state" as defined above by Kwankyu, both parties write a short statement of why they think it should or should not be merged, and this summary is then posted to the mailing list. Not to start a voting, but to raise awareness and invite other devs to join the discussion. Similar calls for PR reviews are not uncommon on the mailing list, so I don't think it would annoy subscribers too much.

Finally, I think Dima raises a very important point. Most of the discussions in these "disputed PRs" are a result of a lack of a common vision for the project and agreement on what projects to work on. It would be immensely more productive to have a general discussion e.g. about how to proceed with sage-the-distribution (replace it?, with what?, how to sunset it? reduce it? enlarge it?). As an example, I think conda is a good candidate to replace sage-the-distribution and thus naturally open PRs with changes in that direction. But if you don't agree with this general direction, it's easy to find these changes annoying. On the other hand, if there would be an agreement that conda was a nice experiment that we don't want to continue, then I'm happy to delete it completely. But instead of a general direction, we have this situation where every developer is having their own ideas and little projects that they are working on, and that are bound to step on toes of others.

On Wednesday, November 29, 2023 at 7:20:12 AM UTC+8 Kwankyu Lee wrote:

I think there needs to be a clear indication that a voting period is active (and when it closes). Perhaps we can use a PR label "s: voting" or "s: needs votes"?

If we do not want to invent a new label, we may add "s: needs review", "s: needs work", "s:needs info" altogether to get attention.

Then the voting period starts when the three labels are added.

I suggest to end the voting when a week has passed after the last vote was casted.

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/3d62ce18-f2b0-408b-878a-6ccb1d202584n%40googlegroups.com.

--

William (http://wstein.org)

[David Joyner]

On Wed, Nov 29, 2023 at 10:12 AM Tobia...@gmx.de <Tobia...@gmx.de> wrote:

At first I was very enthusiastic about this proposed policy, but after thinking about this for a bit I'm no longer convinced this is a good idea.

First of all, the policy sets out to solve the case "where there is a general consensus, but one person (or a few people) disagree". In my experience, this case is not a problem. All the examples mentioned so far (and the few other examples I'm aware of), have usually one positive reviewer and one negative review. This is not a general consensus. The problem is more that a general consensus cannot be reached. Another aspect of the issue is that usually only a very small group of 2 to 3 people is involved in discussing the PR, which perhaps not surprisingly then more easily results in a state where all arguments have been exchanged without finding a solution satisfying everyone.

For example, with the proposed policy, Dima and me would have outvoted Matthias in https://github.com/sagemath/sage/pull/35403. But this PR was largely improved by the discussion on the mailing list (that it is still not clear how to proceed with this PR is another sad story).

In light of this, I would like to propose to change the policy proposal to an automatic system that draws more attention to the PR, with the hope that new people bring new input and ideas, which then resolves the conflict in a natural way. The proposal is something along the following: if a PR is say a week in the "disputed state" as defined above by Kwankyu, both parties write a short statement of why they think it should or should not be merged, and this summary is then posted to the mailing list. Not to start a voting, but to raise awareness and invite other devs to join the discussion. Similar calls for PR reviews are not uncommon on the mailing list, so I don't think it would annoy subscribers too much.

Finally, I think Dima raises a very important point. Most of the discussions in these "disputed PRs" are a result of a lack of a common vision for the project and agreement on what projects to work on. It would be immensely more productive to have a general discussion e.g. about how to proceed with sage-the-distribution (replace it?, with what?, how to sunset it? reduce it? enlarge it?). As an example, I think conda is a good candidate to replace sage-the-distribution and thus naturally open PRs with changes in that direction. But if you don't agree with this general direction, it's easy to find these changes annoying. On the other hand, if there would be an agreement that conda was a nice experiment that we don't want to continue, then I'm happy to delete it completely. But instead of a general direction, we have this situation where every developer is having their own ideas and little projects that they are working on, and that are bound to step on toes of others.

Thank you for this. It seems to me to be a reasonable explanation of what's going on. 

One question: Dima has said he is "... trying to pull the ship into the sea of normal Python packages...". Isn't this consistent with your goal that "conda is a good candidate" for SageMath distribution?

 

On Wednesday, November 29, 2023 at 7:20:12 AM UTC+8 Kwankyu Lee wrote:

I think there needs to be a clear indication that a voting period is active (and when it closes). Perhaps we can use a PR label "s: voting" or "s: needs votes"?

If we do not want to invent a new label, we may add "s: needs review", "s: needs work", "s:needs info" altogether to get attention.

Then the voting period starts when the three labels are added.

I suggest to end the voting when a week has passed after the last vote was casted.

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/c602514f-b7a1-4afc-bb04-cdde37b4a879n%40googlegroups.com.

[Kwankyu Lee]

If we do not want to invent a new label, we may add "s: needs review", "s: needs work", "s:needs info" altogether to get attention.

The pending script (https://github.com/sagemath/sage/pull/36292)  that automatically manages the github labels won't be happy with this (several status labels).

So we may need to make a new label like "s: needs voting" (with some appropriate color to get attention).

For fine points of the proposal, I suggest

(1) The community's decision by voting is only initiated by the author's request (perhaps by a comment like "I request the community's decision")

(2) If the voting results in disapproval, the PR is closed with a resolution label (perhaps "r: wontfix" or a new label "r: disapproved"?).

 

[kcrisman]

To the extent that this specific PR is emblematic of a particular approach to Sage development (a flawed approach in Dima's view, if I understand right), then the whole approach should be discussed here. Probably many of these issues in Sage development have been discussed already, but it's probably time to revisit them, to see if we can reestablish a baseline level of consensus.

As a person who has been involved in Sage a long time, I just want to +1 John's remark that major issues involving the direction and approach to Sage development, even if they have been discussed before, are definitely something that should be discussed again.  The optimal choices for a Sage can easily change as the world changes, and there are many relevant factors to Sage's development that are massively different now than in the past.   Examples: python is much more popular now than ever before; GPU's are vastly more powerful now than before; GitHub with its amazing free CI infrastructure exists; Conda exists; GCC isn't the only free C compiler; WebAssembly exists, ...

Thanks, William, that's great perspective.

Relevant to both the overall issue of project direction *and* the specific one about Sage-as-distribution, I would just add keeping in mind the Sage mission (at least, as it currently stands):

Mission: Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab.

Digression, but not really: That this is related to the packaging discussion is evident in that different packaging scenarios have different outcomes for end users, especially those *not* on Linux, which I for one would like to see more numerous than those on Linux, since that's where the people doing (and teaching!) math probably still are, in reality.  It would be great to live fully in the Python world as well as achieve this, but perhaps these goals are not in the same subspace.  (Or maybe they are.)

As just one example, here is a very recent discussion that (partly) is about how to use Sage on Windows (by people not necessarily averse to CLI stuff).  

https://groups.google.com/g/pretext-support/c/2_4Lz6fRKjM 

Since these people are *atypical* among people doing math in their proclivities to try out all kinds of new toys (as opposed to just doing math), I suspect strongly that the last embray-produced Windows binary is still used WAY more than any WSL solution, because even if people *can* figure out how to use it, would they bother if they have a proprietary option available instead?

This is a good place to thank embray and darthandrus (among many others) for work on previous Windows and Mac "one-click" download options, and especially the 3-manifolds project for the current one for Mac.  Any changes (or lack thereof) to Sage-as-distribution that makes one-download easier for Windows and Mac are probably the best in this important sense - and anything that makes it significantly harder for 3-manifolds is probably one to discuss carefully first.

[Matthias Koeppe]

On Thursday, November 30, 2023 at 3:14:57 PM UTC-8 kcrisman wrote:

This is a good place to thank embray and darthandrus (among many others) for work on previous Windows and Mac "one-click" download options, and especially the 3-manifolds project for the current one for Mac.

+1

[Matthias Koeppe]

In the discussion in one of the PRs linked here, we have identified a separate issue.

The SageMath project has a high complexity, which can be overwhelming to some. 

As part of our goal to make the Sage development community more inclusive, we should expand the developer's guide with strategies, resources, tools for managing cognitive overload in Sage development. I've opened https://github.com/sagemath/sage/issues/36803 for this.

[Dima Pasechnik]

We are discussing shortcomings of a huge mono-repo which only keeps growing. GitHub's idea that you release a whole repo makes it quite impossible to release parts of Sage without a lockstep. While back in 2021 we haven't quite realised this, it's becoming clear now.

The issue #36803  is an attempt to solve software problems by HR methods.
"Meditate enough and the bugs will disappear."

There is however no excuse for e.g. having all the Sage spkgs in one unstructured build/pkgs/ directory, no amount of meditation and breathing exercises will cure it.
As well there is no way in GitHub to separate PRs and issues for a repo into subprojects, they will always be together in one huge pile.

That's why I think splitting the repo is inevitable. One obvious cutting line is to split out all the GUI, i.e. the Jupyter-related things.


Dima

[kcrisman]

Relevant to both the overall issue of project direction *and* the specific one about Sage-as-distribution, I would just add keeping in mind the Sage mission (at least, as it currently stands):

Mission: Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab.

For those who may not have actively followed this thread: discussion on a possible new mission at https://github.com/sagemath/sage/issues/36827

[Matthias Koeppe]

The most recent response in this discussion thread was posted over 3 weeks ago.

(There has been major activity on PRs linked in some of the posts, though.)

Do we have a timeline for the next step in this effort?

Best wishes for the new year to all members of the Sage community!

Matthias

On Friday, November 24, 2023 at 8:18:34 AM UTC-8 David Roe wrote:

[Kwankyu Lee]

Wish you a happy new year!

A year ago, we successfully escaped from a sinking issue management system.

Hopefully, the new year will save us from the current spiral that's drowning us.

[Kwankyu Lee]

Hi,

Another "disputed" PR is piled on the heap: https://github.com/sagemath/sage/pull/36999

Behind a disputed PR, there is a lot of time and energy wasted from the author, the reviewers, and the audience. The disputed PRs discourage everyone in the community.

I am aware that one policy about the disputed PRs is in action: the release manager took the editor role and has made decisions on some disputed PRs.

But I feel that disputed PRs are to be piled up, and the editor policy is not efficient enough. We need some permanent and fast policy. So I suggest the following

1. A PR is a "disputed PR" if there are both approving and disapproving reviewers. 

2. It is the author's right to add "disputed" label to his/her disputed PR (to prevent another dispute on adding the label).

3. The release manager is the lifelong editor for disputed PRs. 

4. We adopt the "automatic poll" policy:

    (a) A disputed PR gets "Approve" or "Request changes" decisions from reviewers.

    (b) If the number of "Approve" decisions is more than or equal to the twice of the number of "Request changes" decisions, then "positive review" label is added to the PR.

    (c) There is no voting period. It is up to the author whether to close the PR or to keep waiting.

If David has no time, then I will post a voting on sage-devel after hearing some comments.

Thanks for attention.