[llvm-dev] RFC: Code Review Process

[Tom Stellard via llvm-dev]

Hi,

# Proposal

The LLVM Foundation Board of Directors is seeking comment on the current state of Code Review
within the LLVM Project and its sub-projects. Phabricator is no longer actively maintained
and we would like to move away from a self-hosted solution, so our goal is to determine if
GitHub Pull Requests are a good alternative to our current code review tool: Phabricator.

Specifically we are looking for feedback on:
- What features or properties make Github Pull Requests better than Phabricator?
- What features or properties make Phabricator better than GitHub Pull Requests?
- What new workflows or process improvements will be possible with GitHub Pull Requests?
- Which workflows aren’t possible with GitHub Pull Requests?
- Any other information that you think will help the Board of Directors make the best decision.

# Where to Direct Feedback

Please provide feedback on the Infrastructure Working Group ticket[1]. This will make
it easier to collect and consolidate the responses. At the end of the comment period
the Infrastructure Working Group will collect the feedback for further analysis and summarization.

# Timeline

The timeline for this RFC will be as follows:

- RFC posted on llvm-dev for public review and comment
- 30 days after the date of posting, public comment closes.
- IWG will have 14 days from closure of public comments to review and summarize public
comments into a pros and cons list to be present to LLVM Foundation Board
- Foundation Board will have 30 days to make a final decision about using GitHub Pull Requests
and then communicate a migration plan to the community.

Thank you,
LLVM Foundation Board of Directors

[1] https://github.com/llvm/llvm-iwg/issues/73

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Renato Golin via llvm-dev]

On Tue, 5 Oct 2021 at 17:06, Tom Stellard via llvm-dev <llvm...@lists.llvm.org> wrote:

- Any other information that you think will help the Board of Directors make the best decision. 

- Foundation Board will have 30 days to make a final decision about using GitHub Pull Requests and then communicate a migration plan to the community.

Hi Tom,

Please help me here, I think I'm severely misunderstanding what this means...

I'm reading it that the "Board of Directors" will make a decision and communicate to the community, apparently through some undisclosed internal process.

For example:

 * What about people that are on holidays during the 30 days comment period?

 * What if the points are not made clear after 30 days?

 * How do we know the IWG will correctly summarise the comments to the board?

 * How does the board guarantee it will take all facts in consideration without bias?

 * What kind of recourse would the community have if the decision alienates a large part of the developers?

Please understand that I'm not assuming malice *at all*. We're all humans, and in the effort to make some change happen we quite often let unconscious bias be the merits of our decisions.

For context...

Since its inception[1], the foundation has always steered away from technical decisions, always referring to the llvm-dev list for those. Previous long running contentious issues (Github, monorepo, CoC) were all decided by the community, in the llvm-dev list, and executed by the foundation.

Recent discussions about the mailing list, irc, discord, discourse have emphasised that, even with an infrastructure working group, the views of the community are still too hard to predict and make it work for the majority. Neither the board of directors, nor the IWG are wide and diverse enough to make decisions that take most people's views into consideration. That is why we still rely on the dev list for large technical discussions and decisions.

Code review and bug tracking are very much technical decisions. Not code directly, but how we all work. And there are a lot of us. Giving feedback and having no insight into the decision making process will certainly divide the community even more, if we're forced to accept whatever outcome.

I can't see how this "solves" the problem of never-ending discussions, other than further fragmenting the community.

cheers,

--renato

[1] http://blog.llvm.org/2014/04/the-llvm-foundation.html

[Tom Stellard via llvm-dev]

On 10/5/21 9:47 AM, Renato Golin wrote:

> On Tue, 5 Oct 2021 at 17:06, Tom Stellard via llvm-dev <llvm...@lists.llvm.org <mailto:llvm...@lists.llvm.org>> wrote:
>
> - Any other information that you think will help the Board of Directors make the best decision.
>
> - Foundation Board will have 30 days to make a final decision about using GitHub Pull Requests and then communicate a migration plan to the community.
>
>
> Hi Tom,
>
> Please help me here, I think I'm severely misunderstanding what this means...
>
> I'm reading it that the "Board of Directors" will make a decision and communicate to the community, apparently through some undisclosed internal process.
>
> For example:
>  * What about people that are on holidays during the 30 days comment period?
>  * What if the points are not made clear after 30 days?
>  * How do we know the IWG will correctly summarise the comments to the board?
>  * How does the board guarantee it will take all facts in consideration without bias?
>  * What kind of recourse would the community have if the decision alienates a large part of the developers?
>
> Please understand that I'm not assuming malice *at all*. We're all humans, and in the effort to make some change happen we quite often let unconscious bias be the merits of our decisions.
>
> For context...
>
> Since its inception[1], the foundation has always steered away from technical decisions, always referring to the llvm-dev list for those. Previous long running contentious issues (Github, monorepo, CoC) were all decided by the community, in the llvm-dev list, and executed by the foundation.
>

In my opinion, this is not a technical issue. The Board owns the infrastructure
for the project and is responsible for ensuring that it is well maintained and
functional. From the blog post:

"The LLVM Foundation" will allow us to:

- Solve infrastructure problems.

This is what we are doing here. The project is very much at risk by using
a self-hosted, unmaintained code review tool. We really need to move forward
with a more robust solution otherwise we risk a major disruption to the community.

> Recent discussions about the mailing list, irc, discord, discourse have emphasised that, even with an infrastructure working group, the views of the community are still too hard to predict and make it work for the majority. Neither the board of directors, nor the IWG are wide and diverse enough to make decisions that take most people's views into consideration. That is why we still rely on the dev list for large technical discussions and decisions.
>
> Code review and bug tracking are very much technical decisions. Not code directly, but how we all work. And there are a lot of us. Giving feedback and having no insight into the decision making process will certainly divide the community even more, if we're forced to accept whatever outcome.
>

What additional information about the decision making process would you like to see?

-Tom

> I can't see how this "solves" the problem of never-ending discussions, other than further fragmenting the community.
>
> cheers,
> --renato
>

> [1] http://blog.llvm.org/2014/04/the-llvm-foundation.html <http://blog.llvm.org/2014/04/the-llvm-foundation.html>

[Renato Golin via llvm-dev]

On Tue, 5 Oct 2021 at 18:09, Tom Stellard <tste...@redhat.com> wrote:

In my opinion, this is not a technical issue.

I find that surprising. But maybe it's just me.

 

  The Board owns the infrastructure
for the project and is responsible for ensuring that it is well maintained and
functional.  From the blog post:

"The LLVM Foundation" will allow us to:

  - Solve infrastructure problems.

This is what we are doing here.

In that view, what's stopping the foundation from moving out of emails and forcing us all to use Discourse?

To me, those are exactly the same kind of problem, and the IWG has already concluded[1] it had to be done. Both mailing lists and phab are self-hosted by us, paid by the foundation.

I'm still confused about the change in the decision process...

cheers,

--renato

[1] https://lists.llvm.org/pipermail/llvm-dev/2021-June/150823.html

[Mehdi AMINI via llvm-dev]

I'd like to dispute this on multiple accounts.

- You write that "the board owns the infrastructure", but the board has never been involved with Phabricator in any way (the hosting is provided by Google from the beginning: both the machine and human-power to keep it running), nor did the board get in touch recently with the folks currently keeping the instance running as far as I know.

- You write that the "project is very much at risk", but Phabricator has been self-hosted for > 8 years and it isn't clear to me why there is a sudden emergency on this side. The claim of "risk a major disruption to the community" to justify the current push looks like FUD to me.

The RFC states that "we would like to move away from a self-hosted solution", but who is "we"? How was this decided and why?

> Recent discussions about the mailing list, irc, discord, discourse have emphasised that, even with an infrastructure working group, the views of the community are still too hard to predict and make it work for the majority. Neither the board of directors, nor the IWG are wide and diverse enough to make decisions that take most people's views into consideration. That is why we still rely on the dev list for large technical discussions and decisions.
>
> Code review and bug tracking are very much technical decisions. Not code directly, but how we all work. And there are a lot of us. Giving feedback and having no insight into the decision making process will certainly divide the community even more, if we're forced to accept whatever outcome.

+1 to everything Renato wrote above, in particular how these tools are fairly core to our development and are technical matters.

With all that said, I think the process and the way the RFC was written distracts unnecessarily from the discussion here: it seems fairly healthy to me to just re-evaluate our tooling and how they fit our needs and revisit the alternatives. 

I'd love it if we were able to move to pull-request, but I'm also quite wary of rushing it for other considerations before we can get a roadmap to get to the same feature level on GitHub that we get through Phabricator today (and the narrative pushed through in the RFC does not bring me confidence here).

Best,

 

-- 

Mehdi

[Tom Stellard via llvm-dev]

On 10/5/21 10:48 AM, Mehdi AMINI wrote:

>
>
> On Tue, Oct 5, 2021 at 10:09 AM Tom Stellard via llvm-dev <llvm...@lists.llvm.org <mailto:llvm...@lists.llvm.org>> wrote:
>
> On 10/5/21 9:47 AM, Renato Golin wrote:

Sorry, 'owns' is probably the wrong word. What I mean is the Board is responsible
for the infrastructure. Yes, it's great that Google has been contributing the
machines and human support to keep it running, this has been a great service to
the community. However, it's not a good position for the Board to be responsible
for something that it doesn't have control over. If Google decided to stop hosting
Phabricator for some reason (unlikely, but not impossible), the Board would be
responsible for finding a replacement.

> - You write that the "project is very much at risk", but Phabricator has been self-hosted for > 8 years and it isn't clear to me why there is a sudden emergency on this side. The claim of "risk a major disruption to the community" to justify the current push looks like FUD to me.
>

This is not meant to insult or diminish the work done by the maintainers over
the last 8 years. The self-hosted part is a risk, but a small one for reasons
mentioned above. The main risk is that Phabricator is no longer maintained upstream.
There was already an issue[1] recently where the arc tool stopped working and won't
be fixed upstream. Using unmaintained software is a bigger risk.

[1] https://lists.llvm.org/pipermail/llvm-dev/2021-September/153019.html

> The RFC states that "we would like to move away from a self-hosted solution", but who is "we"? How was this decided and why?
>

We, meaning the LLVM Board of Directors. And really the problem isn't the self-hosting
so much as it's the lack of an enforceable maintenance agreement the Foundation and the
maintainers.

-Tom

>
> > Recent discussions about the mailing list, irc, discord, discourse have emphasised that, even with an infrastructure working group, the views of the community are still too hard to predict and make it work for the majority. Neither the board of directors, nor the IWG are wide and diverse enough to make decisions that take most people's views into consideration. That is why we still rely on the dev list for large technical discussions and decisions.
> >
> > Code review and bug tracking are very much technical decisions. Not code directly, but how we all work. And there are a lot of us. Giving feedback and having no insight into the decision making process will certainly divide the community even more, if we're forced to accept whatever outcome.
>
>
> +1 to everything Renato wrote above, in particular how these tools are fairly core to our development and are technical matters.
>
> With all that said, I think the process and the way the RFC was written distracts unnecessarily from the discussion here: it seems fairly healthy to me to just re-evaluate our tooling and how they fit our needs and revisit the alternatives.
>
> I'd love it if we were able to move to pull-request, but I'm also quite wary of rushing it for other considerations before we can get a roadmap to get to the same feature level on GitHub that we get through Phabricator today (and the narrative pushed through in the RFC does not bring me confidence here).
>
> Best,
> --
> Mehdi
>
> >
>
> What additional information about the decision making process would you like to see?
>
> -Tom
>
> > I can't see how this "solves" the problem of never-ending discussions, other than further fragmenting the community.
> >
> > cheers,
> > --renato
> >

> > [1] http://blog.llvm.org/2014/04/the-llvm-foundation.html <http://blog.llvm.org/2014/04/the-llvm-foundation.html> <http://blog.llvm.org/2014/04/the-llvm-foundation.html <http://blog.llvm.org/2014/04/the-llvm-foundation.html>>

>
> _______________________________________________
> LLVM Developers mailing list

> llvm...@lists.llvm.org <mailto:llvm...@lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>

[Philip Reames via llvm-dev]

+1 to Renato's response here.  I had the same thought, and Renato phrased it much better than I'd have managed.

Philip

[Renato Golin via llvm-dev]

On Tue, 5 Oct 2021 at 19:16, Tom Stellard <tste...@redhat.com> wrote:

However, it's not a good position for the Board to be responsible
for something that it doesn't have control over.  If Google decided to stop hosting
Phabricator for some reason (unlikely, but not impossible), the Board would be
responsible for finding a replacement.

Sorry, this is a very weak reason for such a strong worded "RFC".

I _cannot_ imagine "Google" stopping to support something so quickly as to leave the foundation without recourse. And even if they did, *no one* would blame the foundation for that.

Even if you ignore all the effort that hundreds of their engineers have done over the past decade to the project, this would hurt Google more than anyone else. It's a far fetched concern.

And if the foundation wants "control" of a piece of infrastructure that Google has been maintaining for years, then this is a different discussion. Hopefully one that doesn't involve unilateral decisions.

 

The main risk is that Phabricator is no longer maintained upstream.
There was already an issue[1] recently where the arc tool stopped working and won't
be fixed upstream. Using unmaintained software is a bigger risk.

I don't like using unmaintained software either, but I think our Phab has had more attention than the upstream project. And no one has to use arc, I certainly never have.

Don't get me wrong, I don't like Phab and I think Github would bring new people to the project, but it's gotta be done the right way, and pushing it isn't it.

We, meaning the LLVM Board of Directors.  And really the problem isn't the self-hosting
so much as it's the lack of an enforceable maintenance agreement the Foundation and the
maintainers.

The problem isn't self-hosting at all, given that Google is doing that.  (apologies, I assumed otherwise earlier).

Neither is maintenance, given Google is doing that too.

The only thing that's left is control, and I don't really understand why this is important, as I explained above.

cheers,

--renato

[Tanya Lattner via llvm-dev]

Hello! The purpose of this email is to start a discussion about our code review tools. No decisions have been made about changing tools. The idea behind a timeline is so that information could be gathered in a timely manner. The Infrastructure Working Group was formed to bring together community members who have an experience and/or passion regarding infrastructure. Anyone can participate in this working group and like the LLVM Foundation, the minutes are all made public. 

The LLVM Foundation’s mission is to support the LLVM project and help ensure the health and productivity of of the community and this is done through numerous ways including infrastructure. I do not think it is a negative thing that the foundation board of directors would be discussing our current tools and gathering information how how well they work and how we can make them better. As the legal entity who bares financial and legal responsibility for a lot of the infrastructure, this would make sense. This also makes sense because of the people involved who care a lot about LLVM and the project. But, the LLVM Foundation does not pay for Phabricator and we are very grateful for Google’s support of this critical piece of our infrastructure. 

Regarding Phabricator, there are a couple of pieces of information that were provided to the LLVM Foundation by maintainers (maybe previous it sounds like) of this instance and how we may need to look into alternative ways to support it.  In addition, Phacility itself has publicly stated that it is winding down operations. (https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_operations/). Lastly, there are questions about why we are not using GitHub pull requests as we are on GitHub and that might be the natural path to take for a number of reasons.

The above reasons are why the RFC was written. Perhaps it wasn’t written in the best way, but I also feel like it is being read in a negative way which is incredibly disappointing given I don’t feel there is a valid reason for this.

-Tanya

[Nick Desaulniers via llvm-dev]

On Tue, Oct 5, 2021 at 9:05 AM Tom Stellard via cfe-dev
<cfe...@lists.llvm.org> wrote:
>
> Hi,
>
> # Proposal
>
> The LLVM Foundation Board of Directors is seeking comment on the current state of Code Review
> within the LLVM Project and its sub-projects. Phabricator is no longer actively maintained
> and we would like to move away from a self-hosted solution, so our goal is to determine if
> GitHub Pull Requests are a good alternative to our current code review tool: Phabricator.
>
> Specifically we are looking for feedback on:
> - What features or properties make Github Pull Requests better than Phabricator?

I appreciate the ability to have bots help participate in code review,
though perhaps this isn't actually specific to github.

In particular, I think rustc's use of bors for merging is pretty
awesome; people literally cannot merge unless the bot has run all unit
tests on all platforms. As is, anyone can commit without running any
tests.

I frequently break the build for non-linux platforms. That LLVM has so
many reverts for breakage on platforms tested post submit is
embarrassing. People chasing such breakages is a waste of their time,
IMO. We should have more pre-submit testing; I think the board should
focus on that particular problem first, BEFORE pursuing changing code
review platforms.

It seems an obvious risk to me that phab is no longer maintained, but
as others have noted, it hasn't all come crashing down yet.

> - What features or properties make Phabricator better than GitHub Pull Requests?
> - What new workflows or process improvements will be possible with GitHub Pull Requests?
> - Which workflows aren’t possible with GitHub Pull Requests?
> - Any other information that you think will help the Board of Directors make the best decision.
>
> # Where to Direct Feedback
>
> Please provide feedback on the Infrastructure Working Group ticket[1]. This will make
> it easier to collect and consolidate the responses. At the end of the comment period
> the Infrastructure Working Group will collect the feedback for further analysis and summarization.
>
> # Timeline
>
> The timeline for this RFC will be as follows:
>
> - RFC posted on llvm-dev for public review and comment
> - 30 days after the date of posting, public comment closes.
> - IWG will have 14 days from closure of public comments to review and summarize public
> comments into a pros and cons list to be present to LLVM Foundation Board
> - Foundation Board will have 30 days to make a final decision about using GitHub Pull Requests
> and then communicate a migration plan to the community.
>
> Thank you,
> LLVM Foundation Board of Directors
>
> [1] https://github.com/llvm/llvm-iwg/issues/73
>
> _______________________________________________

> cfe-dev mailing list
> cfe...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

--
Thanks,
~Nick Desaulniers

[Anshil Gandhi via llvm-dev]

+1 for Nick's comment

 Anshil

[James Henderson via llvm-dev]

Forgive me if I'm wrong, but if the community consensus is that we should continue to use Phabricator, and Phabricator is not being provided/maintained by the LLVM Foundation, isn't it moot what the LLVM Foundation/Infrastructure Working Group recommends/wants to happen? The current maintainers would continue to maintain Phabricator (assuming they wanted to), and people would still be able to review things there. What would happen if the Foundation officially supported PRs, without community consensus (in particular from the Phabricator maintainers), is a potential split in the community, with some continuing in the old way and others using the new way (and presumably some choosing to review on both platforms). This cannot be good.

I'm all for the discussion to be had, about whether we switch, but as far as I can see, nothing's really changed from the previous conversations on PRs versus Github, apart from the announcement of end of support by the upstream company, but that was quite a while ago now, and even with the stale Arcanist issue, there hasn't been a big push from community members to change: the consensus in the posts discussing this and the moving to PRs seems to still be "there are things that are blocking switching still".

At the most, from this IWG/Foundation consultation, it should be that the Foundation recommends one or other approach, and is willing to provide X infrastructure required. The community can then choose to agree with whatever approach is recommended or stick with the status quo. There shouldn't be an edict that says we will do one thing or the other.

Another side-point: whilst the IWG may consist of people who care about LLVM, there are far more people who care as much, but who just don't have the time to participate in such a group. This is particularly important to note, because the community does not elect members to this group. To an extent, the same is also true of the Foundation board itself, since there are plenty of people who may not agree with their decisions, but don't have the time to volunteer for the board. I'm not suggesting that there's any malice in this discussion, and indeed, the fact that it's open to community comments certainly is helpful, but I'd be worried of some kind of echo chamber/unconscious bias within the small groups suggesting there is consensus for one approach, when the wider community thinks otherwise.

James

[Chris Tetreault via llvm-dev]

> … nothing's really changed from the previous conversations on PRs versus Github, apart from the announcement of end of support by the upstream company, but that was quite a while ago now, and even with the stale Arcanist issue, there hasn't been a big push from community members to change …

 

James, If you’ll forgive me for cherry-picking a small part of your point, I think it bears mentioning that human beings tend to ignore future problems until they become current problems. Most of us here want to work on compilers, not deal with infrastructure. This doesn’t mean that the status quo is ok.

 

As I see it, it would be a mistake to just continue on with zombie-phabricator as we have. Perhaps the board of directors could have taken a different tone when presenting this issue, but I think they are doing the right thing by forcing a change soon. Tools are degrading, and security fixes are not being implemented. If we do nothing we’re all going to wake up some day and find that the github repo has had its owner changed or somesuch catastrophe. We need to do *something*, and I think setting a deadline for a change was the right call.

 

From my perspective, there are 4 reasonable things we can do, in order of disruptiveness:

 

1. Investigate a community replacement for phabricator. Does Phorge meet our needs? Is there a maintained fork of phabricator? Can we just drop in some replacement?
2. Fork Phabricator, and take on the maintenance burden ourselves. This sounds like work.
3. Move to github PRs. As others have mentioned, there are pros and cons to this.
4. Something else? We’d have to figure out what this is, and justify it over options 1-3.

 

If the deadline the board has set is unpalatable to the community, then perhaps it makes sense to fork Phabricator, and then decide on a longer term migration plan. But we need to do something and we need to do it now, not when there’s an actual fire.

 

Personally, I like Phabricator, and find github PRs to be tedious to work with. If we went with github PRs, I would be able to work, but I would prefer something more like phabricator.

 

thanks,

   Chris Tetreault

 

From: cfe-dev <cfe-dev...@lists.llvm.org> On Behalf Of James Henderson via cfe-dev
Sent: Wednesday, October 6, 2021 1:47 AM
To: Tanya Lattner <tanyal...@llvm.org>
Cc: llvm-dev <llvm...@lists.llvm.org>; Renato Golin <reng...@gmail.com>; clang developer list <cfe...@lists.llvm.org>; openmp-dev (openm...@lists.llvm.org) <openm...@lists.llvm.org>; LLDB Dev <lldb...@lists.llvm.org>
Subject: Re: [cfe-dev] [llvm-dev] RFC: Code Review Process

 

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

[Mara Sophie Grosch via llvm-dev]

+1 on finding a replacement before everything is burning. Thank you

Am Wed, Oct 06, 2021 at 05:26:24PM +0000 schrieb Chris Tetreault via llvm-dev:
>> … nothing's really changed from the previous conversations on PRs versus Github, apart from the announcement of end of support by the upstream company, but that was quite a while ago now, and even with the stale Arcanist issue, there hasn't been a big push from community members to change …
>
>James, If you’ll forgive me for cherry-picking a small part of your point, I think it bears mentioning that human beings tend to ignore future problems until they become current problems. Most of us here want to work on compilers, not deal with infrastructure. This doesn’t mean that the status quo is ok.
>
>As I see it, it would be a mistake to just continue on with zombie-phabricator as we have. Perhaps the board of directors could have taken a different tone when presenting this issue, but I think they are doing the right thing by forcing a change soon. Tools are degrading, and security fixes are not being implemented. If we do nothing we’re all going to wake up some day and find that the github repo has had its owner changed or somesuch catastrophe. We need to do *something*, and I think setting a deadline for a change was the right call.
>
>From my perspective, there are 4 reasonable things we can do, in order of disruptiveness:
>
>

> 1. Investigate a community replacement for phabricator. Does Phorge meet our needs? Is there a maintained fork of phabricator? Can we just drop in some replacement?
> 2. Fork Phabricator, and take on the maintenance burden ourselves. This sounds like work.
> 3. Move to github PRs. As others have mentioned, there are pros and cons to this.
> 4. Something else? We’d have to figure out what this is, and justify it over options 1-3.

>
>If the deadline the board has set is unpalatable to the community, then perhaps it makes sense to fork Phabricator, and then decide on a longer term migration plan. But we need to do something and we need to do it now, not when there’s an actual fire.
>
>Personally, I like Phabricator, and find github PRs to be tedious to work with. If we went with github PRs, I would be able to work, but I would prefer something more like phabricator.
>
>thanks,
> Chris Tetreault
>
>From: cfe-dev <cfe-dev...@lists.llvm.org> On Behalf Of James Henderson via cfe-dev
>Sent: Wednesday, October 6, 2021 1:47 AM
>To: Tanya Lattner <tanyal...@llvm.org>
>Cc: llvm-dev <llvm...@lists.llvm.org>; Renato Golin <reng...@gmail.com>; clang developer list <cfe...@lists.llvm.org>; openmp-dev (openm...@lists.llvm.org) <openm...@lists.llvm.org>; LLDB Dev <lldb...@lists.llvm.org>
>Subject: Re: [cfe-dev] [llvm-dev] RFC: Code Review Process
>
>
>WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>Forgive me if I'm wrong, but if the community consensus is that we should continue to use Phabricator, and Phabricator is not being provided/maintained by the LLVM Foundation, isn't it moot what the LLVM Foundation/Infrastructure Working Group recommends/wants to happen? The current maintainers would continue to maintain Phabricator (assuming they wanted to), and people would still be able to review things there. What would happen if the Foundation officially supported PRs, without community consensus (in particular from the Phabricator maintainers), is a potential split in the community, with some continuing in the old way and others using the new way (and presumably some choosing to review on both platforms). This cannot be good.
>
>I'm all for the discussion to be had, about whether we switch, but as far as I can see, nothing's really changed from the previous conversations on PRs versus Github, apart from the announcement of end of support by the upstream company, but that was quite a while ago now, and even with the stale Arcanist issue, there hasn't been a big push from community members to change: the consensus in the posts discussing this and the moving to PRs seems to still be "there are things that are blocking switching still".
>
>At the most, from this IWG/Foundation consultation, it should be that the Foundation recommends one or other approach, and is willing to provide X infrastructure required. The community can then choose to agree with whatever approach is recommended or stick with the status quo. There shouldn't be an edict that says we will do one thing or the other.
>
>Another side-point: whilst the IWG may consist of people who care about LLVM, there are far more people who care as much, but who just don't have the time to participate in such a group. This is particularly important to note, because the community does not elect members to this group. To an extent, the same is also true of the Foundation board itself, since there are plenty of people who may not agree with their decisions, but don't have the time to volunteer for the board. I'm not suggesting that there's any malice in this discussion, and indeed, the fact that it's open to community comments certainly is helpful, but I'd be worried of some kind of echo chamber/unconscious bias within the small groups suggesting there is consensus for one approach, when the wider community thinks otherwise.
>
>James
>

>On Tue, 5 Oct 2021 at 20:52, Tanya Lattner via llvm-dev <llvm...@lists.llvm.org<mailto:llvm...@lists.llvm.org>> wrote:
>Hello! The purpose of this email is to start a discussion about our code review tools. No decisions have been made about changing tools. The idea behind a timeline is so that information could be gathered in a timely manner. The Infrastructure Working Group was formed to bring together community members who have an experience and/or passion regarding infrastructure. Anyone can participate in this working group and like the LLVM Foundation, the minutes are all made public.
>
>The LLVM Foundation’s mission is to support the LLVM project and help ensure the health and productivity of of the community and this is done through numerous ways including infrastructure. I do not think it is a negative thing that the foundation board of directors would be discussing our current tools and gathering information how how well they work and how we can make them better. As the legal entity who bares financial and legal responsibility for a lot of the infrastructure, this would make sense. This also makes sense because of the people involved who care a lot about LLVM and the project. But, the LLVM Foundation does not pay for Phabricator and we are very grateful for Google’s support of this critical piece of our infrastructure.
>
>Regarding Phabricator, there are a couple of pieces of information that were provided to the LLVM Foundation by maintainers (maybe previous it sounds like) of this instance and how we may need to look into alternative ways to support it. In addition, Phacility itself has publicly stated that it is winding down operations. (https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_operations/). Lastly, there are questions about why we are not using GitHub pull requests as we are on GitHub and that might be the natural path to take for a number of reasons.
>
>The above reasons are why the RFC was written. Perhaps it wasn’t written in the best way, but I also feel like it is being read in a negative way which is incredibly disappointing given I don’t feel there is a valid reason for this.
>
>-Tanya
>
>
>
>
>
>
>

>On Oct 5, 2021, at 11:35 AM, Renato Golin via llvm-dev <llvm...@lists.llvm.org<mailto:llvm...@lists.llvm.org>> wrote:

>
>On Tue, 5 Oct 2021 at 19:16, Tom Stellard <tste...@redhat.com<mailto:tste...@redhat.com>> wrote:
>However, it's not a good position for the Board to be responsible
>for something that it doesn't have control over. If Google decided to stop hosting
>Phabricator for some reason (unlikely, but not impossible), the Board would be
>responsible for finding a replacement.
>
>Sorry, this is a very weak reason for such a strong worded "RFC".
>
>I _cannot_ imagine "Google" stopping to support something so quickly as to leave the foundation without recourse. And even if they did, *no one* would blame the foundation for that.
>
>Even if you ignore all the effort that hundreds of their engineers have done over the past decade to the project, this would hurt Google more than anyone else. It's a far fetched concern.
>
>And if the foundation wants "control" of a piece of infrastructure that Google has been maintaining for years, then this is a different discussion. Hopefully one that doesn't involve unilateral decisions.
>
>The main risk is that Phabricator is no longer maintained upstream.
>There was already an issue[1] recently where the arc tool stopped working and won't
>be fixed upstream. Using unmaintained software is a bigger risk.
>
>I don't like using unmaintained software either, but I think our Phab has had more attention than the upstream project. And no one has to use arc, I certainly never have.
>
>Don't get me wrong, I don't like Phab and I think Github would bring new people to the project, but it's gotta be done the right way, and pushing it isn't it.
>
>We, meaning the LLVM Board of Directors. And really the problem isn't the self-hosting
>so much as it's the lack of an enforceable maintenance agreement the Foundation and the
>maintainers.
>
>The problem isn't self-hosting at all, given that Google is doing that. (apologies, I assumed otherwise earlier).
>
>Neither is maintenance, given Google is doing that too.
>
>The only thing that's left is control, and I don't really understand why this is important, as I explained above.
>
>cheers,
>--renato
>_______________________________________________
>LLVM Developers mailing list

>llvm...@lists.llvm.org<mailto:llvm...@lists.llvm.org>

>https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>_______________________________________________
>LLVM Developers mailing list

>llvm...@lists.llvm.org<mailto:llvm...@lists.llvm.org>

[Chris Lattner via llvm-dev]

(Note: I’m speaking as an individual here, not with any particular organizational hat; FWIW I personally use both Phabricator and GitHub daily).

Various bits of LLVM infra have had organic roots, e.g. the mailing lists originally hosted at UIUC, Phabricator being set up by individuals, etc.  However, this has historically always become a problem: e.g. UIUC prevented us from having a wiki early on because of user-generated content rules, the mailing lists get overrun by spam, Phabricator becomes unmaintained.

The root issue here is that LLVM is a >20 year old project, and many of the technologies that have successfully propelled it to where it is evolve and change, and the people working on LLVM wander into and out of the project over the years.

Meanwhile, LLVM is critical infra for many teams, companies and products.  I consider it to be a serious strategic risk that we depend on infrastructure that is maintained by (well intentioned, and obvious incredibly valued) community volunteers: for example, SVN used to go down when the server filled up.  This simply isn’t a way to run a scalable project, and makes us very susceptible to the “bus factor”.  We as a project need to depend on maintained and hosted infra, not run it ourselves.

Furthermore, while community consensus is an important part of decision making in the LLVM project, it is clear that not every individual will be happy.  Further, from a project management perspective, we don’t want multiple ways to do patch reviews: we need a single standard solution here, this can’t practically be a matter of personal preference.

Coming back to the details at hand, there is a proposal to move to GitHub PR’s for a variety of reasons that other people have made.  It also seems that there are some serious gaps that we need to figure out before it is plausible replacement, which is what the thread is exploring.

This all seems like a good and healthy discussion to be having.  It seems that some folks are surprised by the premise of wanting to get on maintained and hosted infrastructure (e.g. SVN -> GitHub).  I thought this was an extremely clear direction for years now - and that we were problem solving towards the best way to do this.

-Chris

_______________________________________________

[Philip Reames via llvm-dev]

Since I think we're risking confusion on the point here, let me clarify that at least my response to this thread should not be read as opposition (or support) for a migration to github.  I am expressing no opinion on that matter.  I see the primary point being discussed in this thread being the decision making process proposed, not the decision itself.

Philip

[Raphael Isemann via llvm-dev]

Just two comments on the process here:

1. I don't think we can really make a 'wrong' decision regarding what
code review platform we use. While Github's review interface has its
own problems, it's not so bad that it would actually prevent people
from reviewing code. Having said that, the only way we could mess up
is if we end up with the situation that James describes, where we have
two groups in the community using different review platforms. And the
only way I can see this could happen is if we do the same thing we did
with Discord or Discourse where we just push some new system but we
don't have the consensus or critical mass to make everyone switch.
Both Discord and Discourse were IMHO good ideas in general and were
well intentioned, but the way we switched to them split & hurt the
community in the end. Given how critical code review is, I would
rather wait until we have enough consensus for the switch than just do
it and hope it works out well.

2. I do appreciate that the foundation/infrastructure group is trying
to improve our infrastructure, but I am not sure why the code review
system has become an apparently urgent issue. While Phabricator is
clearly going the way of the Dodo, from my own experience it at least
still works. Meanwhile both the mailing list and the bug tracker are
pretty much completely broken (the mailing list in the weird legal
spam limbo and the bugtracker having a disabled user registration). I
don't want to be the person that tells other people what to do, but if
we have resources/time to spend on fixing up our infrastructure, I
hope we could spend it on fixing the things that currently don't work.
Having a working bug tracker or some new mailing list system where I
(and other moderators) don't have to manually approve messages 24/7
and actually accept subscribers would be nice.

Thanks,
- Raphael

Am Mi., 6. Okt. 2021 um 19:34 Uhr schrieb Philip Reames via cfe-dev
<cfe...@lists.llvm.org>:

> _______________________________________________
> cfe-dev mailing list
> cfe...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

[Michael Kruse via llvm-dev]

Am Do., 7. Okt. 2021 um 05:46 Uhr schrieb Raphael Isemann via cfe-dev
<cfe...@lists.llvm.org>:

> 2. I do appreciate that the foundation/infrastructure group is trying
> to improve our infrastructure, but I am not sure why the code review
> system has become an apparently urgent issue. While Phabricator is
> clearly going the way of the Dodo, from my own experience it at least
> still works. Meanwhile both the mailing list and the bug tracker are
> pretty much completely broken (the mailing list in the weird legal
> spam limbo and the bugtracker having a disabled user registration). I
> don't want to be the person that tells other people what to do, but if
> we have resources/time to spend on fixing up our infrastructure, I
> hope we could spend it on fixing the things that currently don't work.
> Having a working bug tracker or some new mailing list system where I
> (and other moderators) don't have to manually approve messages 24/7
> and actually accept subscribers would be nice.

+1

As I already mentioned on GitHub[1], nothing really has changed since
the winding down of Phacility, Phabricator is continuing receiving
security patches. A project fork already exists [2].

[1] https://github.com/llvm/llvm-iwg/issues/73#issuecomment-936716573
[2] https://we.phorge.it/

[Reid Kleckner via llvm-dev]

On Tue, Oct 5, 2021 at 9:48 AM Renato Golin via cfe-dev <cfe...@lists.llvm.org> wrote:

On Tue, 5 Oct 2021 at 17:06, Tom Stellard via llvm-dev <llvm...@lists.llvm.org> wrote:

- Any other information that you think will help the Board of Directors make the best decision. 

- Foundation Board will have 30 days to make a final decision about using GitHub Pull Requests and then communicate a migration plan to the community.

Hi Tom,

Please help me here, I think I'm severely misunderstanding what this means...

I'm reading it that the "Board of Directors" will make a decision and communicate to the community, apparently through some undisclosed internal process.

...

I want to take the other side here, and say that I appreciate that the board is trying to provide more structure for this decision making process. I don't think the board is trying to step in and take ownership of the decision, they are trying to solicit input and reflect it back to LLVM developers in an efficient way. It has long been clear that LLVM needs a more effective process for building consensus and making decisions, and in the absence of that, the board came up with this ad hoc process. That seems very reasonable to me.

For everyone else who has suggestions on how we could run our infra better, my understanding is that you can get involved in the infrastructure working group by emailing i...@llvm.org.

[Renato Golin via llvm-dev]

On Thu, 7 Oct 2021 at 21:48, Reid Kleckner <r...@google.com> wrote:

I want to take the other side here, and say that I appreciate that the board is trying to provide more structure for this decision making process. I don't think the board is trying to step in and take ownership of the decision, they are trying to solicit input and reflect it back to LLVM developers in an efficient way. It has long been clear that LLVM needs a more effective process for building consensus and making decisions, and in the absence of that, the board came up with this ad hoc process. That seems very reasonable to me.

Ad-hoc isn't really sensible for such an important thing. We have done this before, so it's not lack of prior art either.

In every past similar situation it has been the consensus that the board does not decide on technical matters. They may help, organise, spend resources, gather information, build tools, but the ultimate decision is up to the community (whatever that means).

So far, the harder technical decisions (for example, migrating to Github), have been taken after enough consensus was built in the list and enough discussions happened in the conferences, until such a day the vast majority agreed it should be done.

There are three main pending issues:

 * Bugzilla, where everyone thinks we have to change but GH issues are nowhere near complete enough.

 * Phabricator, where we're mostly in favour of GH PRs, but there's still at least one major hurdle, patch sets.

 * Mailing list, where it's a pretty even split, with the IRC/Discord split being a major counter-example.

Hosting on github vs self-hosted was a small change, and most people were in favour, but the problem was mostly around monorepo vs submodules.

Starting a discord channel is not something people need "permission", but it did fragment the just-in-time interactions. Starting a Slack channel or whatever is the new thing would be the same problem, but nothing too terrible.

However, code review, technical discussions and bug tracking are pretty core to how we all interact, and we should not have more than one of any of those things. so, whatever decision is taken it will be a decision to *move*, not add.

This is a pretty serious decision, and I believe we'd have a lot less friction if we do in the same way we did the Github. Proposals, discussions, BoF sessions and a final decision when it's clear that the majority of the community is on board with the changes.

But to get there, we'll need to hash out all issues. Right now, the discussion is around patch sets, and until that gets sorted, we really shouldn't even try to use PRs. It may take less then 30 days, it may take more, but that discussion must happen in the list, not in a working-group or in the foundation board's meeting.

This is how we've always done it so far and it has been working well. At least most people I know think that this is better than most other alternatives, including ad-hoc decision making plans.

cheers,

--renato

[Vassil Vassilev via llvm-dev]

Hi Tom,

  Thank you for stepping up and encouraging discussion but also a timeline!

  I've been using GitHub Pull Requests and Phabricator for years. I
have been waiting for this switch since years. In my opinion Phabricator
has been holding us off for too long. I do not want to go into too much
details on features, web interface etc. I think both systems are good
for implementing a reasonable code review process. I want to point out
two major features which will introduce drastic improvements. Firstly,
preflight builds will reduce the
land,revert,reland,revert,reapply,revert style of development. Secondly,
GitHub is the choice for many (young?) developers who will have much
easier time to start contributing. Believe it or not the people I have
to introduce to LLVM development have never heard of Phabricator before
and struggle quite a bit...

Best, Vassil

> cfe-dev mailing list
> cfe...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

[David Blaikie via llvm-dev]

On Thu, Oct 7, 2021 at 2:22 PM Renato Golin via cfe-dev <cfe...@lists.llvm.org> wrote:

On Thu, 7 Oct 2021 at 21:48, Reid Kleckner <r...@google.com> wrote:

I want to take the other side here, and say that I appreciate that the board is trying to provide more structure for this decision making process. I don't think the board is trying to step in and take ownership of the decision, they are trying to solicit input and reflect it back to LLVM developers in an efficient way. It has long been clear that LLVM needs a more effective process for building consensus and making decisions, and in the absence of that, the board came up with this ad hoc process. That seems very reasonable to me.

Ad-hoc isn't really sensible for such an important thing. We have done this before, so it's not lack of prior art either.

I think the term "ad-hoc" was applied to the process, not the outcome. I don't think Reid's suggesting we'd end up with a "multiple different kinds of review process", but that we don't have a good formal process for decision making, so folks are experimenting with various ways to help move big, difficult decisions forward in a more reliable way.

I do agree that it's a bit surprising to me that the board is (trying to?) take a more authoritative responsibility over this decision. Though I'm not averse to it in some of these sort of infrastructure cases myself. Might've been better received if it came from the infrastructure working group instead, not sure.

 

In every past similar situation it has been the consensus that the board does not decide on technical matters. They may help, organise, spend resources, gather information, build tools, but the ultimate decision is up to the community (whatever that means).

So far, the harder technical decisions (for example, migrating to Github), have been taken after enough consensus was built in the list and enough discussions happened in the conferences, until such a day the vast majority agreed it should be done.

There are three main pending issues:

 * Bugzilla, where everyone thinks we have to change but GH issues are nowhere near complete enough.

 * Phabricator, where we're mostly in favour of GH PRs, but there's still at least one major hurdle, patch sets.

 * Mailing list, where it's a pretty even split, with the IRC/Discord split being a major counter-example.

Hosting on github vs self-hosted was a small change, and most people were in favour, but the problem was mostly around monorepo vs submodules.

Starting a discord channel is not something people need "permission", but it did fragment the just-in-time interactions. Starting a Slack channel or whatever is the new thing would be the same problem, but nothing too terrible.

However, code review, technical discussions and bug tracking are pretty core to how we all interact, and we should not have more than one of any of those things. so, whatever decision is taken it will be a decision to *move*, not add.

This is a pretty serious decision, and I believe we'd have a lot less friction if we do in the same way we did the Github. Proposals, discussions, BoF sessions and a final decision when it's clear that the majority of the community is on board with the changes.

But to get there, we'll need to hash out all issues. Right now, the discussion is around patch sets, and until that gets sorted, we really shouldn't even try to use PRs. It may take less then 30 days, it may take more, but that discussion must happen in the list, not in a working-group or in the foundation board's meeting.

This is how we've always done it so far and it has been working well. At least most people I know think that this is better than most other alternatives, including ad-hoc decision making plans.

I'm not sure I'd say it's been working well - it took a lot of time, a lot of volunteers and dragging some folks along in the end anyway. I think there's a lot of merit in having a more structured (honestly: faster) decision making process. We often end up in choice/analysis paralysis - where it's easier to object than to address objections, which is tiring for those trying to make change & slows down things quite a bit.

- Dave
 

[Renato Golin via llvm-dev]

On Thu, 7 Oct 2021 at 22:31, David Blaikie <dbla...@gmail.com> wrote:

This is how we've always done it so far and it has been working well. At least most people I know think that this is better than most other alternatives, including ad-hoc decision making plans.

I'm not sure I'd say it's been working well - it took a lot of time, a lot of volunteers and dragging some folks along in the end anyway. I think there's a lot of merit in having a more structured (honestly: faster) decision making process. We often end up in choice/analysis paralysis - where it's easier to object than to address objections, which is tiring for those trying to make change & slows down things quite a bit.

Right, "working well" can mean multiple things... :)

Most people I spoke about this over the years think that it's still better than other models, like "benevolent" dictator, "meritocratic" authority, diverse types of voting systems, elected "officials", etc. 

There are plenty of big projects that follow those models and ours seems to be the most inclusive and open to diversity, but not the most efficient. I think that side of our community still attracts a lot of people from all over the world and is an identifying trait.

Personally, I prefer to be in the diverse end of the spectrum than on the efficient end, mostly because the efficient ones tend to be autocratic in nature.

But this is mostly on the technical but not code decisions. Code decisions I think we can be pretty efficient.

cheers,

--renato

[David Blaikie via llvm-dev]

On Thu, Oct 7, 2021 at 3:02 PM Renato Golin <reng...@gmail.com> wrote:

On Thu, 7 Oct 2021 at 22:31, David Blaikie <dbla...@gmail.com> wrote:

This is how we've always done it so far and it has been working well. At least most people I know think that this is better than most other alternatives, including ad-hoc decision making plans.

I'm not sure I'd say it's been working well - it took a lot of time, a lot of volunteers and dragging some folks along in the end anyway. I think there's a lot of merit in having a more structured (honestly: faster) decision making process. We often end up in choice/analysis paralysis - where it's easier to object than to address objections, which is tiring for those trying to make change & slows down things quite a bit.

Right, "working well" can mean multiple things... :)

Most people I spoke about this over the years think that it's still better than other models, like "benevolent" dictator, "meritocratic" authority, diverse types of voting systems, elected "officials", etc. 

There are plenty of big projects that follow those models and ours seems to be the most inclusive and open to diversity, but not the most efficient. I think that side of our community still attracts a lot of people from all over the world and is an identifying trait.

I don't think diversity necessarily relates to this aspect of managerial structure. Unless we're talking about the less benevolent dictatorships where the authority figures both provide structure, but also set some fairly negative tones for how people should relate. Those things aren't necessarily connected though, and I don't see signs that's the kind of leadership we have or are moving towards in the LLVM community. 
 

But this is mostly on the technical but not code decisions. Code decisions I think we can be pretty efficient.

I think the code decisions also have some problems in LLVM, for what it's worth - "yes" is moderately easy (usually if someone feels empowered enough to say "yes" it's because they are, the rest of the community is comfortable with it, and you go forward with the work that's been approved), but "no" is hard to come by - absence of feedback/clear sense of ownership and authority can make it quite difficult to figure out how to approach an issue. Who's empowered to approve or disapprove of a patch that's on the edges of acceptability is quite often unclear (with various power vacuums opening up as key contributors/project founders have moved on to other focus areas), and what steps would be necessary/sufficient to make forward progress may be less clear still. Steps like the Contentious Decisions Process I think is a step to help mitigate some of that, though there's still a lot of gaps.

- Dave

[Geoffrey Martin-Noble via llvm-dev]

On Thu, Oct 7, 2021 at 2:32 PM David Blaikie via llvm-dev <llvm...@lists.llvm.org> wrote:

I think the term "ad-hoc" was applied to the process, not the outcome. I don't think Reid's suggesting we'd end up with a "multiple different kinds of review process", but that we don't have a good formal process for decision making, so folks are experimenting with various ways to help move big, difficult decisions forward in a more reliable way.

There is actually a formal LLVM decision making process: https://github.com/llvm/llvm-www/blob/master/proposals/LP0001-LLVMDecisionMaking.md. It has only been used a few times before, but may be worth using here.

On Thu, 7 Oct 2021 at 22:31, David Blaikie <dbla...@gmail.com> wrote:

I'm not sure I'd say it's been working well - it took a lot of time, a lot of volunteers and dragging some folks along in the end anyway. I think there's a lot of merit in having a more structured (honestly: faster) decision making process. We often end up in choice/analysis paralysis - where it's easier to object than to address objections, which is tiring for those trying to make change & slows down things quite a bit.

I think this is pretty key, especially with infrastructure work. Infrastructure tends to be neglected because it's rarely the core of the project. And since it affects developer workflows, it tends to be contentious. It is very easy for people to come up with objections or muddy things enough that no progress is made at all.

On Wed, Oct 6, 2021 at 1:46 AM James Henderson via llvm-dev <llvm...@lists.llvm.org> wrote:

Another side-point: whilst the IWG may consist of people who care about LLVM, there are far more people who care as much, but who just don't have the time to participate in such a group.

I would push back pretty strongly against this. Obviously the entire community should be able to voice its opinions, but at some point some people are the ones who are actually implementing things. It doesn't matter if the community comes to some consensus if no one is actually willing to build that thing. Again, and especially with infrastructure, it is very easy for people to complain about any solution, but much more effort to actually contribute to the one they prefer.

[Renato Golin via llvm-dev]

On Thu, 7 Oct 2021 at 23:16, David Blaikie <dbla...@gmail.com> wrote:

I don't think diversity necessarily relates to this aspect of managerial structure. Unless we're talking about the less benevolent dictatorships where the authority figures both provide structure, but also set some fairly negative tones for how people should relate. Those things aren't necessarily connected though, and I don't see signs that's the kind of leadership we have or are moving towards in the LLVM community. 

Sorry, that's not at all what I meant.

LLVM attract all kinds of people, not just from different backgrounds and minorities, but also different cultures. And by culture I mean a lot of things.

We have different countries and continents; academia, enterprise and government; students, professionals, directors; enthusiasts or people just trying to make some money; open and closed source source projects; embedded into or built as a library or being used by a dependency. I myself have belonged to many of those groups over the years.

In my opinion, that variety in how we all use and rely on LLVM is key to its success, but it's also what makes it hard to drive larger changes that affect the least amount of people.

Even foundations and working groups can't be representative of all people and most of the time we don't even know who "the people" are until we try to change something and it breaks for them.

This is why long consensus "works" for us, because eventually by then, most people would have seen it and voiced their opinions. But it's slow and painful.

I personally prefer that pain, then the pain of seeing each new decision alienating a small, but substantial, part of the community, and making the project less and less palatable to new contributors from different cultures.

[David Blaikie via llvm-dev]

Not making changes (or making them especially slowly) can also exclude people, which is one of the things we're grappling with in this decision too (every patch that comes in through a pull request and is automatically rejected - where that contributor doesn't then go and do all the extra work of figuring out phab, creating an account, etc, is lost new contributions/contributors, for instance)

I don't think the longer process we've used in the past necessarily created higher quality decisions - I think moving to github, preserving commit/author history, and maintaining a linear version history might've been a decision that could've been made (& would likely have been made, moreso than other options that were considered) relatively quickly, for instance.

All this said - I think the place where the IWG/board can be most helpful is to lower the costs of dealing with the infrastructure problems - in the github migration, figuring out tooling to preserve history/manage the migration and enforce the linear version history, etc - that took time to consult with github, build scripts and test them, etc. Doing that pre-emptively/more aggressively and being able to demonstrate a path forward to the community would probably be pretty helpful.

Similarly, taking the feedback from a review like this around github pull requests for review, and using whatever weight/consulting/paid consultants/contractors/etc might be available to help implement any feasible feature improvements to smooth the transition might be quite helpful. (tricky to frame that as "exploratory"/lowering barriers without it being "we've already decided the direction and the community will just have to accept it" is tricky, and how much of that sort of work is worth doing without it being clear that the work will be used/will pay off in the end - that's complicated) But hopefully a general "what're the concerns, how broad reaching are they/how many people have them" and then some time to figure out how feasible addressing each of them is, what sort of timeframe, etc, seems like a place to start.

- Dave

 

[Keane, Erich via llvm-dev]

I’m concerned that by spending our limited-infrastructure-repair time on changing one of the tools that work reasonably well, we are not concentrating on what is actually important.  Our bug tracker is both disorganized and functionally-impossible to join.

 

Filing a bug into Bugzilla with the ‘temporary’ filtering of new users (mixed with an arduous and error-fraught system to get into it!) is multiple orders of magnitude less inclusive than a slightly-less-familiar review tool.  I’ve brought about a half-dozen junior devs (plus numerous larger ones!) into the LLVM development community, and none have had a problem with picking up Phab. 

 

Filing a bug on the other hand…

 

From: cfe-dev <cfe-dev...@lists.llvm.org> On Behalf Of Renato Golin via cfe-dev
Sent: Thursday, October 7, 2021 3:44 PM
To: David Blaikie <dbla...@gmail.com>
Cc: llvm-dev <llvm...@lists.llvm.org>; clang developer list <cfe...@lists.llvm.org>; openmp-dev (openm...@lists.llvm.org) <openm...@lists.llvm.org>; LLDB Dev <lldb...@lists.llvm.org>
Subject: Re: [cfe-dev] [llvm-dev] RFC: Code Review Process

 

On Thu, 7 Oct 2021 at 23:16, David Blaikie <dbla...@gmail.com> wrote:

[Tom Stellard via llvm-dev]

On 10/5/21 9:05 AM, Tom Stellard wrote:
> Hi,
>
> # Proposal
>
> The LLVM Foundation Board of Directors is seeking comment on the current state of Code Review
> within the LLVM Project and its sub-projects.   Phabricator is no longer actively maintained
> and we would like to move away from a self-hosted solution, so our goal is to determine if
> GitHub Pull Requests are a good alternative to our current code review tool: Phabricator.
>
> Specifically we are looking for feedback on:
> - What features or properties make Github Pull Requests better than Phabricator?
> - What features or properties  make Phabricator better than GitHub Pull Requests?
> - What new workflows or process improvements will be possible with GitHub Pull Requests?
> - Which workflows aren’t possible with GitHub Pull Requests?
> - Any other information that you think will help the Board of Directors make the best decision.
>
> # Where to Direct Feedback
>
> Please provide feedback on the Infrastructure Working Group ticket[1].  This will make
> it easier to collect and consolidate the responses.   At the end of the comment period
> the Infrastructure Working Group will collect the feedback for further analysis and summarization.
>
> # Timeline
>
> The timeline for this RFC will be as follows:
>
> - RFC posted on llvm-dev for public review and comment
> - 30 days after the date of posting, public comment closes.

Hi,

Just a reminder, please add your comments to the github issue[1] by
November 4.

-Tom

> - IWG will have 14 days from closure of public comments to review and summarize public
>   comments into a pros and cons list to be present to LLVM Foundation Board
> - Foundation Board will have 30 days to make a final decision about using GitHub Pull Requests
>   and then communicate a migration plan to the community.
>
> Thank you,
> LLVM Foundation Board of Directors
>
> [1] https://github.com/llvm/llvm-iwg/issues/73

_______________________________________________