[llvm-dev] Anyone doing code reviews via mailing lists?

111 views
Skip to first unread message

Christian Kühnel via llvm-dev

unread,
Apr 23, 2021, 5:50:13 AM4/23/21
to llvm-dev
Hi folks,

I just read in our Code Review policy, that contributors can also do the code reviews on the mailing lists. I checked the XXX-commits mailing list archives since Feb 1st 2021 and could not find any code review there. I only found automatic emails from Phabricator reviews or git commits.

Is anyone actually doing code reviews on the mailing list any more? 
Can you send me a link to a recent example?

If nobody is doing code reviews on the mailing lists any more, we could also remove it from our policy...


Best,
Christian

Philip Reames via llvm-dev

unread,
Apr 23, 2021, 10:47:22 AM4/23/21
to Christian Kühnel, llvm-dev

It's not uncommon to have email responses to phab emails that don't make it into the web interface.  A recent example is D99976.

There's also a bunch of post commit discussion which happens entirely in email.  Check any of the commit threads with responses in the last week.  There are many.

Philip

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Christian Kühnel via llvm-dev

unread,
Apr 23, 2021, 11:25:11 AM4/23/21
to Philip Reames, llvm-dev
Thx Philip,

for sharing this. That helped me better understand how people are interacting with the mailing lists!

It's not uncommon to have email responses to phab emails that don't make it into the web interface.  A recent example is D99976.

I missed those when scanning the lists! I was looking for a code review that *started* on the mailing list, but that assumption was wrong. 

So it looks like the information we have on the mailing list and in Phabricator is diverging, as those emails do not get parsed back into Phabricator. Which is unfortunate as folks not following the llvm-commits mailing lists (like me) will then not see discussions happening there.

However I do understand that it's sometimes more convenient to reply via email client than via Phabricator. It looks like Phabricator would actually support this. Not sure how difficult it would be to set that up.

There's also a bunch of post commit discussion which happens entirely in email.  Check any of the commit threads with responses in the last week.  There are many.

Same here: I missed the replies to commit messages. Here's one example in case someone else is interested.

Discussions about commits could also happen on Phabricator. You can also reply to commits there, e.g. https://reviews.llvm.org/rG2f67267a93c87261414a4aa4c6cb9d20a489a0df But again I understand it is more convenient to do that via an email client.

My conclusions:
  1. Code reviews don't usually *start* on the mailing list.
  2. Folks do use the mailing lists to reply to commit and review emails, so we still need those. 
  3. For the future we might want to look into parsing these replies back into Phabricator to get a consistent view there. Not sure how hard that would be to set up.


Best,
Christian

    

Philip Reames via llvm-dev

unread,
Apr 23, 2021, 11:33:55 AM4/23/21
to Christian Kühnel, llvm-dev

JFYI, our copy of phabricator carries a bunch of custom patches for email integration.  It's an area we put a lot of effort into in the past.  The case I pointed to below just happens to be one in the long tail where that integration didn't catch the response.

Philip

Hubert Tong via llvm-dev

unread,
Apr 23, 2021, 11:57:58 AM4/23/21
to Christian Kühnel, llvm-dev
On Fri, Apr 23, 2021 at 11:25 AM Christian Kühnel via llvm-dev <llvm...@lists.llvm.org> wrote:
So it looks like the information we have on the mailing list and in Phabricator is diverging, as those emails do not get parsed back into Phabricator.

I recall noticing that Phabricator also doesn't emit inline code change suggestions into the e-mail record.

Mehdi AMINI via llvm-dev

unread,
Apr 23, 2021, 12:01:52 PM4/23/21
to Hubert Tong, llvm-dev
Yes: which is an indication that not all the content is on the mailing-list either :(

-- 
Mehdi

Krzysztof Parzyszek via llvm-dev

unread,
Apr 23, 2021, 12:55:39 PM4/23/21
to llvm-dev

I think we should phase-out email reviews in favor of doing it directly via Phabricator.

 

My reasons:

  1. Phabricator allows for both pre- and post-commit reviews.  You can “raise concern” with any commit, including those that did not have a pre-commit review on phab.
  2. The email-phabricator integration is still deficient despite a lot of effort having been put into it.  Moreover, it’s not likely that it will ever be fully functional.
  3. Email communication is “fragile”. This one is based more on my personal experience, but even simply following a discussion on llvm-dev has been difficult.  I now have to use Outlook (corporate reasons…) and Outlook fails at keeping email threads together.  Replies to the same tread are scattered into a mini-forest instead staying as a single tree.  There are issues with every email client formatting the replies differently: top-post mixed with bottom-post mixed with inline text, with people quoting 500 lines of text only to insert a single-line response, and so on, and so forth.  Some of it is due to my use of Outlook, some of it is independent from it.  This “infinite flexibility” of email structure is the reason why I doubt that the Phabricator integration will ever work.
  4. Phabricator’s interface makes every review look the same, is readable, and doesn’t make it easy to unintentionally clutter it with junk.
    1. Finally, it seems like nowadays it’s easier to create a Phabricator account than to sign up to the mailing lists…

 

 

 

--

Krzysztof Parzyszek  kpar...@quicinc.com   AI tools development

Philip Reames via llvm-dev

unread,
Apr 23, 2021, 1:26:10 PM4/23/21
to Krzysztof Parzyszek, llvm-dev

If you want to propose this, please move it to a top level thread for visibility. 

(My thoughts inline to help you refine your proposal.)

On 4/23/21 9:55 AM, Krzysztof Parzyszek via llvm-dev wrote:

I think we should phase-out email reviews in favor of doing it directly via Phabricator.

 

My reasons:

  1. Phabricator allows for both pre- and post-commit reviews.  You can “raise concern” with any commit, including those that did not have a pre-commit review on phab.
  2. The email-phabricator integration is still deficient despite a lot of effort having been put into it.  Moreover, it’s not likely that it will ever be fully functional.
  3. Email communication is “fragile”. This one is based more on my personal experience, but even simply following a discussion on llvm-dev has been difficult.  I now have to use Outlook (corporate reasons…) and Outlook fails at keeping email threads together.  Replies to the same tread are scattered into a mini-forest instead staying as a single tree.  There are issues with every email client formatting the replies differently: top-post mixed with bottom-post mixed with inline text, with people quoting 500 lines of text only to insert a single-line response, and so on, and so forth.  Some of it is due to my use of Outlook, some of it is independent from it.  This “infinite flexibility” of email structure is the reason why I doubt that the Phabricator integration will ever work.
Phabricator is also fragile unfortunately.  I find phabricators inline discussions to be very hard to follow for anything which becomes involved.  As such, I tend to default to phab, but quickly move to email if discussion gets complicated.


  1. Phabricator’s interface makes every review look the same, is readable, and doesn’t make it easy to unintentionally clutter it with junk.

As someone with vision restrictions, please be careful about this line of argument.  One of the major advantages of email is that I can use my own client at whatever zoom/scale I want.  Phab "sorta works" from an accessibility perspective, but frankly is inferior to plain old email.  I frequently end up reading phabricator emails, and then replying through the web interface. 

I'm just point this out because I find that visual appear is often rated much more highly by some folks than others.  Personally, functionality is pretty much the only thing I care about. 

    1. Finally, it seems like nowadays it’s easier to create a Phabricator account than to sign up to the mailing lists…

 

 

 

--

Krzysztof Parzyszek  kpar...@quicinc.com   AI tools development

 

From: llvm-dev <llvm-dev...@lists.llvm.org> On Behalf Of Mehdi AMINI via llvm-dev
Sent: Friday, April 23, 2021 11:01 AM
To: Hubert Tong <hubert.rein...@gmail.com>
Cc: llvm-dev <llvm...@lists.llvm.org>
Subject: [EXT] Re: [llvm-dev] Anyone doing code reviews via mailing lists?

 

 

 

On Fri, Apr 23, 2021 at 8:57 AM Hubert Tong via llvm-dev <llvm...@lists.llvm.org> wrote:

On Fri, Apr 23, 2021 at 11:25 AM Christian Kühnel via llvm-dev <llvm...@lists.llvm.org> wrote:

So it looks like the information we have on the mailing list and in Phabricator is diverging, as those emails do not get parsed back into Phabricator.

 

I recall noticing that Phabricator also doesn't emit inline code change suggestions into the e-mail record.

 

Yes: which is an indication that not all the content is on the mailing-list either :(

 

-- 

Mehdi

 


Krzysztof Parzyszek via llvm-dev

unread,
Apr 23, 2021, 1:58:05 PM4/23/21
to Philip Reames, llvm-dev

Thanks for the feedback.  I didn’t take accessibility concerns into account.  I’ll make sure to mention it if I decide to create a proposal (which seems likely).

 

--

Krzysztof Parzyszek  kpar...@quicinc.com   AI tools development

 

Chris Lattner via llvm-dev

unread,
Apr 23, 2021, 2:08:00 PM4/23/21
to Krzysztof Parzyszek, llvm-dev
I think it makes sense to phase out the email review path as well - it is better to have a single way to do things, and Phabricator is where our center of gravity is.

-Chris

Renato Golin via llvm-dev

unread,
Apr 23, 2021, 4:35:26 PM4/23/21
to Chris Lattner, llvm-dev
Yeah, I can't remember the last time I did a review by email. But I'm sure that they were replies to a commit that broke our bots, and not an actual review of a pull request.

Worse, I think if someone does a review by email and doesn't copy me directly, I won't even notice. 

Martin Storsjö via llvm-dev

unread,
Apr 23, 2021, 4:43:49 PM4/23/21
to Renato Golin, llvm-dev
On Fri, 23 Apr 2021, Renato Golin via llvm-dev wrote:

> Yeah, I can't remember the last time I did a review by email. But I'm sure
> that they were replies to a commit that broke our bots, and not an actual
> review of a pull request.
> Worse, I think if someone does a review by email and doesn't copy me
> directly, I won't even notice. 

Especially for cases where a commit didn't go through review on
Phabricator, it's probably common to react to it via e.g. a reply to the
commit mail, as it both goes to the list and hopefully reaches the author
that way.

// Martin

James Henderson via llvm-dev

unread,
Apr 26, 2021, 4:01:16 AM4/26/21
to Martin Storsjö, llvm-dev
As an interesting coincidence and data point, my sony.com email just got CC'ed to a new patch request, posted directly on llvm-commits. I'm guessing the email  actually bounced or got stuck in some sort of moderation queue, as I haven't seen it appear in the archives to link to it directly. Anyway, it appears the user didn't know about Phabricator being the preferred method (not helped due to the previously mentioned documentation no doubt), so I've pointed them at it, along with the relevant llvm.org pages to help them out.

(+1 to the side point of switching entirely to Phabricator reviews by the way)

Krzysztof Parzyszek via llvm-dev

unread,
Apr 28, 2021, 1:32:36 PM4/28/21
to llvm-dev

Just FYI, we are working on an official proposal to deprecate email reviews.

 

--

Krzysztof Parzyszek  kpar...@quicinc.com   AI tools development

 

From: llvm-dev <llvm-dev...@lists.llvm.org> On Behalf Of James Henderson via llvm-dev
Sent: Monday, April 26, 2021 3:01 AM
To: Martin Storsjö <mar...@martin.st>
Cc: llvm-dev <llvm...@lists.llvm.org>
Subject: [EXT] Re: [llvm-dev] Anyone doing code reviews via mailing lists?

 

As an interesting coincidence and data point, my sony.com email just got CC'ed to a new patch request, posted directly on llvm-commits. I'm guessing the email  actually bounced or got stuck in some sort of moderation queue, as I haven't seen it appear in the archives to link to it directly. Anyway, it appears the user didn't know about Phabricator being the preferred method (not helped due to the previously mentioned documentation no doubt), so I've pointed them at it, along with the relevant llvm.org pages to help them out.

Krzysztof Parzyszek via llvm-dev

unread,
May 3, 2021, 2:39:12 PM5/3/21
to llvm-dev

 

--

Krzysztof Parzyszek  kpar...@quicinc.com   AI tools development

 

Reply all
Reply to author
Forward
0 new messages