Gerrit email spam

[Pravin]

Hi All,

I am not sure if this has been brought up before, but as
the subject states Gerrit generates huge email traffic
which at times is not really useful or required. Hence
starting a discussion to look at possible ways to make
the email more useful especially for reviewers.
I welcome your comments/suggestions.

Few suggestions:
[Assumption: All the below suggestions assume that
authors still receive an email as happens presently]
[All the below suggestions can be over-riden by a new
global configuration setting or user watch preference.]
-------------------------------------------------------------------------
Comments:
Though mostly useful, at times the user may want to
just save the comments without sending it to everyone
on the change. Gerrit by default sends an email to
authors as well as reviewers i.e. everyone on the change.
(I have looked at the group setting for sending emails
to only authors, however it is useful only for automated
users since they can be put under there own group. If a
manual user decides to not trigger an email, there is no
option)

Suggestion:
We can split the button 'Publish Comments' into two:
1. 'Publish' - Save the comment
2. 'Publish and Send' - Save and send the comment
Same flexibility can be made available in gerrit
review command to ensure that the user is able to decide
whether the email goes out to reviewers or not.

--------------------------------------------------------------------------
Published Changes:
Drafts introduced in 2.3 enables the engineer to work on
changes till they are ready for review. However frequently
after the changes are published, they may get -2 and
changes cannot go back from 'Publish --> Drafts'. At this
point multiple iterations (multiple patchsets uploaded) may
take place to rectify the problem. Every such iteration
triggers an email to reviewers.

Suggestion:
By default Gerrit should not ask for review. Flexibility
should be present to request for review when the engineer
is again ready with the change.
A button 'Request For Review' can be placed alongside
'Review'. This button can be used by author to highlight
the change when they are comfortable for review. This
button can also facilitate adding a summary message to
be sent to all the reviewers.

A global configuration setting 'sendEmailToReviewers' can
decide if gerrit should ask for review on every uploaded
patchset. If the global setting is True, an email is sent
everytime a new patchset is uploaded (as it happens now)
or the button may be hidden.
If False, the button will be visible and an email goes out
only when the user clicks on 'Request For Review'

----------------------------------------------------------------------------
Submitting the change:
For a reviewer who has already given +1 or +2, s(he) will
not necessarily be interested in knowing whether merge
failed or a new patchset was uploaded to resolve the
conflict. It is again the authors who are interested and
should be receiving such emails.

Suggestion:
A global configuration setting 'sendEmailToReviewers' can
decide whether such emails are triggered to reviewers or
not.

----------------------------------------------------------
A new checkbox column 'Watch if reviewer' can be added on
'Settings-->Watched Projects'. If checked, this checkbox
will ensure that even if the global setting
'sendEmailToReviewers' is set to False, an email is sent
for every action on a change on which the user is present
as a reviewer.


That's all for my initial suggestions. I have created
mock-ups for the above suggestions but didn't find a way to
include them with the text. I think the above suggestions
pretty much covers all the possible cases to make the email
more meaningful and also make the review process more
stream-lined (The email spam gets magnified manifold for
+1 reviewers who get introduced in the review process quite
early and remain till the end). I think many of you face the
same problem, but the above suggestions may differ from your
setups. Please evaluate the suggestions in the light of your
own requirements. I look forward to hearing more from you.


Thanks

[Scott Lavender]

Pravin,

We have been having internal discussion about the amount of emails coming from our Jenkins integration. Whenever one of our Gerrit triggered Jenkins jobs runs, it's post to Gerrit causes an email, both for the start of the job as well as the completion(success/failure). When a user subscribes to a project, it would be nice if the granularity of these email could be configured. Developers are only interested in if the validation job fails...

--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

--

Scott Lavender | Cloud Release Engineer
_____________________________________ _______

Google Voice 678.744.8194
slav...@internap.com • www.internap.com

INTERNAP ®
connectivity | colocation | managed hosting | cloud

One Ravinia Drive • Suite 1300 • Atlanta • GA • 30346

[Pursehouse, David]

> For a reviewer who has already given +1 or +2, s(he) will
> not necessarily be interested in knowing whether merge
> failed or a new patchset was uploaded to resolve the
> conflict. It is again the authors who are interested and
> should be receiving such emails.
>

In what case would a reviewer not want to know that a new patchset has been uploaded and (s)he needs to approve (or reject) it again? It seems to me that suppressing those notifications would only cause delay in getting the change re-reviewed.

--
David Pursehouse
Configuration Manager
Sony Mobile Communications Japan, Inc.

[Pursehouse, David]

> Whenever one of our Gerrit triggered Jenkins jobs runs, it's post to
> Gerrit causes an email, both for the start of the job as well as the
> completion(success/failure). When a user subscribes to a project, it
> would be nice if the granularity of these email could be configured.
> Developers are only interested in if the validation job fails...
>

This gets very spammy when the change triggers 5 or more builds...

It would be nice if it could be configured so that the notification mails for comments from Jenkins (or any other user) are only sent when I'm the owner of the change. If I'm only a reviewer on someone else's change, I'm not interested in them.

[Pravin]

There is a group setting which ensures that comments from
users belonging to a particular group are only sent to authors.
Though Jenkins can be put under its own group since it
comes under automated user, other manual users cannot be
part of the same group.

This is where splitting 'Publish Comments' into two buttons:
'Publish' & 'Publish & Send' helps. These same options can
be replicated for command line comments through 'gerrit review'
command i.e. '--publish', '--publishandsend'.

[Pravin]

> In what case would a reviewer not want to know that a new patchset has been uploaded and (s)he needs to approve (or reject) it again?   It seems to me that suppressing those notifications would only cause delay in getting the change re-reviewed.
>

Let's say a change has been reviewed and granted +1, +2.
In a setup where some other team is responsible to submit
and the change fails to submit due to conflicts, a new
patch-set has to be uploaded. If conflict resolution is not
significant and it's a small change, it does not add much
value in sending email to all the reviewers and requesting
for review again. If the conflict resolution is significant, then
yes review should be requested. Having the flexibility to
request for review helps a lot. Imagine the amount of email
which a reviewer receives if he is on the reviewer list of a
lot many changes.

Another situation, imagine a change which was published
and received a -2. An engineer might need multiple iterations
to be comfortable for review again. For each iteration if a
review is requested, a reviewer unnecessarily receives an
email when the engineer is not even ready with the change.

Let's say a change is in published state and after a new
patch-set was uploaded, verification failed. To resolve this
an engineer will again upload a patch-set.

Sending an email without the engineer being ready for review
takes up unnecessary resources.

For all the above scenarios, it would have helped the reviewers
if they received an email only if the author wanted them to
review. To further ensure that existing Gerrit workflow does not
change, this can be achieved using a global configuration
setting 'sendEmailToReviewers'. If the setting is 'True', an email
is sent as usual. If the setting is 'False', an email is sent only
is someone requests for review.

Thanks

[Swindells, Thomas]

> Sending an email without the engineer being ready for review takes up
> unnecessary resources.
>
> For all the above scenarios, it would have helped the reviewers if they
> received an email only if the author wanted them to review. To further
> ensure that existing Gerrit workflow does not change, this can be achieved
> using a global configuration setting 'sendEmailToReviewers'. If the setting is
> 'True', an email is sent as usual. If the setting is 'False', an email is sent only is
> someone requests for review.

I'm not sure it is as black and white as you make it.
Many times (but not always) I want to be informed of new changes even if the committer isn't actually ready for them to be reviewed.
This way there is an opportunity to nip issues in the bud before too much effort has been spent on them.
The submitter doesn't always have the necessary information to correctly make this decision.
Adding in extra required mandatory steps also runs the risk of people forgetting and a reviewer sitting there when it should be being reviewed.
Not all decisions to perform a review are taken by receiving the email - I often just look at my list of pending reviews, if we are trying to provide
a solution for authors being able to say when they want a review we need to look at the full picture. I've not used it but I thought drafts was
geared towards helping here, but perhaps some form of flag is also needed (and then more advanced email preferences based upon this flag).

Thomas


**************************************************************************************
This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postm...@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes. To protect the environment please do not print this e-mail unless necessary.

NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603 8808 40-00
**************************************************************************************

[Pravin]

> I'm not sure it is as black and white as you make it.
> Many times (but not always)  I want to be informed of new changes even if the committer isn't actually ready for them to be reviewed.
> This way there is an opportunity to nip issues in the bud before too much effort has been spent on them.
> The submitter doesn't always have the necessary information to correctly make this decision.
> Adding in extra required mandatory steps also runs the risk of people forgetting and a reviewer sitting there when it should be being reviewed.
> Not all decisions to perform a review are taken by receiving the email - I often just look at my list of pending reviews, if we are trying to provide
> a solution for authors being able to say when they want a review we need to look at the full picture. I've not used it but I thought drafts was
> geared towards helping here, but perhaps some form of flag is also needed (and then more advanced email preferences based upon this flag).
>

I agree with the points you have highlighted. I think the points I
have brought up
and the points you have cited are different aspects of same problem.
While
Gerrit addresses one aspect, it does not expose any way to address the
other.
I discussed internally with a number of reviewers and a lot of them
had the same
feedback that it takes good chunk of their time to look at changes
which are not
review ready (Drafts doesn't help after a change is published and has
received -2).

Gerrit should provide a way to address the other aspect by having a
global
setting which applies to all the repos or by having repo level (admin
controlled)
configuration.

[Pravin]

> I'm not sure it is as black and white as you make it.
> Many times (but not always)  I want to be informed of new changes even if the committer isn't actually ready for them to be reviewed.
> This way there is an opportunity to nip issues in the bud before too much effort has been spent on them.
> The submitter doesn't always have the necessary information to correctly make this decision.
> Adding in extra required mandatory steps also runs the risk of people forgetting and a reviewer sitting there when it should be being reviewed.
> Not all decisions to perform a review are taken by receiving the email - I often just look at my list of pending reviews, if we are trying to provide
> a solution for authors being able to say when they want a review we need to look at the full picture. I've not used it but I thought drafts was
> geared towards helping here, but perhaps some form of flag is also needed (and then more advanced email preferences based upon this flag).
>

Formatting:

[Chad Horohoe]

On Tue, May 15, 2012 at 8:00 AM, Swindells, Thomas <TSwin...@nds.com> wrote:
> I'm not sure it is as black and white as you make it.
> Many times (but not always)  I want to be informed of new changes even if the committer isn't actually ready for them to be reviewed.
> This way there is an opportunity to nip issues in the bud before too much effort has been spent on them.
> The submitter doesn't always have the necessary information to correctly make this decision.
> Adding in extra required mandatory steps also runs the risk of people forgetting and a reviewer sitting there when it should be being reviewed.
> Not all decisions to perform a review are taken by receiving the email - I often just look at my list of pending reviews, if we are trying to provide
> a solution for authors being able to say when they want a review we need to look at the full picture. I've not used it but I thought drafts was
> geared towards helping here, but perhaps some form of flag is also needed (and then more advanced email preferences based upon this flag).
>

I agree wholeheartedly. We've had similar complaints about Gerrit being
overly spammy at times. A good example was someone attempting a
merge a couple of times and it being rejected due to unmet dependencies.
Everyone on the review list ended up with about 5 messages saying
essentially the same thing. Yes, this is probably user error, but it's worth
mentioning.

At the same time, if we shut up certain e-mails I'm sure I'd get the opposite
complaint from other users who want to hear more and don't mind filtering
the volume.

I think the ideal solution would be allowing users more customization over
what sorts of e-mail they get so everyone can tailor their spam to their own
tolerance levels.

-Chad

[Pravin]

Trimming the list of suggestions to start somewhere...

Comments:
Though mostly useful, at times the user may want to just save the comments without sending it to everyone on the change. Gerrit by default sends an email to authors as well as reviewers i.e. everyone on the change.

(I have looked at the group setting for sending emails to only authors, however it is mostly useful only for automated users since they can be put under there own group. If a manual user decides to not trigger an email,

there is no option)

Suggestion:

'Publish Comments' button can be split into two:

1. 'Publish' - Save the comment
2. 'Publish and Send' - Save and send the comment

Since Publish seems to be a stronger word, the buttons can as well be renamed to 'Save', 'Save and Send', 'Save and Submit'
Same flexibility can be made available in gerrit review command i.e. '--publish', '--publishandsend' to ensure that the user is able to decide whether the email goes out to reviewers or not.

For users who will want to receive notification about such comments, 'Settings --> Watched Projects' can be modified to add one more column 'Watch if reviewer' or having a separate section on the same screen: 'Settings --> Watched Reviews'.
Let me know your thoughts.

[R. Tyler Croy]

On Tue, 15 May 2012, Pursehouse, David wrote:

> > Whenever one of our Gerrit triggered Jenkins jobs runs, it's post to
> > Gerrit causes an email, both for the start of the job as well as the
> > completion(success/failure). When a user subscribes to a project, it
> > would be nice if the granularity of these email could be configured.
> > Developers are only interested in if the validation job fails...
> >
> This gets very spammy when the change triggers 5 or more builds...
>
> It would be nice if it could be configured so that the notification mails for comments from Jenkins (or any other user) are only sent when I'm the owner of the change. If I'm only a reviewer on someone else's change, I'm not interested in them.

If you're using the Gerrit Trigger plugin for Jenkins, it will join the
triggered jobs at the end posting only one comment to the Change. We currently
have ~5 jobs triggered for every Change going into Gerrit.


This has turned out to be quite useful, especially for the reviewer as I will
not even consider a change until the Jenkins user comments on the change
indicating that all tests have psassed. Until then I don't bother looking at
changes unless specifically requested :)


- R. Tyler Croy
--------------------------------------
Code: http://github.com/rtyler
Chatter: http://twitter.com/agentdero
rty...@jabber.org

[Pursehouse, David]

> If you're using the Gerrit Trigger plugin for Jenkins, it will join the
> triggered jobs at the end posting only one comment to the Change. We
> currently have ~5 jobs triggered for every Change going into Gerrit.

The Trigger Plugin will consolidate the results of all the jobs into a single comment, but unless the default configuration is changed it will still send a separate mail for each build that is started. That setting must be done by the administrator, and is global. It cannot be set in the trigger config per job.

Also if you have jobs that run in "silent mode" and send a score and/or report message directly to Gerrit using the review command over ssh, these are not included in the summary comment.

[Scott Lavender]

David,

Can you help me to find this "global" Gerrit Trigger option?

[Pursehouse, David]

It’s in the management settings under “Gerrit Verified Commands”, but having looked at it again I don’t think it actually does what I thought.

 

For Started, Successful, Failed and Unstable there are default gerrit commands, e.g.:

 

    gerrit approve &lt;CHANGE&gt;,&lt;PATCHSET&gt; --message 'Build Started &lt;BUILDURL&gt; &lt;STARTED_STATS&gt;' --verified &lt;VERIFIED&gt; --code-review &lt;CODE_REVIEW&gt;

 

which will be sent to gerrit over ssh.

 

The defaults can be changed, but must still be valid gerrit commands.  If the setting is left empty, or contains an invalid command (which will cause the ssh call to fail) an error will be written to the log.

--