Update on sheriff-assisted checkin-needed bugs

[Ryan VanderMeulen]

As many of you are aware, the sheriff team has been assisting with landing checkin-needed bugs for some time now. However, we've also had to deal with the fallout of a higher than average bustage frequency from them. As much as we enjoy shooting ourselves in the foot, our team has decided that we needed to tweak our process a bit to avoid tree closures and wasted time and energy.

Therefore, our team has decided that we will now require that a link to a recent Try run be provided when requesting checkin before we will land the patch. To be clear, this *ONLY* affects checkin-needed bugs where we're assisting with the landing. We have no desire to police what other developers do before pushing. As has always been the case, developers are expected to ensure that their patches have received adequate testing prior to pushing whether they are receiving our assistance or not.

Our team is also not going to dictate which specific builds/tests are required. We're not experts in your code and we'll defer to your judgment as to what counts as sufficient testing. As mentioned earlier today in another post, if in doubt, we do have a set of general best practices for Try that can be used as a guide [1]. We just want to ensure that patches have at least received some baseline level of testing before being pushed to production. We've been testing the water with this policy for the past couple weeks and have already seen a reduction in the number of backouts needed.

For those of you mentoring bugs for new contributors, please also keep this in mind in order to keep patches from being held up in landing. And consider vouching for Level 1 commit access to further empower those contributors!

Thanks!

-Ryan

[1] https://wiki.mozilla.org/Sheriffing/How:To:Recommended_Try_Practices

[Bobby Holley]

(Reducing the thread scope for the followup)

One issue I often run into is that the contributor doesn't have access to
try, and pushing it on their behalf disrupts the rhythm of the other things
I'm doing. If we go forward with this, can we also get some kind of
sheriff-assisted try push flag? Something like try-needed?

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Ed Morley]

Autoland should solve that use case :-)
https://bugzilla.mozilla.org/show_bug.cgi?id=657828

On 19 May 2014 22:01:25, Jonas Sicking wrote:
> Try-from-bugzilla would be awesome!
>
> / Jonas

[Bobby Holley]

Can we get a stopgap solution in the mean time?

How about this: If a sheriff comes across a checkin-needed bug without a
try push, _and_ the most-recent comment in the bug includes a try-chooser
string that the path author would have used if {s,}he had try access, the
sheriff can push to try on the author's behalf?

[David Burns]

It doesn't feel like a quick |hg qimport bz://123456; hg qref -m "<my
favourite try syntax>"; hg try| would really put a dent in a mentors
productivity. If someone has already put in the effort to update the bug
with try syntax why not just do 1 more step and push to try?

David

> <mailto:dev-pl...@lists.mozilla.org>

> https://lists.mozilla.org/listinfo/dev-platform
>
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org

> <mailto:dev-pl...@lists.mozilla.org>

> https://lists.mozilla.org/listinfo/dev-platform
>
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org

> <mailto:dev-pl...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/dev-platform
>
>
>
>
> _______________________________________________
> Sheriffs mailing list
> Sher...@mozilla.org
> https://mail.mozilla.org/listinfo/sheriffs

[Bobby Holley]

It's an extra interruption that's going to me make me marginally more
averse to mentoring bugs (in general, I would suggest that the mentee build
the try syntax using the syntax builder).

On Tue, May 20, 2014 at 12:46 PM, David Burns <dbu...@mozilla.com> wrote:

> It doesn't feel like a quick |hg qimport bz://123456; hg qref -m "<my
> favourite try syntax>"; hg try| would really put a dent in a mentors
> productivity. If someone has already put in the effort to update the bug
> with try syntax why not just do 1 more step and push to try?
>
> David
>
>
> On 20/05/2014 19:33, Bobby Holley wrote:
>
> Can we get a stopgap solution in the mean time?
>
> How about this: If a sheriff comes across a checkin-needed bug without a
> try push, _and_ the most-recent comment in the bug includes a try-chooser
> string that the path author would have used if {s,}he had try access, the
> sheriff can push to try on the author's behalf?
>
>
> On Tue, May 20, 2014 at 2:01 AM, Ed Morley <emo...@mozilla.com> wrote:
>
>> Autoland should solve that use case :-)
>> https://bugzilla.mozilla.org/show_bug.cgi?id=657828
>>
>>
>> On 19 May 2014 22:01:25, Jonas Sicking wrote:
>>
>>> Try-from-bugzilla would be awesome!
>>>
>>> / Jonas
>>>
>>> On Mon, May 19, 2014 at 1:53 PM, Bobby Holley <bobby...@gmail.com>

>>> wrote:
>>>
>>>> (Reducing the thread scope for the followup)
>>>>
>>>> One issue I often run into is that the contributor doesn't have access
>>>> to
>>>> try, and pushing it on their behalf disrupts the rhythm of the other
>>>> things
>>>> I'm doing. If we go forward with this, can we also get some kind of
>>>> sheriff-assisted try push flag? Something like try-needed?
>>>>
>>>>
>>>> On Fri, May 16, 2014 at 1:54 PM, Ryan VanderMeulen <

>>>>> https://lists.mozilla.org/listinfo/dev-platform
>>>>>
>>>>> _______________________________________________
>>>> dev-platform mailing list
>>>> dev-pl...@lists.mozilla.org
>>>> https://lists.mozilla.org/listinfo/dev-platform
>>>>
>>> _______________________________________________
>>> dev-platform mailing list
>>> dev-pl...@lists.mozilla.org

>>> https://lists.mozilla.org/listinfo/dev-platform
>>>
>>
>
>
> _______________________________________________
> Sheriffs mailing listSheriffs@mozilla.orghttps://mail.mozilla.org/listinfo/sheriffs
>
>
>

[Ryan VanderMeulen]

> One issue I often run into is that the contributor doesn't have access to
> try, and pushing it on their behalf disrupts the rhythm of the other things
> I'm doing.

>From http://www.mozilla.org/hacking/commit-access-policy/

Level 1 - Try/User/Incubator Access
Because this is all it gives, this sort of access can be given out generously to anyone who would find it convenient when helping us or working on a developer's personal project, without worrying about them affecting core code. In other words, the target audience for this sort of access might be defined as "friends of and collaborators with Mozilla".

At least to me, that reads as "vouch early and vouch often!". Something something...teach a man to fish...something something... :)

-Ryan

[Steve Fink]

On Wed 21 May 2014 08:42:28 AM PDT, Ryan VanderMeulen wrote:
> Level 1 - Try/User/Incubator Access
> Because this is all it gives, this sort of access can be given out generously to anyone who would find it convenient when helping us or working on a developer's personal project, without worrying about them affecting core code. In other words, the target audience for this sort of access might be defined as "friends of and collaborators with Mozilla".
>
> At least to me, that reads as "vouch early and vouch often!". Something something...teach a man to fish...something something... :)

I think the quote you're looking for is, "if you teach a man to fish,
you'd better teach him how to gut and clean the fish at the same time."
Otherwise you'll be forever stuck doing it for him.

Or not.

[Bobby Holley]

Is it really the most effective learning experience and use of everyone's
time to make first-patch contributors get set up with try access?

I try to mentor as many bugs as possible. My ideal workflow would be to
grant r+, suggest a try: string, and set checkin-needed in a single act,
without having to determine whether the contributor has try access and/or
editbugs. If we already have people scanning for checkin-needed and looking
for try pushes, it seems pretty logical to have them just trigger any
missing pushes.

bholley

[Mike Conley]

>> I try to mentor as many bugs as possible. My ideal workflow would be to
>> grant r+, suggest a try: string, and set checkin-needed in a single act,
>> without having to determine whether the contributor has try access and/or
>> editbugs. If we already have people scanning for checkin-needed and looking
>> for try pushes, it seems pretty logical to have them just trigger any
>> missing pushes.

Or, alternatively, attempt to automate this with Autoland
(https://bugzilla.mozilla.org/show_bug.cgi?id=657828).

I think we should lean on that for this use case, personally.

[Chris Peterson]

On 5/21/14, 1:51 PM, Mike Conley wrote:
> Or, alternatively, attempt to automate this with Autoland
> (https://bugzilla.mozilla.org/show_bug.cgi?id=657828).

Is anyone actively working on Autoland? Rail had been working on
Autoland, but when I spoke with him in 2013 Q4, I think he said he would
not have time to work on it in 2014 Q1.

For a tool as important and often requested as Autoland, we should get
it on someone's schedule. :)


chris

[Ehsan Akhgari]

I think Taras knows more details.

[Taras Glek]

Ehsan Akhgari wrote:
> On 2014-05-21, 5:15 PM, Chris Peterson wrote:

> I think Taras knows more details.

I expect some exciting stuff in
https://bugzilla.mozilla.org/show_bug.cgi?id=1005235 soon

[Ryan VanderMeulen]

Just as a quick follow-up to this - we're already seeing much lower checkin-needed backout rates since this change went into affect, so thank you all for your help!

-Ryan

[Gijs Kruitbosch]

Playing devil's advocate for a bit - are there more non-checkin-needed
backouts? That is, people who, err, feel it is unnecessary to push to
try to land something with checkin-needed, and therefore then land it
themselves and burn the tree? :-)

And also: has the throughput in checkin-needed landings gone down, ie
are we trading this off against making it a teeny bit "harder" for
people to contribute? (not intending a value judgment there - it is
likely still the right thing to do, but it would be good to know)

~ Gijs