Need moar schedule

[Dimitri Glazkov]

Ian,

It appears we need another crank of the Big Gardening Scheduler
Machine. Can you help me figure that out?

webkit-gardening, if you have any specific scheduling requests, please
post them here.

:DG<

[Adrienne Walker]

2012/11/1 Dimitri Glazkov <dgla...@google.com>:

> webkit-gardening, if you have any specific scheduling requests, please
> post them here.

Can EST gardeners be considered PST gardeners and not non-PST gardeners?

-enne

[Kenneth Russell]

I know that request has been discussed before but honestly I would be
concerned about making that change. As a PST gardener, I find that all
I can usually do is stabilize the tree during the PST workday, and
then roll during off-PST hours. An EST gardener would realistically
not be able to work late enough to get the tree to a stable state.

I have had at least one very good partnership with an EST gardener who
kept the tree in good shape during most of the PST workday,
significantly and helpfully reducing the workload. Japan-time
gardeners have also been immensely helpful during more stressful
shifts, doing rolls when all that it was possible to do was get the
tree green during the PST workday.

My requests would be to reduce the shifts from 4 to 3 days -- the 4
day shifts are grueling -- and to recruit more people for WebKit
gardening duty.

-Ken

[Ian Fette (イアンフェッティ)]

This is now checked in outside of google3/ - anyone can run it.

svn://svn.chromium.org/chrome-internal/trunk/tools/build/scripts/tools/sheriff


As I'm not dealing with WebKit that much anymore, I'd really like to hand this off to someone else...

[Stephen White]

Actually I think EST gardeners are being considered PST right now.  My co-gardeners this shift were in STP and SYD.

Honestly, it kind of sucks either way:  if EST == PST, then the EST gardener is on their own into the PST peak hours, and there's missing coverage for 3-4 hours in the PST afternoon/evening.  If EST == non-PST, then we have no coverage for 8-10 APAC hours and things are often busted when the EST gardener starts in the morning.

I think I prefer it as it is now (the former).  Although it might be helpful for the PST gardener to have an EST partner, there's a much greater coverage hole overnight, which falls on the EST gardener in the morning.

Stephen

--

[Stephen Chenney]

For my shifts I've always been the non-PST gardener physically located in EST. I like it in that I start early and mostly fix the tree and then can back off a bit and get some other stuff done in the afternoon. Fortunately I've been paired with PST folks who tend to start a bit later in their day and cover the west coast evening and get the roll done.

I think that this leads to more tree redness for EMEA when they are not gardening, and APAC to a lesser extent, but I think it does minimize the total amount of pain.

Effectively there is always at least 12-16 hours of overlapping coverage with a 8-12 hour gap with no coverage. In every coverage period the first person gets the tree green and the second person maintains and rolls. It's quite efficient and not too unpleasant.

In my mind the "minimally red, maximum rolls" solution is to have 3 shifts (APAC, EMEA, PST) and achieve almost round-the-clock coverage. But that is more overhead for everyone, particularly EMEA where we have fewer WebKit people (I think). I do not see the benefits out-weighing the costs.

Cheers,

Stephen.

--
 
 

--

Stephen Chenney | Software Engineer | sche...@google.com | 404-314-1809

[Stephen White]

On Fri, Nov 2, 2012 at 10:26 AM, Stephen Chenney <sche...@google.com> wrote:

For my shifts I've always been the non-PST gardener physically located in EST. I like it in that I start early and mostly fix the tree and then can back off a bit and get some other stuff done in the afternoon. Fortunately I've been paired with PST folks who tend to start a bit later in their day and cover the west coast evening and get the roll done.

I think that this leads to more tree redness for EMEA when they are not gardening, and APAC to a lesser extent, but I think it does minimize the total amount of pain.

Effectively there is always at least 12-16 hours of overlapping coverage with a 8-12 hour gap with no coverage. In every coverage period the first person gets the tree green and the second person maintains and rolls. It's quite efficient and not too unpleasant.

I disagree.  The only way you get 16 hours of overlapping coverage (assuming 8 hour workday) is with non-overlapping shifts.  When EST == PST, this means e.g., 8AM-4PM EST, 1PM-9PM PST.

I have worked several hellish EST == PST shifts where things were majorly busted when I get in in the morning, with not enough time to sort it all out before PST landings ramp up.

With JST + EST, e.g., this is much more sane, e.g., even 9AM-5PM EST, 9AM-5PM JST are non-overlapping and give 16 hours coverage easily.

Stephen

[Drew Wilson]

Please note that I have relocated to Munich, so I should be considered a non-PST gardener. Looks like I just need to tweak webkit.rotation.json - what's the proper way to update the files under that SVN repository, do I just svn checkout, edit, then svn dcommit? I wasn't sure if they were part of a larger repository that I should be using a different toolset with...

:DG<

--

[Dimitri Glazkov]

Congrats! I heard Munich is... well, actually I don't know much about
Munich. Are you still planning to work on WebKit?

:DG<

[Drew Wilson]

Munich is groovy so far - I'm still on Chrome (working on chrome enterprise), and it seems like a better use of my committer status to do WK gardening rather than chrome sherriffing...

-atw

[Ojan Vafai]

On Fri, Nov 2, 2012 at 7:26 AM, Stephen Chenney <sche...@google.com> wrote:

For my shifts I've always been the non-PST gardener physically located in EST. I like it in that I start early and mostly fix the tree and then can back off a bit and get some other stuff done in the afternoon. Fortunately I've been paired with PST folks who tend to start a bit later in their day and cover the west coast evening and get the roll done.

I think that this leads to more tree redness for EMEA when they are not gardening, and APAC to a lesser extent, but I think it does minimize the total amount of pain.

Effectively there is always at least 12-16 hours of overlapping coverage with a 8-12 hour gap with no coverage. In every coverage period the first person gets the tree green and the second person maintains and rolls. It's quite efficient and not too unpleasant.

In my mind the "minimally red, maximum rolls" solution is to have 3 shifts (APAC, EMEA, PST) and achieve almost round-the-clock coverage. But that is more overhead for everyone, particularly EMEA where we have fewer WebKit people (I think). I do not see the benefits out-weighing the costs.

Does anyone have the actual numbers for each of these groups (presumably with EST people being part of EMEA)? Data could help resolve this. If we do have enough people for it, I think it's worth it. It distributes the pain more, or at least, lowers the extremes of brokenness likely to be encountered.

 

--
 
 

[Stephen White]

On Fri, Nov 2, 2012 at 10:05 AM, Stephen White <senor...@chromium.org> wrote:

Actually I think EST gardeners are being considered PST right now.  My co-gardeners this shift were in STP and SYD.

I take that back -- aboxhall is in MTV, not SYD.  So apparently EST is sometimes PST, sometimes !PST.  :)

Stephen

[TAMURA, Kent]

> My requests would be to reduce the shifts from 4 to 3 days -- the 4
day shifts are grueling

The reduction sounds nice.  I prefer two 2-days-shift to one 4-days-shift.

--

--
TAMURA Kent
Software Engineer, Google

[Drew Wilson]

So, when we shifted from two-day shifts to 4-day shifts, the idea was that this would reduce the rampup time, and we were going to try this for a while and then revisit whether we liked 4-day shifts.

Did we ever revisit this to see if people prefer 4-day shifts, or are they just living on from inertia? I find that I really dislike how 4-day shifts blow up an entire week.

--
 
 

[Alice Boxhall]

Well, I was in SYD when the schedule was written, so I think the intention was that I be the !PST gardener.

[Alec Flett]

I almost can't believe I'm going to say this: I'm a semi-newbie, only had my first 4-day gardening shift last month.. I think I prefer a 4-day shift even though it totally disrupted all of the rest of my work, and it was a pretty long time.

what I found on my first shift:

day 1: WTF am I doing, how do I rebaseline? oh crap I'm barely getting a roll in, I'm letting everyone down

day 2: Ok, no problem getting a roll in... Oh wait, this is taking forever, damn commit queue....oops, it almost didn't happen today

day 3: Ok, starting to get the hang of this - I got a roll in and rebaselined something myself! Yay! Now to queue up another one... oops, not today

day 4: Got a roll in! Now how does this garden-o-matic thing actually work? Maybe I could add a feature..

basically, it took me at least 2 days to get my footing and be able to even think about contributing back to the tools. I think that even now, if I did a 2 day shift, it would be much like days 2 and 3 again... But give me another 4 day shift and I'll probably be fully ramped up, and not distracted by my "regular" job.

my team basically wrote me off for the week (appropriately) and that was a very good thing - it was like I was on vacation - except that I learned WAY more about webkit than I had in the 3 months leading up to this first shift..

so anyway, as a newbie, I reluctantly vote for 4-day shift..

Alec

--
 
 

[Ojan Vafai]

That roughly matches my experience. Having 4 days is more disruptive to my normal work, but it also meant it gave me time to do a better job of addressing performance regressions that happened during my shift and filing/fixing bugs with the bots and the gardening infrastructure.

I'd actually prefer to go the other direction and just make it a proper full week. Seems simpler than shifts being split awkwardly over weekends.

--
 
 

[Kenneth Russell]

I can appreciate the ramp-up time for new gardeners, having been one
not long ago, and still finding I learn something on each shift. Two
day shifts would certainly be too short to get any rhythm going. I
think three-day shifts might be a reasonable balance, but won't push
on this.

Gardening shifts are already a significant chunk of community service
time, and I can't support making them longer than the current 4 days.

-Ken

> --
>
>

[Ian Fette (イアンフェッティ)]

Drew, it's part of a normal chrome-internal checkout. go/chrome has info (including code review info), see "Development Setup, Checkout and Build Instructions" and search for src-internal

[Ojan Vafai]

I'm OK with trying 3 day shifts. We can always change it again if we find it doesn't work.

[Stephen Chenney]

I find the two days of overlap between gardeners to be good from a social perspective, in that once you establish a rhythm with another gardener you can keep it going for another day. 3-day rotations would lose that, and give variable 1 or 2 day overlaps. Would you really want to have your 1st ever shift when your co-gardener is on their 2nd day ever?

Also, nobody has pointed out that with 3-day shifts we would each be called up 33% more often. That means more shorter periods of schedule disruption. Personally I like fewer and longer disrupted periods. Hence, I support the current system.

3 day shifts would be good if we had 3 people active each day (the previously discussed one person in each major region). Each day would have 2 people who were mid-rotation and one new person. That's a lot more shifts per person, but I would support that option because the stress associated with each shift would probably be lower. I just don't know we have wider buy-in for that. 

Cheers,

Stephen.

--
 
 

[Ojan Vafai]

Looking at chrome-internal/trunk/tools/build/scripts/tools/sheriff/rotations/webkit.rotation.json, we currently have 30 PST gardeners and 32 non-PST. Just at a quick glance, I see the pdr and aboxhall need to be moved non-PST->PST. So, with the current list, it seems to me that splitting the non-PST gardeners into two groups might put an undue burden on some folks.

That said, do we only have 62 webkit contributors capable of being gardeners? (lol..."only")

I think maybe this list hasn't been updated. For example, this mailing list has 76 members. And there are probably new webkit committers that don't know they should be signing up for this.

Anyways, what we really need is someone to take ownership of this script and to email the appropriate chrome mailing list (chrome-owp? chrome-team?) to let people know they should sign up for gardening if they regularly work on webkit.

If we get more non-PST folks, then doing 3 separate 3 day shifts sounds great to me.

[Dimitri Glazkov]

Adding Rebecca to the thread. She's going to drive gardening
scheduling and stuff.

:DG<

> --
>
>