7/22 CY Updates

[Ojan Vafai]

Tree open:

Chromium: This week: 54.17%, Last week: 59.16%

Blink: This week: 91.82%, Last week: 79.59%

-Improved the auto-reopener to open as soon as the step finishes instead of when the whole run finishes. The blink tree's worst 8-hour shift since this change was pushed was 73% open, compared to 0% before! This graph is particularly happy: https://blink-status.appspot.com/status_viewer?curView=peak&startTime=TODAY&numDays=30

-Steady progress on sheriff-o-matic and it's new backend. Optimistic that we'll have a dogfoodable product that chromium sheriffs can try by the end of the week.

-Lots of discussions/conclusions around design and implementation of the flaky test ignorer bot.

-In preparation for infra failures being purple instead of red, the CQ now treats red and purple failures the same way instead of going into a potential infinite loop.

On Tue, Jul 22, 2014 at 1:30 PM, John Abd-El-Malek <j...@chromium.org> wrote:

This week's updates

-maruel turned on swarming for linux by default! bot running time goes from 60 minutes to about 35. mac/win are being tested for a percentage of try runs.

-sky expanded the code to early out of bots if they're not affected by a change to Mac. hopefully it'll spread to all the bots soon.

-with help from vadimsh, I changed telemetry tests to run after triggering swarm jobs. the complication was that recipes code treated gtest-based tests separately from telemetry. since these take ~11 minutes on mac bots today, that'll help parallelize the steps on the VMs.

-I enabled swarming for content browser and unit tests

[Julie Parent]

Monitoring: Focus: dashboard to evaluate CY Exit Criteria

- EC: Lower CQ latency (median -> 1h, 90th% -> 2h) --> Graph to track CQ latency (and, bonus, length): https://trooper-o-matic.googleplex.com/cq

- EC: Lower Chromium tree bots cycle time (non-clobber median -> 30m, max -> 1h, clobber median -> 2h, max -> 3h) --> Graph to track whenever chromium and blink trees violate CY EC (https://trooper-o-matic.googleplex.com/tree/chromium, https://trooper-o-matic.googleplex.com/tree/blink) [ Still need to separate out clobber/non-clobber ]

Next up:

- Graphs for other 2 EC (Tree open times, CQ false rejection rates), finish the above graphs, consolidate into one "State of the CY" dashboard

- Create template for post-mortems

- Code check into common location - side by side with sherrif-o? infra.git? (Will take this convo into separate thread)

- Recruit more people for monitoring area :)

--
You received this message because you are subscribed to the Google Groups "Chromium Hackability Code Yellow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hackability-c...@chromium.org.
To post to this group, send email to hackabi...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/hackability-cy/CANMdWTsmAdnzY3ONYrYeGEF_eC0v0GfHphb1DDyX8xKt5AwLkg%40mail.gmail.com.