- Win bots were suffering from indefinite hangs; dpranke@ and
cmp@managed to clean it up (we think)note that the Win DEPS bots will
still be
   hanging until we roll past dpranke@'s change at
r133076<http://trac.webkit.org/changeset/133076>
   - - Luckily no major build breakages while Win bots were out of
   commission
   - - MountainLion bots (which are not core yet) are very noisy and tend
   to make garden-o-matic hard to read; dpranke@ is working on cleaning
   them up
   - - ChromeOS trybots seem verrrry slow and flaky
   -
   - - got a try in for 133073 which looks good (will put it in the CQ
   before I leave)
   -   - unfortunately this includes
r133044<http://trac.webkit.org/changeset/133044> that
   introduces http, isolatedWorlds test crashiness
   -   - Adam reverted in r133078, so once we roll past that it should
   clear up
   - - fischman@'s change http://trac.webkit.org/changeset/133069 to clear
   cache in DRT between tests doesn't seem to have caused major badness (yay)
   -   - exposed some flakiness in accessibility tests:
   https://bugs.webkit.org/show_bug.cgi?id=100887

Overall, disappointed that I didn't get more rolls in, but at least we're
not too far behind ToT.

Stephen

-- Dirk

Hopefully rolling past the revert will work.

Adam

Ongoing issues:

   - Builder Win7
(dbg)<http://build.chromium.org/p/chromium.webkit/waterfall?builder=Win7+(dbg)>
has
   been consistently failing 10-15 browser_tests.  This seems to be
   bot-specific; will ping a trooper for a reboot.

Resolved issues:

   - Please join me in congratulating aboxhall@ on a very successful first
   day gardening:  several successful rolls, and many test suppressions and
   cleanups.
      - tc@ fixed a number of flaky tests exposed by the cacheing change (
   http://trac.webkit.org/changeset/133069).
   - I landed a speculative fix for flakiness in a filter blur test (
   http://trac.webkit.org/changeset/133399) which ended up de-flaking about
   five compositing tests.

Some observations about the long tail of flakiness (mostly from staring at
the flakiness dashboard):

   - Linux seems to be the least flaky of all the platforms right now.
    Once you get past the first 30 or so tests, the rest have very low flake.
   - The #1 culprit on MacOS Debug seems to be
   https://bugs.webkit.org/show_bug.cgi?id=81803, causing many CSS
   selectors failures
   - The #1 culprit on Windows seems to be HTTP tests, many of which have
   intermittent timeouts (and many of which are filed under different bugs)
   - Assuming they each have a single root cause, nailing either of the
   above two bugs would go a long way to cleaning up a lot of gardening noise

Stephen

On Mon, Nov 5, 2012 at 5:13 AM, Stephen White <senorbla...@google.com>wrote:

Thanks for looking into actually fixing flakiness issues!