Issue 1515071 in chromium: external/wpt/css/css-backgrounds/background-attachment* tests are flaky

[andruud via monorail]

Updates:
Cc: styl...@chromium.org

Comment #3 on issue 1515071 by and...@chromium.org: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c3

Issue 1520694 has been merged into this issue.

--
You received this message because:
1. You were specifically CC'd on the issue

You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings

Reply to this email to add a comment or make updates.

[Git Watcher via monorail]

Comment #4 on issue 1515071 by Git Watcher: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c4

The following revision refers to this bug:
https://chromium.googlesource.com/chromium/src/+/ecb6ebb068a5135b8485b57e6c66c1c350734fb6

commit ecb6ebb068a5135b8485b57e6c66c1c350734fb6
Author: Anders Hartvoll Ruud <and...@chromium.org>
Date: Fri Jan 26 10:34:18 2024

Triage various issues

Bug: 1520748, 1521187, 1515071
Change-Id: I828dabe14b1acde1510ab2b03c8e906d51a1ac51
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5237618
Commit-Queue: Anders Hartvoll Ruud <and...@chromium.org>
Reviewed-by: Rune Lillesveen <fut...@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1252540}

[modify] https://crrev.com/ecb6ebb068a5135b8485b57e6c66c1c350734fb6/third_party/blink/web_tests/TestExpectations
[modify] https://crrev.com/ecb6ebb068a5135b8485b57e6c66c1c350734fb6/third_party/blink/web_tests/ChromeTestExpectations

[dbaron via monorail]

Comment #5 on issue 1515071 by dba...@chromium.org: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c5

In all these cases, it looks like the problem is that the scroll position ends up a little bit lower than it should be.

The only reason I can think that this might happen is that:
* the scroll happens before the Ahem font loads
* then the Ahem font loads, causing a re-layout

But even with that scenario, given the test, it's not clear to me why the layout would change in a substantive way.

[dbaron via monorail]

Comment #6 on issue 1515071 by dba...@chromium.org: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c6

I'l prepare a CL to use the TestRendered event instead of the load event in the hopes that waiting for font loads fixes the failures.

[dbaron via monorail]

Updates:
Status: Started

Comment #7 on issue 1515071 by dba...@chromium.org: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c7

(No comment was entered for this change.)

[Git Watcher via monorail]

Comment #8 on issue 1515071 by Git Watcher: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c8

The following revision refers to this bug:

https://chromium.googlesource.com/chromium/src/+/01e05d80148126353ffea9bcdc6603d24e8a5c77

commit 01e05d80148126353ffea9bcdc6603d24e8a5c77
Author: L. David Baron <dba...@chromium.org>
Date: Fri Jan 26 17:06:18 2024

Scroll background-attachment tests after font loads complete.

This scrolls these tests in response to the TestRendered event rather
than the load event so that the scroll happens after Ahem webfont loads
have completed.

It's not completely clear to me why the font load changes layout, but
despite that this seems like the most likely cause for the tests flaky
failures due to slightly incorrect scroll position.

Bug: 1515071
Change-Id: I4d9114e9dceeac82dbd5f897a0c455e7fe61da9a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5241084
Reviewed-by: Traian Captan <tca...@chromium.org>
Auto-Submit: David Baron <dba...@chromium.org>
Commit-Queue: Traian Captan <tca...@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1252696}

[modify] https://crrev.com/01e05d80148126353ffea9bcdc6603d24e8a5c77/third_party/blink/web_tests/external/wpt/css/css-backgrounds/background-attachment-fixed-inline-002.html
[modify] https://crrev.com/01e05d80148126353ffea9bcdc6603d24e8a5c77/third_party/blink/web_tests/ChromeTestExpectations
[modify] https://crrev.com/01e05d80148126353ffea9bcdc6603d24e8a5c77/third_party/blink/web_tests/external/wpt/css/css-backgrounds/background-attachment-local-block-002.html
[modify] https://crrev.com/01e05d80148126353ffea9bcdc6603d24e8a5c77/third_party/blink/web_tests/external/wpt/css/css-backgrounds/background-attachment-fixed-block-002.html
[modify] https://crrev.com/01e05d80148126353ffea9bcdc6603d24e8a5c77/third_party/blink/web_tests/external/wpt/css/css-backgrounds/background-attachment-local-inline-002.html

[dbaron via monorail]

Comment #9 on issue 1515071 by dba...@chromium.org: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c9

My preferred queries for checking (later, not enough data yet) if the fix worked:
https://ci.chromium.org/ui/test/chromium%3Aci/ninja%3A%2F%2F%3Achrome_wpt_tests%2Fexternal%2Fwpt%2Fcss%2Fcss-backgrounds%2Fbackground-attachment-fixed-block-002.html
https://ci.chromium.org/ui/test/chromium%3Aci/ninja%3A%2F%2F%3Achrome_wpt_tests%2Fexternal%2Fwpt%2Fcss%2Fcss-backgrounds%2Fbackground-attachment-fixed-inline-002.html
https://ci.chromium.org/ui/test/chromium%3Aci/ninja%3A%2F%2F%3Achrome_wpt_tests%2Fexternal%2Fwpt%2Fcss%2Fcss-backgrounds%2Fbackground-attachment-local-block-002.html
https://ci.chromium.org/ui/test/chromium%3Aci/ninja%3A%2F%2F%3Achrome_wpt_tests%2Fexternal%2Fwpt%2Fcss%2Fcss-backgrounds%2Fbackground-attachment-local-inline-002.html

[dbaron via monorail]

Updates:
Status: Fixed

Comment #10 on issue 1515071 by dba...@chromium.org: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c10

Confirmed as fixed based on test history.

[dbaron via monorail]

Comment #11 on issue 1515071 by dba...@chromium.org: external/wpt/css/css-backgrounds/background-attachment* tests are flaky
https://bugs.chromium.org/p/chromium/issues/detail?id=1515071#c11

Also, I think the corrected link for the dashboard where these were the top flakes is:
https://data.corp.google.com/sites/chrome_generic_flakiness_dashboard_datasite/top_flakes/?p=selected_status:%22FLAKY%22,%22CONSISTENTLY_UNEXPECTED%22,%22HAS_UNEXPECTED_RESULTS%22&f=variant_test_suite:in:chrome_wpt_tests

(which is specific to chrome_wpt_tests).