WPT import blacklists

[Chris Harrelson]

This file list various blacklisted directories for WPT import. There was a discussion this morning at the rendering meeting about this - blacklisting import vs skipping/marking-as-failing imported tests. I think it would be better to import more directories and mark them as failing if they are features or semantics we *should* be supporting but don't yet. I suggested auditing this list for rendering and re-enabling various directories, to get a more accurate picture of what is actually failing or not in the CSS etc specs. This also has the benefit of increasing test coverage in cases where the test does pass and it exercises some feature combo not previously tested.

Ecosystem infra team: is the reason for this blacklist just practicality and need for work triaging failures from import? Or are these directories mostly "deprecated" ones?

Chris

[Chris Harrelson]

I should probably have said "skipped" rather than "blacklisted".

--
You received this message because you are subscribed to the Google Groups "rendering-core-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rendering-core-...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/rendering-core-dev/CAOMQ%2Bw8KHU0abwrucxKC41JhbcbobbkwuMkqB%2BnmCvvKZSLB6A%40mail.gmail.com.

[Stephen Mcgruer]

> Ecosystem infra team: is the reason for this blacklist just practicality and need for work triaging failures from import? Or are these directories mostly "deprecated" ones?

It is a combination of directories we don't have use for (docs/, bits of infrastructure, etc), directories excluded for specific reasons (which may now be out of date), and things excluded just for lack of someone to triage any incoming failures.

A lot of it is very old and likely outdated; some directories no longer exist and some reasonings for excluding others are likely no longer valid. I'd be happy to see an effort to clean it up - @Robert Ma suggested we should start by doing some automated cleanup of directories/files that no longer exist in upstream WPT, which I think is a great idea!

You received this message because you are subscribed to the Google Groups "ecosystem-infra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ecosystem-inf...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/ecosystem-infra/CA%2BN6QZtwx62%3D9LZXRPwYZWJaQwzuZYORudy6nN-QnA2PfTxPTQ%40mail.gmail.com.

[Chris Harrelson]

On Tue, Apr 21, 2020 at 1:40 PM Stephen Mcgruer <smcg...@chromium.org> wrote:

> Ecosystem infra team: is the reason for this blacklist just practicality and need for work triaging failures from import? Or are these directories mostly "deprecated" ones?

It is a combination of directories we don't have use for (docs/, bits of infrastructure, etc), directories excluded for specific reasons (which may now be out of date), and things excluded just for lack of someone to triage any incoming failures.

A lot of it is very old and likely outdated; some directories no longer exist and some reasonings for excluding others are likely no longer valid. I'd be happy to see an effort to clean it up - @Robert Ma suggested we should start by doing some automated cleanup of directories/files that no longer exist in upstream WPT, which I think is a great idea!

Excellent. As part of this, would your team be willing to unskip directories that seem to have legit rendering features in them and send them to us for code review? Or we can wait for this first pass you mentioned above and then unskip some directories. It seems we already found one with some tables tests for example..

[Stephen Mcgruer]

> Excellent. As part of this, would your team be willing to unskip directories that seem to have legit rendering features in them and send them to us for code review? Or we can wait for this first pass you mentioned above and then unskip some directories. It seems we already found one with some tables tests for example.

The automated pass is https://chromium-review.googlesource.com/c/chromium/src/+/2159623, which didn't remove that many lines (20 entries, ~5% of the total).

I'd like to do a manual pass at some point too, but it will be lower priority for us atm. I suspect you may actually be faster in finding directories that are interesting, since you have the domain knowledge. Feel free to ping us if there's any questions though or if you think we could be of help :)

Once a line is removed from W3CImportExpectations the next importer run will pick it up, and will automatically add expectations for any failing files. If that is either too much latency for you, or if you want to 'try out' a directory (noting that you can approximately rely on the results in wpt.fyi as similar to what will happen post-import), there is a manual import process. 

[Chris Harrelson]

Got it. We'll do a pass over this file and add some directories relevant to rendering.

[Sam Sneddon]

On 21/04/2020 18:58, Chris Harrelson wrote:
> Ecosystem infra team: is the reason for this blacklist just practicality
> and need for work triaging failures from import? Or are these
> directories mostly "deprecated" ones?

The majority of lines in /css are CSS2 ones, which I believe we didn't
import due to already having an older copy of it imported with various
local modifications
(https://bugs.chromium.org/p/chromium/issues/detail?id=656669).

/Sam