Renaming LayoutTests/ ?

[TAMURA, Kent]

In The Great Blink mv, we propose renaming LayoutTests/ to NOT layout_tests/, but to something else like page_tests/, to stop using the name "layout test". 

Tests in the directory are not only for layout behavior.  We have used the name for a historical reason and no one wanted to pay cost to switch to another name.

Does anyone want to keep the name "layout test"?  Any idea of better names?

--

TAMURA Kent
Software Engineer, Google

[Stefan Zager]

On Wed, Apr 19, 2017 at 8:32 AM, TAMURA, Kent <tk...@chromium.org> wrote:

In The Great Blink mv, we propose renaming LayoutTests/ to NOT layout_tests/, but to something else like page_tests/, to stop using the name "layout test". 

Tests in the directory are not only for layout behavior.  We have used the name for a historical reason and no one wanted to pay cost to switch to another name.

Does anyone want to keep the name "layout test"?  Any idea of better names?

I'm not sure that page_tests is any better.  I'd prefer a name that communicates the fact that the tests are built on top of the web platform (plus some internals and testRunner magic).

web_platform_tests is probably a bad idea -- because the tests largely aren't focused on proving web platform compliance, and because it would get confused with the external web platform tests repository.

platform_tests is similarly misleading.  I guess web_tests is a possibility, but I feel pretty lukewarm towards that.

[Ken Rockot]

"web_tests" seems concise and accurate.

[Emil A Eklund]

Unless there is a proposal that is significantly better I'd rather
keep the name layout_tests as that is already a well known name and
one that we share with WebKit. Renaming it even if the new name is
slightly better is probably not worth the effort imho.

[Kentaro Hara]

Yeah, I'm also leaning toward investing on more important code health projects unless renaming LayoutTests/ has a clear benefit.

--

Kentaro Hara, Tokyo, Japan

[Dimitri Glazkov]

I agree that LayoutTests name is not the best name and there could be a better name that describes the purpose of the tests in this directory. At the same time, what Emil says about the cognitive cost of the change resonates with me. Could we leave it alone for now as "layout_tests"? 

:DG<

[Fernando Serboncini]

The reason for renaming things like this is to make things clearer, more intuitive and more discoverable for new people.

Everyone who already works on Chromium knows that layout_test is the place where we put all our web related tests and that they have very little to do with "layout".

We would want to pay a little inconvenience cost (renaming something you are used to) for a longer term benefit (clearer name for when a reasonable programmer that is new to the project wants to know what to tests when she is changing something).

Claiming that we shouldn't change because you already know that even though it's called layout_test is not about layout is the weirdest explanation. Of course you'd know, but this change is not for you.

[Daniel Bratell]

Sounds like there will be a rename either way. Renaming it to layout_tests seems suboptimal when it has been known for a long time that name is misleading.

/Daniel - colouring the shed

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--

/* Opera Software, Linköping, Sweden: CET (UTC+1) */

[Anton Vayvod]

"web_tests" sounds good to me. If we rename blink / webkit to web, blink_tests and run_webkit_tests naturally become web_tests / run_web_tests. Otherwise, replace "web" with whatever third_party/WebKit becomes after the rename.

Another thought is "html_tests" since they're html files?

[Jeffrey Yasskin]

What purpose do these serve as tests?

• Are they Blink's end-to-end or system tests? Integration tests?
• Are they content_shell's integration tests, while browser_tests are Chrome's integration tests?
• Are they Web Platform Tests that simply haven't been upstreamed yet, so local_web_platform_tests?
• Are there some LayoutTests that are un-upstreamed Web Platform Tests and others that serve a different purpose and so maybe should land in a separate directory?
  

Jeffrey

[Mustaq Ahmed]

I was about to suggest "html_tests" because all tests here are based on htmls. This will also make it clear that these are not necessarily web-platform tests.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Mustaq Ahmed]

On Wed, Apr 19, 2017 at 12:09 PM, Jeffrey Yasskin <jyas...@chromium.org> wrote:

What purpose do these serve as tests?

• Are they Blink's end-to-end or system tests? Integration tests?
• Are they content_shell's integration tests, while browser_tests are Chrome's integration tests?

Some (most?) tests here are like integration tests for Blink (or content_shell) only but some tests cover part of chromium too. E.g. the tests in external/wpt_automation/pointerevents/ uses Chrome's GPU benchmarking extension, effectively serves as integration tests for part of the input pipeline in browser, at least from renderer_host all the way to html/js.

• Are they Web Platform Tests that simply haven't been upstreamed yet, so local_web_platform_tests?
• Are there some LayoutTests that are un-upstreamed Web Platform Tests and others that serve a different purpose and so maybe should land in a separate directory?

Few of the tests are web-platform tests, kept in sync with the w3c repo (e.g. external/wpt/pointerevents/). Some are chromium-implementation specific (e.g. fast/events/pointerevents/pointer-use-count.html). Most of the rest of the tests are for web-platform features but seems non-upstreamable because they heavily rely on chromium-specific test code (e.g. eventSender).

[Stephen Chenney]

The existing LayoutTests are not all html tests, some are xml, some svg, some other web supported content. And there is an existing LayoutTests/html directory and there is a distinct Blink>HTML bug component. So right now html is a subset of web (nee Blink) and I am against using the html_tests name.

Personally I support web_tests. I'm also fine with leaving it as layout_tests.

Stephen.

[Stefan Zager]

On Wed, Apr 19, 2017 at 6:09 PM, Jeffrey Yasskin <jyas...@chromium.org> wrote:

What purpose do these serve as tests?

• Are they Blink's end-to-end or system tests? Integration tests?
• Are they content_shell's integration tests, while browser_tests are Chrome's integration tests?
• Are they Web Platform Tests that simply haven't been upstreamed yet, so local_web_platform_tests?
• Are there some LayoutTests that are un-upstreamed Web Platform Tests and others that serve a different purpose and so maybe should land in a separate directory?

They are a mix of web-platform-style tests; integration tests that exercise chromium-specific behavior; and lots and lots of regression tests from bug fixes over the years.

[Dirk Pranke]

We've had this discussion a few times over the years, and not yet found something that is obviously better enough to justify changing the name.

However, if we really didn't want to keep "LayoutTests" and hence wanted to switch to something that was more naming-guideline consistent, I
would also argue that we should change it to something better than "layout_tests". I think "web_tests" is close, but after having thought about it

for a bit, I actually like "webpage_tests" better, because that's pretty close to describing what the actually do: load a web page that does *something*.

-- Dirk

[Aleks Totic]

I see LayoutTests as chromium-specific version of w3c's
web-platform-tests. That is why I'd s/web/<something chromium
specific>

( blink | chrome )_platform_tests sound intuitive to me.

Other names that should be consistent with this rename:

WebKit/Tools/Scripts/run-webkit-tests -> run_( blink | chrome )_tests

We also have other test suites:

webkit_unit_tests
browser_tests (never ran these, so not sure)

Then there are telemetry benchmarks:
src/tools/perf/run_benchmark.

As a developer, a consistent, logical naming scheme would be great.
Something like:
run-platform-tests
run-unit-tests
run-benchmark

> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to blink-dev+...@chromium.org.

[Peter Kasting]

It seems like the reason no one can agree on a good name is that the existing set of tests combines tests from many origins, using many implementation methods, testing many parts of the stack.  If there is a problem to be solved here at all, it is not that we have a bad name, it is that we have an unnameable conglomerate.

If people don't like "layout tests", I suggest someone propose a specific division of tests into distinct groups, which can then be named based on their distinguishing characteristics.

If that's not worth doing, then I don't see how changing the name is going to reduce confusion or achieve something else positive, because those factors are artifacts of the tests themselves, not directory nomenclature.

PK

[Dirk Pranke]

One aspect of the confusion is that there are tests that have nothing to do w/ layout in a directory called LayoutTests. If one could find an adequately generic name, you could at least fix that, even if that doing so didn't fix every other source of confusion caused by that directory.

I also disagree with the "many implementation methods" part. Every test in layout tests is run the same way, by loading a web page. Which is why "page tests" or "web page tests" makes a certain amount of sense to me. It is true that many browser_tests and interactive_ui_tests also load web pages, so the name isn't perfect, but "web pages that are intended primarily to test the implementation of the web page rendering engine and various associated libraries" seems impractical.

-- Dirk

[Steve Kobes]

"web_tests" sgtm.

[Stefan Zager]

On Thu, Apr 20, 2017 at 1:40 AM, Peter Kasting <pkas...@chromium.org> wrote:

It seems like the reason no one can agree on a good name is that the existing set of tests combines tests from many origins, using many implementation methods, testing many parts of the stack.  If there is a problem to be solved here at all, it is not that we have a bad name, it is that we have an unnameable conglomerate.

There is some truth to this, but the real problem is not the name of the top-level directory; it is the organization of the subdirectories ("LayoutTests/fast" FTW).  However, I believe the long-standing consensus is that reorganizing the subdirectories would be an overwhelming and tedious task, and the end result would not justify the investment.

As Dirk pointed out, everything in LayoutTests uses the same test framework: load a page in content_shell.  Maybe the directory should be content_shell_tests.  Ugh.

[Colin Blundell]

There would be a lot of ambiguity between content_shell_tests and content_browsertests IMO.

+1 to web_tests as opposed to layout_tests given that a renaming is happening in any case. 

[Dimitri Glazkov]

Folks,

I am worried about the transparency, thoughtfulness, and efficiency of this process of collective pondering the name of a directory, and would like to avoid the typical negative impact of the bikeshedding threads (poor visibility of the current state of discussion, decision-by-fatigue).

So, I am proposing the following decision framework:

1) The decision driver (Kent-san in this case) creates a publicly editable decision doc, documenting the problem, constraints uncovered, and current state of discussion. A simple list of proposals with a summary, plus pros/cons listed for each would probably suffice.

2) The stakeholders (all y'all) contribute to the doc. The stakeholders add their proposals and populate pros/cons of all proposals. They strive to avoid bikeshed discussions in doc comments by formulating pros/cons as neutral, objective statements.

3) The decision driver monitors the doc and determines the point when all proposals and pros/cons are no longer actively updated. At that point, the driver evaluates all proposals and makes a recommendation.

4) The decision driver works with the owners of the parent directory to ratify the recommendation. In this particular case, the owners are the API_OWNERS.

5) Once the recommendation is ratified, the driver writes the summary of the decision in the decision doc proceeds with the change.

6) Once the change is made, the driver documents the decision in the README.md of the parent directory by linking to the decision doc. ("why is this directory named [foo]? see this decision doc for details")

:DG<

[Sasha Bermeister]

+1, I support this framework. Blink has had real problems recently with good community decision making, and resorting to decision-by-fatigue. I'm in favor of any process to reduce that by formalizing the process.

[Alan Cutter]

"decision-by-fatigue" is a fantastic articulation of one of the problems with this format. Glad to see it being addressed.

[Stuart Langley]

sgtm

On Fri, Apr 21, 2017 at 2:38 AM, Dimitri Glazkov <dgla...@chromium.org> wrote:

[TAMURA, Kent]

It sounds a nice procedure. Let's do it.

I created a document.  It doesn't reflect the discussion of this thread yet.

Feel free to update it.

https://docs.google.com/document/d/1wLSfod8apDy2sZsmXpTLRacfjweXP86C-V0wx5koJBg/edit?usp=sharing

[TAMURA, Kent]

We should discuss the rename now because cost to move //third_party/WebKit/LayoutTests to //web/LayoutTests and cost to move it to //web/something_else are not so different.  Even if we keep LayoutTests, we need to check and update all of code and document about LayoutTess.

[Dimitri Glazkov]

Kent-san, what's the status on this?

:DG<

[Stephen Chenney]

Looks like very few people have commented on the doc. I think if everyone jumped on there and commented soon there would then be enough data for someone to decide.

https://docs.google.com/document/d/1wLSfod8apDy2sZsmXpTLRacfjweXP86C-V0wx5koJBg/edit?usp=sharing

Cheers,

Stephen.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOtFfx7QThD00LdxnVHUW3WJbBtycok4-FeUd62KrrM-4eZAUQ%40mail.gmail.com.

[Dimitri Glazkov]

On Fri, May 26, 2017 at 10:37 AM Stephen Chenney <sche...@chromium.org> wrote:

Looks like very few people have commented on the doc. I think if everyone jumped on there and commented soon there would then be enough data for someone to decide.

The lack of comments is probably due to the fact that I recommended in the decision framework that peeps avoid comments, and instead help populate the document. I still stand this by recommendation and would like to avoid changing direction at this point.

:DG<

[Stephen Chenney]

That's fine by me too. I'll delete my comment while leaving my suggested name.

Cheers,

Stephen.

[Dimitri Glazkov]

\o/

:DG<

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAObCcUq3t5GBZ-y8HOiQN%2Bk3KjU5fxrrQs8q7AynUQaCkGP5pg%40mail.gmail.com.

[TAMURA, Kent]

Let's resume the discussion.

Please add/update candidates and pros/cons in the document by September 5 23:59 in your timezone.

https://docs.google.com/document/d/1wLSfod8apDy2sZsmXpTLRacfjweXP86C-V0wx5koJBg/edit?usp=sharing

[TAMURA, Kent]

Thank you for many feedback.

We decided to rename LayoutTests/ to web_tests/.

[Steve Kobes]

Just curious, is this still being worked on?  We seem to be in a halfway state now, with the tools renamed but the tests still in third_party/WebKit/LayoutTests.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGH7WqFbuXkUAG%2B%2BN8_qqYNWB6ejTJ2h2uXRwY1P7cJ7%3DPmyqQ%40mail.gmail.com.

[Robert Ma]

Philip and I were discussing this very recently. This is still something we'd like to do, and we would also like to take the chance to simplify the directory structure a bit (e.g. omitting "/external/" as wpt is the only external suite now).

[Emil A Eklund]

On Wed, Sep 12, 2018 at 2:41 PM, Robert Ma <robe...@chromium.org> wrote:
> Philip and I were discussing this very recently. This is still something
> we'd like to do, and we would also like to take the chance to simplify the
> directory structure a bit (e.g. omitting "/external/" as wpt is the only
> external suite now).

Might also be interested in revering the external/internal convention
to make it clear that wtf is the way forward and the "default".
I.e. having all wpt test directly in the web_test (or whatever it ends
up being called) directory and then the layout tests in an internal or
legacy sub directory.

[TAMURA, Kent]

We'd like to do the renaming, but no one is working on it right now.  Volunteers are welcome!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAAjAU%3DZAf74tO3wqSXeqzw1eWRyaxV%2BHwO4EPJz8YegGOV_BzQ%40mail.gmail.com.