Plan for switching the default layout tests viewer

[Quinten Yearsley]

A little while ago, atotic@ added a new layout test viewer, originally for the purpose of making it easier to update test expectations files. Example: example of the new viewer for a test run on Mac 10.9.

Now, I'd like to switch over to use this new viewer by default and skobes@ suggested that we could switch it to the default and keep the old one as a backup "escape hatch" for a while.

Although, before we switch, atotic@ suggested we make a few changes before doing that -- e.g. changing the filters to be checkboxes.

Then, to make the transition more convenient, and because "test-expectations.html" might be a confusing name, we could change the name of the new viewer to "results.html" and change the name of the old viewer to something like legacy_results.html.

Does that sound like a good general plan? Any thoughts or things that people would like to suggest?

[Steve Kobes]

On Mon, Jul 17, 2017 at 10:28 AM, Quinten Yearsley <qyea...@chromium.org> wrote:

Then, to make the transition more convenient, and because "test-expectations.html" might be a confusing name, we could change the name of the new viewer to "results.html" and change the name of the old viewer to something like legacy_results.html.

I think this is a good idea.  The escape hatch link already says "go back to legacy results.html". :)

It would be nice to add a flag to run-webkit-tests to specify which viewer to open by default.

[Aleks Totic]

Before the switch, I'd like to make the new page fully featured. There
are a few things I want. Is there anything you'd like to see?

Email me your suggestions, or add them to the tracking bug is crbug.com/726520

Things I'd like to implement:

1) make queries cleaner, more useful. Right now, all queries are
buttons. I'd like to switch it to two rows:
1st row are buttons for high level query: "Unexpected fail",
"Unexpected pass", "Test Expectations", "Slow tests"
2nd row are checkboxes for additional filters by failure type: "Crash"
| "Timeout" ......

2) Add timing information
It'd be nice to know which tests are slow. Maybe add another high
level query: "Speed" that displays all tests sorted by time taken.

3) Results display for retries
Retried tests can have multiple results. We only display one. Is there
any interest in displaying all failing results? I personally do not
need it, but it might be interesting to see why the test failed
initially before passing.

Aleks

On Mon, Jul 17, 2017 at 10:28 AM, Quinten Yearsley
<qyea...@chromium.org> wrote:

[Aleks Totic]

Spent some time on polish: https://chromium-review.googlesource.com/c/580827/

I am happy to release this. Ping me if you can think of anything else.

The only feature I'd really like to add is "local path override" preference. 

It would be used to provide link to local tests If remote test links are not available.

But I ran out of gas today.

Aleks