PSA: Changing how layout test results are archived and displayed

[mart...@chromium.org]

If you don’t care about layout test results, you can stop reading now.

Hi,

As part of my work to improve CQ speed and reliability, I noticed that archiving layout test results on Windows takes ~10 minutes, when it should take less than a minute (tracking bug here).

The easiest solution to this problem is to remove the recursive upload of the entire results directory. This would break the existing test results “Show results” functionality (screenshot of this attached). This functionality can be fixed, but it’s unclear how widely used this is; if you have used the “Show Results” functionality recently (past few months), please let me know. You can reply all to this email, to me directly, or post on the bug.

I’m also planning on removing the “latest” result upload for tryservers. This is used on the waterfall to compare a particular run of a test to the “latest” result for the bot. This doesn’t make sense for tryservers, since the “latest” result is roughly meaningless. If the above paragraph’s plan of removing recursive upload is implemented, this probably won’t be done.

If you have any comments or issues with this, feel free to contact me.

Thanks,

Stephen

[Christian Biesinger]

It is not clear to me what exactly you are removing? The example in
your screenshot is not something I personally use, but I frequently do
use the "layout_test_results" from the trybots, e.g.
https://storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/618905/layout-test-results/results.html
-- is that something you are considering removing as well?

Thanks,
Christian

> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f69d5654-2d5a-4257-b6fc-6249478c2d5f%40chromium.org.

[mart...@chromium.org]

To follow up on the email; I got several emails from people who say they're still using the "Show Results" feature on the test results dashboard. In addition, other people have pointed out (like Christian did) that they use the results page (example here) frequently, which would have also been deleted.

Therefore, I'm not going to be removing these any time soon. It's possible they'll eventually be removed, but I'll make sure that these workflows continue to function as expected. If you're interested, star this bug for updates.

[Chris Harrelson]

On Wed, Jan 10, 2018 at 1:33 PM, Christian Biesinger <cbies...@chromium.org> wrote:

It is not clear to me what exactly you are removing? The example in
your screenshot is not something I personally use, but I frequently do
use the "layout_test_results" from the trybots, e.g.
https://storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/618905/layout-test-results/results.html
-- is that something you are considering removing as well?

+1, I use this all the time.

 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPTJ0XF%2B6zGZPie6g0iaP5ngVqLEahnB1vRoXKjrvL3RUueMMg%40mail.gmail.com.

[mart...@chromium.org]

I'm happy to announce that I've made progress on this. I've implemented a small app engine service which unzips the results file stored in cloud storage. It does pretty simple redirection; to view a URL like https://storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/618905/layout-test-results/results.html , you would just go to https://test-results.appspot.com/data/layout_results/linux_chromium_rel_ng/618905/layout-test-results/results.html.

I landed a CL yesterday which switched all the "layout_test_results" links on trybots to point to this new page. There were a few bugs uncovered with this page, however, so I reverted that change and worked on fixing them. As of now, these fixes should be live in production. I'm landing two CLs today to switch everything to point at test results, instead of google storage. If you find any bugs, please file a bug using this link, which has the Infra>Flakiness>Dashboard component, and is assigned to me.

After I switch the links generated for "layout_test_results" on build pages over to the test results server, I'm going to wait a few days for any outstanding feedback, and then remove the existing recursive uploads. 

As a note, the handler is a bit slow; it fetches the entire zip file from cloud storage to display a single file. There are some optimizations I can do if needed, but hopefully the existing speed will be good enough.

[Dirk Pranke]

This is fantastic, thank you for doing this :).

-- Dirk

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/b580b41a-2ab1-4722-9723-1ba7d963dcf9%40chromium.org.

[Stephen Martinis]

I landed this CL today to turn off recursive uploads for layout test archival. I checked today, and there were no signs of anyone accessing any newly uploaded layout test results using the recursive upload. The handler in the test results server also seemed to be running fine. 

As a note, existing links will not break; https://storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/618905/layout-test-results/results.html still works. There will be some residual access to cloud storage, because old builds still have those links. Going forward, however, there won't be any recursive uploading.

I'd check to see if this seems to be working ok right now, but CQ seems to be having issues right now so I'll have to wait until tomorrow.

Let me know if you see any issues related to this!

Stephen

On Sun, Feb 11, 2018 at 1:59 PM Dirk Pranke <dpr...@chromium.org> wrote:

This is fantastic, thank you for doing this :).

-- Dirk

On Fri, Feb 9, 2018 at 1:06 PM, <mart...@chromium.org> wrote:

I'm happy to announce that I've made progress on this. I've implemented a small app engine service which unzips the results file stored in cloud storage. It does pretty simple redirection; to view a URL like https:https://storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/618905/layout-test-results/results.html//storage.googleapis.com/chromium-layout-test-archives/linux_chromium_rel_ng/618905/layout-test-results/results.html , you would just go to https://test-results.appspot.com/data/layout_results/linux_chromium_rel_ng/618905/layout-test-results/results.html.

[Christian Biesinger]

Thanks for doing this! Slow layout test results uploads have been
annoying me for a long time.

Christian

> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CANheWBk14LpbbcqBmKEn-NXy1r9fv22AGEO5jDSP3woo5_2GhQ%40mail.gmail.com.

[Aleks Totic]

unzip on demand feels slow when looking at test results details. 

Each failed test result takes ~2s to display. This makes paging through 100 test failures annoying.

It would be great if we could speed it up.

Try going through unexpected failures at https://test-results.appspot.com/data/layout_results/linux_layout_tests_layout_ng/3748/layout-test-results/results.html by hitting spacebar. This used to be almost instant.

Aleks

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPTJ0XG%2Bmgc4jqRX_iBQB%3DQwfQUCHZJmhz_fqmZ9WAgq_xMwQw%40mail.gmail.com.

[Koji Ishii]

Ah, that was why reviewing failures was very slow today...

+1 to Aleks, great if we can make it faster.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMdyzDuPqOw5kKW3uGSR1r%2Bs3kgEorBsJ4_h%2BXyG5gUhcQoRpQ%40mail.gmail.com.

[Stephen Martinis]

I'm working on this in https://crbug.com/812303.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPTJ0XG%2Bmgc4jqRX_iBQB%3DQwfQUCHZJmhz_fqmZ9WAgq_xMwQw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.