Should webkit_layout_tests run earlier?

[Christian Biesinger]

Hi all,

I was looking at a CQ run here:
https://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_ng/builds/511059

I noticed that webkit_layout_tests ran last. Wouldn't it be better if
it ran earlier? That way, it can run the upload and
archive_webkit_results step potentially in parallel with other swarmed
test suites.

Christian

[Quinten Yearsley]

I'm not sure if this is the best way to visualize job runs, but if you go to the build linked above and click "Show Gantt Chart" (at the bottom), it looks like the general flow of the job is something like:

• Set up: bot_update, compile, isolate. This is sequential and takes about half of the time.
• Trigger swarming tasks and wait. These all run in parallel. The difference between starting triggering first, and starting triggering last, is only about 1 minute.
• Finally, archive layout test results. This takes 5 minutes and depends on layout tests being finished.

So if my understanding is correct, we wouldn't save much time by changing the order of triggering. But we could still potentially save several minutes by making the archive step faster (crbug.com/310382).

Christian

--
You received this message because you are subscribed to the Google Groups "blink-infra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-infra+unsubscribe@chromium.org.
To post to this group, send email to blink...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-infra/CAPTJ0XFLuuj_XqZCoO%2B7_%2BAZajpZfnbvgWenNS71opWXuRZmnA%40mail.gmail.com.

[Christian Biesinger]

Ah I see -- I think what you're saying is, the UI is somewhat
misleading and in fact the archiving happens as soon as the
layout_test is done, it doesn't necessarily wait for other steps to
finish first? Makes sense.

I definitely support making the archive step faster :) Especially on
Windows, where it seems especially slow (like so many things)

Christian

>> email to blink-infra...@chromium.org.

[Marc-Antoine Ruel]

Archival performance is actively being worked on: https://crbug.com/692940

We observed one specific VM having significant bandwidth issue but we didn't find the root case yet: https://crbug.com/748261

When tests are run on Swarming, they are all run in parallel. This does make the UI less intuitive.

While build #511059 on mac_chromium_rel_ng took 38 mins to complete, the time to run the tests accounted for 11 minutes of this build. The rest was checkout, build, archive and "5 mins, 36 secs just for archive_webkit_tests_results" (!?!)

What's interesting is the amount of testing that occurred during these 11 minutes: 3 hours 41 minutes of tests were run as 64 shards. A 20x acceleration. Just for this single build. A lot of engineering went into making this a reality and as seamless as possible.

Thanks,

M-A

>> email to blink-infra+unsubscribe@chromium.org.

>> To post to this group, send email to blink...@chromium.org.
>> To view this discussion on the web visit
>> https://groups.google.com/a/chromium.org/d/msgid/blink-infra/CAPTJ0XFLuuj_XqZCoO%2B7_%2BAZajpZfnbvgWenNS71opWXuRZmnA%40mail.gmail.com.
>
>

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.
To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAPTJ0XHpZX-n-0Y3AqoeVwW2a11xryJs0UMPcAxhBGU2_%3DhRMg%40mail.gmail.com.

[Quinten Yearsley]

On Thu, Jul 27, 2017 at 11:26 AM, Marc-Antoine Ruel <mar...@chromium.org> wrote:

Archival performance is actively being worked on: https://crbug.com/692940.

To clarify, before I was talking about archive_webkit_tests_results, which is just the part that uploads files to Google Storage after a layout tests run, which is different from isolate archive. isolate [exp]archive is also relevant to performance of these runs though, without Tim and Michael's work there, the performance would have been much worse.

While build #511059 on mac_chromium_rel_ng took 38 mins to complete, the time to run the tests accounted for 11 minutes of this build. The rest was checkout, build, archive and "5 mins, 36 secs just for archive_webkit_tests_results" (!?!)

 

What's interesting is the amount of testing that occurred during these 11 minutes: 3 hours 41 minutes of tests were run as 64 shards. A 20x acceleration. Just for this single build. A lot of engineering went into making this a reality and as seamless as possible.

Definitely! This is excellent :-D