[PSA] Enabling Isolate

[Christopher Sharp]

TL;DR; We're rolling out base_unittests_run using isolate on all the waterfalls on April 2.

After 12995019 lands April 2nd, base_unittests will be running in isolate mode for Linux and Windows. E.g. the test is running isolated in a temporary directory on the local VM.

This means that starting April 2nd: base_unitttests.isolate will need to be maintained. Any new test files that aren't listed in it will not be accessible to the test.

Plan

Once base_unittests is converted, if there are no major problems, on April 9th the CQ will begin to run base_unittests through Swarm for Linux and Windows. I don't expect there be be any noticeable change in the run times from this change. 

Then all the remaining tests will be assimilated converted, which should finally complete our OKR of "running all tests in O(1)". M-A, Ben and I are working on making the .isolate format more human editable so it should eventually be self-serve.

How to try this out at home

http://chromium.org/developers/testing/isolated-testing/isolate-basics

If anyone has any problems or concerns with this plan, please let me know.

Thanks,

C#

Quick glossary

run_test_cases: Runs a single executable's test cases in parallel on a single machine, like sharding_supervisor. It assume the VM has more than one CPU.

isolate: runs a test outside of a checkout. This means listing all the run time dependencies,

http://www.chromium.org/developers/testing/isolated-testing

swarm: runs an executable on an arbitrary worker VM. Swarm knows about google-test so it can shards the work across multiple VMs to further reduce the end-to-end latency.

http://www.chromium.org/developers/testing/isolated-testing/swarm

[Christopher Sharp]

This change landed this morning as r194588. Any tests files accessed by base_unittests must now be listed in base_unitttests.isolate

Future Plans:

Tomorrow: Start running net_unittests in isolate mode for Linux and Windows tomorrow

April 29: Start running unit_tests, browser_tests and interactive_ui_tests in isolate mode for Linux and Windows

C#

[Marc-Antoine Ruel]

Hi

net_unittests was enabled just now on linux and Windows. base_unittests has been enabled since the last email.

You didn't feel anything? That's by design. Still, read on.

The Try Server will catch any problem with missing entries in the .isolate file. Given the low churn of the test data files on these two tests, I do not foresee any problem.

Since each test has a "data/" or "test/data/" directory and they are listed in the corresponding .isolate file, all the files inside are mapped automatically. For example if you look at net_unittests.isolate line 37, you can see that everything under data "data/", which is really "src/net/data/", is automatically included. So add your new test data files in the right data/ directory and you'll be fine. In practice, changes to .isolate should be rare and are usually new binaries in <(PRODUCT_DIR).

Note: Anything on OSX or Windows x64 or Debug is not running in isolated mode for now. We'll try to fix OSX and Windows x64 soonish, but this will only be in Release for now.

M-A

2013/4/17 Christopher Sharp <csh...@chromium.org>

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
 
 
 

[Christopher Sharp]

Base_unittests are now running through swarm for linux_rel and win_rel jobs that have testfilter='defaulttests'.

Currently some test runs are failing due to missing keys, I'm working on this and it should be fixed in a few hours. If you see any other errors, please let me know.

Missing Key Error Output:

Error: Unable to find any tests with the name, swarm_triggered-*-base_unittests, on swarm server

Error: No test keys to get results with

Thanks,

C#