Changes to tests build system integration (mostly xpcshell for now)

[Gregory Szorc]

Just landed in inbound and bound to stick sooner or later are some
changes to how tests are integrated with the build system.

* The per-directory xpcshell make targets have been removed. Use mach
(preferred) or TEST_PATH=... make xpcshell-tests top-level make target.

* xpcshell.ini files are now required to annotate their "support" files
so the build system knows what needs to be installed. i.e. files added
to xpcshell test directories are no longer automatically packaged by
default.

* Support for test manifests for all the flavors of mochitests have been
added to moz.build files. This means we should not be defining
mochitests in Makefile.in any more. Instead, create {browser-chrome.ini,
mochitest.ini, chrome.ini, a11y.ini} files (like xpcshell.ini files) and
reference said files in moz.build files via A11Y_MANIFESTS,
BROWSER_CHROME_MANIFESTS, MOCHITEST_MANIFESTS, and
MOCHITEST_CHROME_MANIFESTS. Bug 920185 has been filed to track this
effort. **Moving files over and reducing the total number of Makefile.in
and moz.build files will make the tree build faster.**

* Test manifests now support some additional features. Run |mach
build-docs| or see
http://build-docs.paas.allizom.org/test_manifests.html for the canonical
documentation.

[Gregory Szorc]

Just landed in inbound is a mass conversion of mochitests to use manifests.

Not all references were moved. However, the intention is to move them
all eventually. If you want to help with the effort, the tracking bug is
920185. See
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=22a30d73daf6
for what is involved.

Please try to avoid adding new mochitest-related functionality to
Makefile.in files, as you'll just be creating more work for the eventual
conversion.

The manifest file format is documented at
https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/test_manifests.html.

> _______________________________________________
> firefox-dev mailing list
> firef...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev

[Karl Tomlinson]

Gregory Szorc writes:

> Just landed in inbound is a mass conversion of mochitests to use manifests.

When adding a new test, the process used to be:

1. add new file path/to/test/directory/new-file
2. add "new-file" to MOCHITEST_FILES in path/to/test/directory/Makefile.in
3. make -C obj/path/to/test/directory
4. run test

Is the following meant to work now?

1. add new file path/to/test/directory/new-file
2. add "[new-file]" in path/to/test/directory/mochitest.ini
3. make -C obj/path/to/test/directory
4. run test

I tried the above without success.
I also tried replacing 3 with "./mach build path/to/test/directory"
without success.

Is there a new passphrase?

[Gregory Szorc]

Until bug 920184 lands (patch up for review), you'll need to run the
following to ensure test files are up to date:

$ NO_REMOVE=1 mach build install-tests

If you are simply modifying test files (not adding) and your platform
supports symlinks (not Windows), you likely don't have to run this.

I apologize for the temporary inconvenience. We intended bug 920184 to
land as close to the manifest patches as possible. Chaos around Summit
travel is partially to blame :/

[Gregory Szorc]

On 10/1/13 10:43 AM, Gregory Szorc wrote:
> On 10/1/13 10:26 AM, Karl Tomlinson wrote:

> Until bug 920184 lands (patch up for review), you'll need to run the
> following to ensure test files are up to date:
>
> $ NO_REMOVE=1 mach build install-tests
>
> If you are simply modifying test files (not adding) and your platform
> supports symlinks (not Windows), you likely don't have to run this.
>
> I apologize for the temporary inconvenience. We intended bug 920184 to
> land as close to the manifest patches as possible. Chaos around Summit
> travel is partially to blame :/

Bug 920184 just landed in inbound.

While I'm here,
https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/test_manifests.html
contains the documentation for manifests, complete with the set of
variables available to filter on.