[vim/vim] test suite is slow because it cannot run in parallel (PR #20094)
Jesse Rosenstock
Problem: The test suite takes over 17 minutes serially and cannot
be run with make -j because tests share the testdir/
working directory and create temporary files with
well-known names (X*, test.log, messages, vimcmd).
Solution: Create per-test isolated working directories under
workdir/ using symlink trees that mirror the real
directory layout, keeping temp files isolated while
preserving all relative path assumptions.
(Jesse Rosenstock)
Each test runs in workdir/$test/testdir/ where the symlink tree preserves relative-path depth: ../ resolves to src/ contents and ../../ resolves to the vim root. The layout is:
workdir/ symlinks to vim root (runtime/, ...)
workdir/$test/ symlinks to src/ contents
workdir/$test/testdir/ symlinks to testdir/ contents, test cwd
Level-1 symlinks are created once in the nolog target rather than per-test, because ln -s follows existing symlinks to directories (creating e.g. runtime/runtime in the repo root) and there is no race-free way to create them from parallel test processes. A setup_workdir.sh POSIX shell script creates the per-test levels 2 and 3.
The Makefile avoids GNU Make extensions to stay POSIX-compatible: setup_workdir.sh replaces a define/call macro, shell TESTDIR=pwd replaces $(CURDIR), and recursive $(MAKE) replaces order-only prerequisites (|).
The test_stacktrace.vim Filepath() function is updated to use getcwd() for Xscript files since they are created in the workdir cwd, while runtest.vim is resolved from the original testdir.
An alternative approach would eliminate the symlink tree entirely by rewriting all ~127 relative path references (../ and ../../) across 35 test files to use absolute paths via g:testdir. This was rejected because: (1) it touches many files unrelated to the parallelism fix, creating review noise and merge conflicts; (2) tab-completion and fnamemodify tests genuinely test relative path resolution against real directory structure, requiring per-test fixture directories anyway; (3) future test authors must remember to avoid ../ in favor of g:testdir, an easy regression with no enforcement mechanism; and (4) the symlink approach preserves the natural relative-path idiom that existing tests use, keeping the diff focused on infrastructure.
On a 48-core machine, wall clock time drops from 17m32s to ~50s.
This supersedes 6146f33 / #20082 (patch 9.2.0410: test suite races when run with parallel make) which added .NOTPARALLEL as a workaround for the same race conditions.
related: #20082
You can view, comment on, or merge this pull request online at:
https://github.com/vim/vim/pull/20094
Commit Summary
- 277ae86 test suite is slow because it cannot run in parallel
File Changes
(6 files)
- M .gitignore (2)
- M src/testdir/Makefile (202)
- M src/testdir/runtest.vim (32)
- A src/testdir/setup_workdir.sh (38)
- M src/testdir/test_stacktrace.vim (4)
- M src/testdir/util/shared.vim (10)
Patch Links:
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.
Jesse Rosenstock
@jmr pushed 1 commit.
- 781e64e test suite is slow because it cannot run in parallel
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.
Jesse Rosenstock
@jmr pushed 1 commit.
- 20f2db5 test suite is slow because it cannot run in parallel
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.
Jesse Rosenstock
@jmr pushed 1 commit.
- 30a7be6 test suite is slow because it cannot run in parallel
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.
Jesse Rosenstock
@jmr pushed 1 commit.
- 96ee796 test suite is slow because it cannot run in parallel
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.
Christian Brabandt
chrisbra left a comment (vim/vim#20094)this has stalled, so closing
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.