Advice for submitting file queue from multiple users?
AndyNic
We've been using Jenkins internally at my work for a while now. We're
considering trying to expand Jenkins' usage to providing a "gauntlet"
type of test screening prior to allowing developers to submit into our
subversion repositories.
Some background: We have an enormous set of serialized, time-consuming
tests (5-8 hour) which *should* be run prior to submitting and we've
found that developers often take risky shortcuts by omitting (e.g.,
subsampling) tests. The testing shortcuts have caused caused repeated
extremely expensive, time-consuming downstream problems for our entire
organization.
Obviously breaking the serialization of our tests would make
everything go faster, but... Maintaining a personal massive parallel-
test system seems too difficult to foist on each one of our
developers. So, we think that we could maintain a single, massive
parallel test system which would have much faster throughput than
running our test-suite, serialized on each developer's desktop
machine. This has the advantage of freeing-up the developer's local
machine for additional local testing.
We are hoping that we could accelerate our testing process using
Jenkins to accept a package of developer-locally-built binaries the
form of an incoming (600 MB compressed) .zip file from a developer.
Then trigger a series of downstream tests... and then send the results
of those tests back to the submitter via email. Submitting the 600
MB .zip file of our built binaries from a developer's local build
seems like the fastest and least resource-consuming procedure (11
seconds via 7zip to compress to 600MB -vs- 2+ minutes to compress all
of the sources to 3.4GB).
I've spent some time reading the Jenkins wiki trying to figure out
what series of plugins to use, but didn't see anything obvious which
would allow a queue of files to be copied to a single machine to then
trigger a Jenkins "build" (start running our test-suites). We may have
20 users who all simultaneously decide "I have created a local build
which needs to be tested". We have enough disk-space to handle
queueing all of the submitted files. Each developer can handle typing
"submit-to-test" from the command-line on Windows... but not more
complicated procedures.
I guess I'm wondering if we need to build some kind of file-queueing
mechanism which triggers Jenkins to start or if that sort of plugin
already exists? Ideally we'd like to be able to remotely monitor the
state of the submission-queue because we suspect that the queue would
occasionally have trouble.
Thanks,
andy
AndyNic
that
shipping a patch-file, while tiny, requires that the Jenkins machine
perform a time-consuming (1 hour) build locally. We'd like to avoid
this
if possible.
Les Mikesell
> Also, one of the reasons that we want to ship binaries to Jenkins is
> that
> shipping a patch-file, while tiny, requires that the Jenkins machine
> perform a time-consuming (1 hour) build locally. We'd like to avoid
> this
> if possible.
It should work to set up a subversion repository - perhaps a separate
one for each developer - where they can commit the binaries and
jenkins will notice and queue the job. It is difficult to ever
remove anything from a subversion repository, so you'd want to be able
to delete the whole thing periodically and start over. If the changes
between runs are small, I wouldn't zip the files - just let subversion
manage the binary diffs between revisions.
--
Les Mikesell
lesmi...@gmail.com
Sami Tikka
build is parameterized". Click on the popup that appears and choose
"File parameter" and you're mostly done :) The rest is fine-tuning.
Because your test run is time-consuming and you probably have multiple
developers, you should allow Jenkins to run multiple builds of this
job in parallel. You get this by checking the box "Execute concurrent
builds if necessary". Also tune the number of executors according to
the number of tests you think your server can run in parallel.
You might find that you need to run more parallel test runs than your
server can handle. Then you can get another server and configure
Jenkins to run a slave on the new server and thus increase the
capacity of your Jenkins.
As you are probably aware already, your most pressing problem is the
long test duration. When the tests take so long, the developers invent
new ways to avoid running them and the ones who run them waste a full
day waiting for feedback.
If I were you, immediately after setting up the test system described
above, I would start working on being able to run the tests in
parallel.
Where I work every developer has their own personal Jenkins job. The
Jenkins job is set to build from the developer's personal git repo and
run tests. (You could maybe use work-in-progress (wip) branches if you
use Subversion.) The test run deploys a cluster of vms, installs the
build artifacts on the vms and runs tests as much in parallel as
possible. This takes about 30 minutes.
A developer is only allowed to submit his changes to the main
repository/master/trunk (take your pick) if he can prove his changes
do not break anything by showing a build of his personal Jenkins job
that has tested his changes.
-- Sami
2012/3/11 AndyNic <andy.nich...@gmail.com>: