Best pluging for hierarchical, parametrized jobs, aggregating test results

[Gaetano Giunta]

Hello all. A noobie question.

We have a platform using Jenkins to drive functional tests of a web platform .

The tests are executed using Selenium (via behat/mink, controlled by a phing script).
There is a huge combination of tests to execute (testing the dev vs. prod servers, a particular site out of 50 deployed, a particular feature set, etc...), so all of the test scripts make heavy usage of parameters to drive execution. About 10 browsers are available to Selenium to execute the tests in parallel.
Each test produces junit-formatted output.

The requirements we have are:
1. create parametrized jobs, so that the job parameters can be used to drive the tests (so we don't need to write thousands of jobs, maybe just a few dozens)
2. create groups of jobs, which are executed in parallel. The master job driving the parallel ones should also be parametrized and pass on some of its parameters to the slave jobs
3. have multi-level job groups
4. aggregate tests results from all the groups and display them in their master jobs. Ideally we'd like to have:
    - the graph showing evolution of failed/passed tests over job builds, in the main-page for each job, displayed usually at top-right
    - the detailed tables displaying all test results, in the main-page for each job and for each job build as well, displayed usually as an icon+link in the center column, below the 'recent changes' and 'workspace' links

So far, point 1 has proven easy, and 2 and 3 doable.

For point 4, we are struggling a bit, as there seems to be a lot of master/slave plugins available and described in different blog posts, but none which fits the needs perfectly, at least out of the box.

We tried:

a) "MultiJob Project"
+ works with passing parameters on to child jobs
+ has a nice graphical display of the child jobs in the main job page of master job, showing status of each child job
- does not display the aggregated tests trend graphs nor table. This seems to be possible to do, but complex to set up, with custom scripts needed to copy the junit xml outputs around

b) "Build flow" job
+ works with passing parameters on to child jobs
+ does display the aggregated tests table
- does not display test trends graph
- does not display the child jobs in its main page (given the nature of the build-flow plugin, this is most likely close to impossible anyway)

Any suggestion?

Thanks
Gaetano