running Fitnesse from Jenkins on different hosts, getting test results

[Chris McMahon]

Hi,

I've been investigating doing this and wondered if anyone else has solved it, using the Fitnesse plugin for Jenkins. 

From what I can tell,  running a Fitnesse suite from Jenkins yields a "fitnesse-results.xml" file on the Fitnesse host under a directory named after the Jenkins build name.   That file seems to be moved from the Fitnesse host to the Jenkins host and in Jenkins properly displays Right/Wrong/Ignored/Exceptions values. 

The contents of "fitnesse-results.xml" seem to be a slightly altered version of the normal results file from a Suite run found under the testResults directory of Fitnesse.  Both files contain a <pageHistoryLink> tag pointing to the results of individual tests within the Suite that was run. 

However, on the Jenkins host there are no test history files available, and on the Fitnesse host, the default location for "fitnesse-results.xml" isn't going to find the individual test results correctly. 

Since I want to have the test history files available in the proper directory to be accessed by "fitnesse-results.xml" from within Jenkins, it seems the correct approach would be to use the "Publish artifacts to SCP Repository" feature in Jenkins to get the test history files from the Fitnesse host to the Jenkins host.  For various unrelated reasons I have so far been unable to try doing that and I just wondered if anyone else has gone down this road and has any advice. 

Thanks,
-Chris

[Lisa Crispin]

The FitNesse plug-in for Jenkins/hudson is lame. Bob expects you to go out and look at the history files in FitNesse, or something, to see the results.

We run our FitNesse tests in Jenkins with a JUnit that Gojko Adzic wrote. Here is our source. The comments tell you how to use the command line for running FitNesse. I can provide more details if needed. We can look at FitNesse results for each run in Jenkins exactly the same way as we would thru the FitNesse wiki. It rocks.

package com.eplan;

import fitnesse.junit.JUnitHelper;
import fitnesse.junit.JUnitXMLTestListener;
import junit.framework.TestCase;

/**
 * Runs a given Fitnesse suite from a given home, and produces output to the given directory.
 * <p>Behavior is modified through system properties:
 * <ul>
 * <ol>fitnesse.home - The directory containing FitNesseRoot
 * <ol>output.dir - Directory where fitnesse output is to go
 * <ol>suite.name - Full url/name of suite to run
 * </ul>
 */
public class SuiteJUnitTest extends TestCase {
    private JUnitHelper helper;
    String xmlOutputDir = System.getProperty("xml.output.dir");
    JUnitXMLTestListener xmlTestListener = new JUnitXMLTestListener(xmlOutputDir);

    public void setUp() {
        helper = new JUnitHelper(System.getProperty("fitnesse.home"), System.getProperty("html.output.dir"), xmlTestListener);
        helper.setPort(5000);
    }

    public void testFixtureJUnit() throws Exception {
        helper.assertSuitePasses(System.getProperty("suite.name"));

--
Lisa Crispin
Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers and Agile Teams_ (Addison-Wesley 2009)
Contributor to _Beautiful Testing_ (O'Reilly 2009)
http://lisacrispin.com
@lisacrispin on Twitter
http://entaggle.com/lisacrispin

[Dan Woodward]

I have an approach for this that we use and that I keep meaning to fully document for the world, but then I go home at night and just don't get around to it.

We don't use the plugin, and cannot use Gojko's solution either because of our environment setup.

Here's what we do, roughly:

1. We setup a wiki server for fitnesse as a service on the computer to run in the background.  We call this the history server.
1. The service points at the same folder as the checkout of fitnesse run in Hudson
2. The service, since not run from our command line environment, doesn't contain the env-variables to be runable, so it is just for viewing
2. We have an ant target that we run to execute the tests, it does the following:
1. Launches our product to be tested and Selenium server (if it is a web test)
2. Executes the fitnesse suite using the command line runner as a java task
3. Executes a command to fetch the last result for the suite
4. Transforms that result into Junit.XML format for Hudson to report
5. Transforms that result into an HTML page with links to the test result in the wiki server mentioned in step 1

When it is done, we can see the trend of passes and fails, just like a junit.  We can also view a web page that allows us to link through to the detailed results of the test rendered as HTML.

Gojko's solution is undoubtedly more elegant and a better choice if it works for you.  If, like us, the environment doesn't support the JUnit runner, I can share with you the pieces that we used to get our system to work.  

Daniel A Woodward

----------------------------

FitNesse using Slim and Java

@woodybrood

http://whotestedthis.com 

[Lisa Crispin]

Speaking of FitNesse, last year we put our FitNesse tests into SVN. We soft link the FitNesse directory where the tests live to our SVN project (or maybe vice versa, now I've confused myself) so we can edit the content.txt files in IntelliJ Idea, and at least get some benefits of the IDE, and manage the files under SVN there, but we can still run them via the FitNesse wiki. (it's possible to run them thru IntelliJ as well, now I've forgotten the name of the tool that lets you do that). It greatly improved my quality of life.

[Chris McMahon]

We've had our selenesse tests in git from the start, it makes life a lot easier.  But I do wish the Fitnesse Jenkins plugin were a little smarter than it is.

-C

[Lisa Crispin]

Maybe if more people asked Uncle Bob and Micah for a better plug-in, they would respond... There was no plug-in at all when we started running FitNesse tests from Hudson.