Jenkins Integration

[Luke Robertson]

Hi,

I have configured a Jenkins CI server which uses the maven-dotnet-
plugin
to build .NET projects which uses Gallio to run NUnit tests.

The tests run fine, and we receive the expected number of failures -
however we get the following output:



[INFO] Generating reports.

[INFO] Disposing the test runner.
[INFO] Stop time: 11:41 (Total execution time: 18.124 seconds)
[INFO]
[INFO] 168 run, 157 passed, 11 failed, 0 inconclusive, 0 skipped
[INFO]
[JENKINS] Archiving D:\Application-Data\Jenkins...
[INFO]
------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Failure during the Gallio tests generation that ended with
status=1
[INFO]
------------------------------------------------------------------------

Does Gallio automatically exit with status=1 when failures are
detected?
This obviously shows the build as failing when it has built
successfully
(i.e. a Red ball on Jenkins instead of a yellow ball). Is there
currently
a solution to get this to work?

Thanks, Luke

[Jeff Brown]

Yes, the Gallio.Echo result code varies depending on whether the tests ran successfully:

    /// <summary>
    /// Describes the result codes used by the application.
    /// </summary>
    public static class ResultCode
    {
        /// <summary>
        /// The tests ran successfully or there were no tests to run.
        /// </summary>
        /// <value>0</value>
        public const int Success = 0;

        /// <summary>
        /// Some tests failed.
        /// </summary>
        /// <value>1</value>
        public const int Failure = 1;

        /// <summary>
        /// The tests were canceled.
        /// </summary>
        /// <value>2</value>
        public const int Canceled = 2;

        /// <summary>
        /// A fatal runtime exception occurred.
        /// </summary>
        /// <value>3</value>
        public const int FatalException = 3;

        /// <summary>
        /// Invalid arguments were supplied on the command-line.
        /// </summary>
        /// <value>10</value>
        public const int InvalidArguments = 10;

        /// <summary>
        /// No tests were found.
        /// </summary>
        /// <value>16</value>
        public const int NoTests = 16;
    }

--
You received this message because you are subscribed to the Google Groups "gallio-user" group.
To post to this group, send email to galli...@googlegroups.com.
To unsubscribe from this group, send email to gallio-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gallio-user?hl=en.

[Espen Albrektsen]

We use the Gallio Plugin - and also we did a couple of tricks to get
both the XML and HTML versions of the test report.

Our Jenkins test runner is a windows batch task that does this:

:: "Set default run properties:"
set GALLIO="%PROGRAMFILES%\Gallio\bin\Gallio.Echo.exe" /report-
directory:"%WORKSPACE%\TestReports" /report-type:xml /
runner:IsolatedProcess /verbosity:debug YourUnitTestDell.dll
:: Run the tests
%GALLIO% /rnf:QuickTest /filter:Category:QuickTest
:: Store away the gallio exit code:
SET ERR1=%ERRORLEVEL%

:: "Now convert XML to more readable HTML: (Can be skipped, by
cancelling the build, but then no artifacts will be stored)"
cd "%WORKSPACE%\TestReports"
"%GALLIO_3_3_312%\bin\Gallio.Utility.exe" FormatReport QuickTest.xml /
rt:html
:: Make sure a failing test will fail the build:
EXIT /B %ERR1%

If you want builds with failing tests to go yellow, not green - simply
remove the last line (EXIT /B %ERR1%). That way the "ERRORLEVEL" of
the unit test batch command is allways 0 - build succeeds, but since
the XML shows failing tests, Jenkins will mark the build yellow.

> On Tue, May 10, 2011 at 3:49 AM, Luke Robertson <lrobertso...@googlemail.com

[JenkinsUser]

 Hi Espen,

 

I have followed your steps, but I can't find any reports on the jenkins workspace folder.  Where can I find the report on the directory?

[Espen Albrektsen]

Hi - sorry about the delayed response.

You control the report folder with the switch /report-directory:"yourpath"

In the sample below, I use the Jenkins environment variable %WORKSPACE% to make the path fully qualified.

So you should expect to find reports in the %WORKSPACE%\TestReports folder.

You may also use /rnf switch to set the report-name-format - this would control the _name_ of the report...