Advice requested: bash vs python scripts in Jenkins

[David Aldrich]

Hi

We use Jenkins to run regression tests on our C++ console application. Each test consists of a bash script that typically runs the application with defined test parameters and then compares the output files with reference files using 'diff'.

I am now thinking of moving from bash scripts to Python. My expectation is that Python will be much easier to code.

To use Python, would I still use an "Execute shell" build step and specify the following?

/path/to/build.py -i

When using a bash script, I know that the test will fail if any script command fails. In an equivalent Python program should I just ensure that my Python program returns an error code if any test step fails?

Best regards

David

[clem]

Hi,

yes you are correct.
Im running my python scripts like this : #python "location/file.py" parameter paramerter# in a "Execute Window Batch command"

You can catch error messages in your python script with : sys.exit('Your Error Code');
For example :
if re.match(---)
    do this
else:
   sys.exit('Not matching parameter x.')

So in case of an Script failure, it will stop and this "Error Code" will published at the Jenkins console
and stop the build.

Clem

[clem]

Just to make sure that there is no misunderstanding there are no # at my python line :)

Greetings

[David Aldrich]

Thanks Clem,

Best regards

David

[Mark Waite]

Yes, that technique will work, and switching to Python then opens you to the opportunity to write your tests in C or in Python using the Python ctypes library, and would allow you to use either "py.test" or "nose" to automatically discover, execute, and parallelize your tests.

Using that technique would also let you easily write the success / failure status of your tests in JUnit output format so that you could have "success", "unstable", and "failed" states for your builds, where "success" means everything compiled and all tests passed, "unstable" means everything compiled and some tests failed, and "failed" means that compilation failed.

There is a "testing in python" mailing list which can help with questions about py.test and nose.

Mark Waite

---

From: David Aldrich <David....@EMEA.NEC.COM>
To: "jenkins...@googlegroups.com" <jenkins...@googlegroups.com>
Sent: Friday, November 23, 2012 4:47 AM
Subject: Advice requested: bash vs python scripts in Jenkins

[David Aldrich]

Hi Mark and Clem

Thanks very much for your replies. They were both very helpful.

Best regards

David