[CCPPETMR/SIRF-SuperBuild] Gadgetron tests fail but Travis stays green (#192)

[Richard Brown]

Travis shows that all builds are green for the Superbuild (e.g., most recent build).

However, if you check one of the OSX builds (e.g., this one), you'll notice:

3: The following tests FAILED:
3: 	  1 - test_all (Failed)
3: Errors while running CTest
3/3 Test #3: Gadgetron_TESTS ..................***Failed   20.27 sec

These tests are also failing on SIRF, but they are actually showing up as having failed (e.g., link).

The reason they should be red is that they are failing the Gagetron unit tests (added in PR #182). This is a problem to be addressed elsewhere, but Travis should be flagging up the fact that it fails.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[Casper da Costa-Luis]

possibly unrelated, but:

• https://stackoverflow.com/questions/51289456/test-for-non-zero-exit-status-using-ctest-cmake
• https://stackoverflow.com/questions/33693486/how-can-i-use-cmake-to-test-processes-that-are-expected-to-fail-with-an-exceptio

[Richard Brown]

I think these are unrelated, as they are for cases that are expected to fail, so we want to return the opposite of the exit code of the script.

In the case of Gadgetron, if the test fails, we want to report that it fails.

Since the ctest fails, but travis is happy, my guess is that the answer lies in the way we've set up the .travis.yml (not sure though).

[Kris Thielemans]

I guess this then needs

ctest -VV || travis_terminate 1

in the .travis.yml

[Richard Brown]

Ok, waiting to see what travis has to say about that: https://travis-ci.org/CCPPETMR/SIRF-SuperBuild/builds/483358628.

[Casper da Costa-Luis]

doesn't explain why SIRF fails without needing the travis_terminate

[Richard Brown]

This seems to fix it link, but also shows up the docker builds in red. @casperdcl is this normal?

[Casper da Costa-Luis]

ah I didn't look at the correct .travis.yml - was looking at SIRF, not the superBuild. The PR needs some modification, will do

[Richard Brown]

Closed #192 via 4475f2b.

[Richard Brown]

Reopened #192.

[Richard Brown]

This has started happening again. Most recent error that should be causing it to be red happened during the cmake of SIRF:

CMake Error at /usr/local/Cellar/cmake/3.11.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):

  Could NOT find FFTW3 (missing: FFTW3F_LIBRARY FFTW3_INCLUDE_DIR)

Call Stack (most recent call first):

  /usr/local/Cellar/cmake/3.11.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)

  /Users/travis/build/CCPPETMR/SIRF-SuperBuild/INSTALL/lib/cmake/ISMRMRD/FindFFTW3.cmake:114 (find_package_handle_standard_args)

  src/xGadgetron/CMakeLists.txt:36 (find_package)





-- Configuring incomplete, errors occurred!

See also "/Users/travis/build/CCPPETMR/SIRF-SuperBuild/builds/SIRF/build/CMakeFiles/CMakeOutput.log".

make[2]: *** [builds/SIRF/stamp/SIRF-configure] Error 1

make[1]: *** [CMakeFiles/SIRF.dir/all] Error 2

make[1]: *** Waiting for unfinished jobs....

[Edoardo Pasca]

on what OS?

[Richard Brown]

both unfortunately...