How to let Bazel expect a test to crash or failure

[ben...@gmail.com]

Hi,

I'm currently working on a windows project using bazel. There's a component which write the dump file the program crashes. I'm trying to write a unit test for this component. In the test I intentionally created a crash and let the component to write the dump file. In previous CTEST, it is possible to tell CTEST that this test is expected to crash.

Is there anything similar in bazel?

Thanks!

[Brian Silverman]

I've been wrapping those kind of tests with sh_tests that basically invert the exit codes. Wrapping it in a macro makes it easy to reuse. Like this for C/C++ tests, and similarly for other rule types:

def _xfail_test(do_binary, name,

                size = None, timeout = None, shard_count = None,

                visibility = None, **kwargs):

  if 'flaky' in kwargs:

    fail('not supported for xfail_tests', 'flaky')

  if 'args' in kwargs:

    fail('not yet implemented for xfail_tests', 'args')

  if 'shard_count' in kwargs:

    fail('not yet implemented for xfail_tests', 'shard_count')

  do_binary(

    name = name + '__binary',

    visibility = ['//visibility:private'],

    testonly = True,

    **kwargs

  )

  native.sh_test(

    name = name,

    visibility = visibility,

    size = size,

    timeout = timeout,

    shard_count = shard_count,

    srcs = [

      '//tools/test:run_xfail_test',

    ],

    data = [

      ':%s__binary' % name,

    ],

    args = [

      '$(location :%s__binary)' % name,

    ],

  )

def cc_xfail_test(**kwargs):

  _xfail_test(native.cc_binary, **kwargs)

Where //tools/test:run_xfail_test is a filegroup containing a .sh file that has this:

set -u

set -e

readonly TEST_BINARY="$1"

shift 2

set +e

"${TEST_BINARY}" "$@"

RESULT="$?"

set -e

if (( ${RESULT} == 0 )); then

  echo "Expected test to fail, but it succeeded" >&2

  exit 1

fi

Using it is just importing cc_xfail_test from wherever you put that .bzl file and then writing cc_xfail_test instead of cc_test.

I also have some additional complexity to allow certain shards of a sharded test to be required to fail and others to succeed, but that seems kind of excessive thinking about it now. The basic idea is to have the _xfail_test macro call a custom rule (needs to be a custom rule instead of a genrule to allow using select) to write out the information to another file which is passed as data like __binary, and then the shell script parses that file if you want to add something like that.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/53e62167-e30f-4734-af9f-654129a66f90%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ben...@gmail.com]

Hi Brian,

thanks for your reply. I'll check it out!

> To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.