When are rackunit tests delayed? (different between 7.8 and 8)

[William J. Bowman]

Below is an example that behaves "correctly", as in all tests run and are
counted as failured or errors correctly, in Racket 7.8, but which crashes in
Racket 8.

> #lang racket
>
> (require
> rackunit
> rackunit/log)
>
> (define (suite1)
> (test-suite
> ""
> (test-begin
> (check-not-equal? (error "actual") (error "expected")))))
>
> (define (suite2)
> (test-suite
> ""
> (check-not-equal? (error "actual") (error "expected"))))
>
> (module+ test
> (require rackunit/text-ui)
>
> ;; Correctly counts the tests as errors in 7.8 and 8.0
> (check-pred
> integer?
> (run-tests (suite1)))
>
> ;; Counts the tests as errors in 7.8, but crashes in 8.0
> (check-pred
> integer?
> (run-tests (suite2)))
>
> ;; Gets run in 7.8, but not in 8.0
> (check-equal? 0 0))

This has something to do with when test-suite delays a test, which seems
inconsistent across the Racket versions.
I'm not sure whether the problem was some undefined behaviour in test-suite or
not.

I'm a bit confused about the semantics of test-suites and tests, since the
documentation claims a test (unlike a check) is delayed, yet test-case and
test-begin do not delay tests, while test-suite does produce a delayed suite of
tests.
However, test-begin DOES seem to delay a test in the context of a test-suite.

I'd appreciate any insight.

--
William J. Bowman

[jackh...@gmail.com]

The test-suite semantics in rackunit are complex, under-specified, and quite brittle. I recommend avoiding them entirely and sticking entirely to test cases and test-begin.

[Sorawee Porncharoenwase]

This is a regression caused by me (https://github.com/racket/rackunit/commit/d7665ddc17405b25c7365eda5c560b671340853d). I will try to find a way to fix.

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/73510d26-7405-4c0a-a0e3-1b06f77f3576n%40googlegroups.com.