Proposal: ExUnit.on_exit callback should always run before the a test process dies

[Anthony Accomazzo]

Hey all,

As I understand it, the `on_exit/2` callback is *not* guaranteed to complete before a test process exits *if the test has a failure*. For example, consider this flow:

```
1. Boot a process tree (Supervisor with child processes)
2. Register an `on_exit/2` callback to stop the Supervisor
3. Make an assertion -- the assertion fails
```

Here, the assertion failure will crash the test process before `on_exit/2` runs.

Assuming I'm understanding this correctly, let me share a very common pain point for our team. A test will fail, and we'll see an output like this:

```
** (Mox.UnexpectedCallError) no expectation defined for MyMock.some_func/0

1) some failing test
match (=) failed
:ok = Module.some_operation()
```

The issue is that we get error output *that was actually a result of the test pid being torn down before the process tree*. So, it's a red herring. There was *another* bug that caused the assertion to fail. In this case, the Mox.UnexpectedCallError just happened because the test_pid went DOWN while the process tree was still running, and they called a mock registered to that test_pid and failed.

However, in certain situations, it's super hard to know if the error output is signal or noise.

You can imagine that in the place of a Mox error, we have the same situation with database connections, no_procs when we try to send signals back to a dead test_pid, etc.

We think it would be ideal to be able to register a callback that will always run before a test_pid exits, regardless of if the test ran successfully or not.

What do you think? Or what other strategies might we employ?

Thanks,

Anthony

[José Valim]

on_exit is guaranteed to run *after* the test process finishes.

It is impossible to guarantee that something runs before it exits, due to links.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/52c85fb1-6a4e-4646-910f-aeb95ec90efdn%40googlegroups.com.