Idea for new ExUnit feature: teardown(:guaranteed)

[Martin Schurrer]

Currently ExUnit's teardown hooks run in the same process as the test.
This means teardown will not run under lots of circumstances, e.g. if
the process exits.

I propose adding an option :guaranteed to teardown which makes ExUnit
ensure that the teardown hook gets called, either in-process or if that
process is dead in a new process.

That would make implementing all kinds of cleanup tasks like TRUNCATEing
a test DB or cleaning up spawned processes a breeze.

Thoughts?

--

Kind regards,
Martin Schurrer

[Peter Hamilton]

Would you ever not want it guaranteed?

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[José Valim]

That's an excellent point and an excellent idea.

The proposed solution is tricky though because in which process would it run? It would considerably change the semantics of teardown due to an option... maybe it would be better to have an easy way to spawn a process that monitors the process running the test and runs a callback on the test exit? Something like:

    setup do

      on_exit fn ->

        # clean up the DB

      end

    end

Of course, those callbacks would need to be synchronously after the test exits so we do need some kind of support from/integration with the test runner.

Maybe it is worthy deprecating teardown? A teardown that does not run all the time is not really useful...

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[José Valim]

Well maybe we could just fix teardown and make it clear it runs in another process... the on_exit feature though is interesting because we could register things dynamically, for example, only if a particular feature is used.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[José Valim]

on_exit would also help with the current issue where a setup fails but we don't know what is the opposite teardown for that setup is. So we are torn in between running all teardown (which could potentially rollback a transaction when one was not even started) and none (which would not rollback the transaction at all).

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Martin Schurrer]

On 20/05/14 20:58, Peter Hamilton wrote:
> Would you ever not want it guaranteed?

When I know my cleanup will only be needed when the process still
exists. E.g. cleaning up process dictionary or ets tables.

On 20/05/14 20:59, José Valim wrote:
> Maybe it is worthy deprecating teardown? A teardown that does not run
> all the time is not really useful...

IMO teardown is a well known pattern and I'd expect it to be there in
ExUnit for all the cases where I only need teardown and no setup.

I originally expected teardown to simply be guaranteed to run. I thought
ExUnit would just run it in-process or in a newly spawned one, but it
would definitely always run it.

On 20/05/14 21:00, José Valim wrote:
> Well maybe we could just fix teardown and make it clear it runs in
> another process... the on_exit feature though is interesting because
> we could register things dynamically, for example, only if a
> particular feature is used.

Agreed. Both features are useful IMO. I think having even more options
on teardown might be worth it. Consider some use cases:

1. Cleaning up process dictionary or ets tables: Don't care if the
process died, don't need to run then.

If run in a different process and it e.g. does ets:all and then filters
by ets:info(owner) or does :erlang.erase this might interfere with
ExUnit (or whichever process it is run in).

2. Rolling back an open Postgres connection: This needs to run but
doesn't especially care about the process it's run in (as long as it can
get at the Postgres connection) Also it needs to know if the setup hook
actually ran.

3. Killing spawned processes: This doesn't care where it runs (since it
would just do :gen_server.call(:some_name, :die) or Process.exit/2

And how about an around hook? (with yield ;) )

[José Valim]

Agreed. Both features are useful IMO. I think having even more options on teardown might be worth it. Consider some use cases:

1. Cleaning up process dictionary or ets tables: Don't care if the process died, don't need to run then.

You get a new process per test so you can just let the process crash if the reason is cleaning up ets / pdict.

 

2. Rolling back an open Postgres connection: This needs to run but doesn't especially care about the process it's run in (as long as it can get at the Postgres connection) Also it needs to know if the setup hook actually ran.

3. Killing spawned processes: This doesn't care where it runs (since it would just do :gen_server.call(:some_name, :die) or Process.exit/2

 

So it must be something like on_exit/1 IMO.

 

And how about an around hook? (with yield ;) )

Unfortunately it won't solve the issue here which is to guarantee something will run even when there are linked failures (which is not guaranteed on something like an after clause in a try).

You are right that most test frameworks have teardown but that could be exactly the argument for not having one in Elixir. Because that would be what everyone would use and it is not guaranteed to do what they want.