[Proposal] Contextual data in ExUnit.assert/1 or assert/2

[Derek Wiers]

So, we have the assert/1 macro in ExUnit, which does a really nice job of pointing out actual failures - a diff, etc, depending on what you give it. We also have assert/2, a function where you can insert your own message. I propose that we have perhaps another option: a variant of assert(assertion, contextual_data: %{foo: "bar"}). Basically, I love that diffing we do in assert/1, and just want to append some relevant data to the message, which is particularly helpful if I'm running an assertion in a loop (think poor man's property-based testing) with different data -- the line number isn't going to allow me enough data to figure out what exactly just failed.

I understand it wouldn't be trivial to do, pretty much because there's already an assert/2 with opts and with message, and they're functions not macros...

Sorry if I'm missing anything about proposals - this is my first one!

I'm willing to maybe put some time into a PR here, if the feedback is positive and has a bit of guidance (Elixir is my day-job, so I do have some experience)

Thanks,

Derek

[Ben Wilson]

+1 to this from me. We have a variety of asserts inside of a list that always have commented out dbg statements above them to help with failures. Passing those into assert would be great.

[Jason Axelson]

Big +1 from me as well. I often skip using `assert/2` because of the need to re-implement the rich diffing logic that `assert/1` gives. Generally I only truly want to provide additional context that will help the developer understand what failed rather than wanting my own "diffing" output.

-Jason

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/afe45a12-091d-4980-b64d-e5974926e7cdn%40googlegroups.com.

[Jon Rowe]

+1 here, more context is often useful but the diff support is crucial.

Cheers

Jon

To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/CAHMz6kzuE%2BEZK5Bhb6Q0SMKO6T8S1XsAeYkeK0NjhFv4ZjBT_w%40mail.gmail.com.

[José Valim]

If someone would like to tackle this, please go ahead. I assume it will require a reasonable amount of work.

José Valim

https://dashbit.co/

To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/5b0c7550-d2a0-41b8-9a07-e9b063d2dd00%40app.fastmail.com.

[Derek Wiers]

So, an update on this idea:

 v1 of this was rejected (https://github.com/elixir-lang/elixir/pull/15309). I think I understand the reasoning. Makes sense. To José's point, perhaps a different API would be better, and less brittle. What about something that looks more like this?

my_test_list

|> Enum.each(fn test_case ->

  with_assertion_context test_case do

    # do all the assertions & refutes you want

    # and it'll be transformed at compile time to a fancier call that has the assertion context in it for a nicer diff based error message
    assert test_case.foo == "bar"

    refute test_case.bar == 1

    assert test.baz == "quux", "this is assert/2 and is implemented as a function and thus would not get replaced at compile time by with_assertion_context"

  end

end)

The nice things about this:

- the lexical scope of the context is obvious

- it's not terrible to implement

- probably not significant performance impact

- probably usually used at the same "level" as a loop/iteration, and it kinda fits

- if you need multiple contexts, it can easily be done by just having multiple assertion contexts consecutively

- it feels declarative, not imperative, so it feels like a good match for Elixir's style and BEAM's immutability characteristics.

Downside:

- Not obvious what should happen in the case of nested assertion contexts (open question...should it fail? Should one win? Should there be some form of concatenation?)

- a bit noisier than I'd like (though less noisy than my v1 idea of putting it on every assert, so fair enough)

- that's one extra indentation inside of a loop during testing - though I'm not in the habit of testing this way all the time (it's just handy for some cases) so the tradeoff to me is worth it

Thoughts? I'm not totally sold on the `with_assertion_context` name either - that feels like a compiler plumbing sort of name, not a user facing sort of name, but I'm at a loss of what obvious thing to call this that specifically, semantically makes it obvious that we're talking about modifying/adding to error messages in assertions.

You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/QcgZ5HkAQAs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KjA%2Bzd0ZqUMo1KPOr%3DASao30_NT6we7JZoX%3DXPk2itvQ%40mail.gmail.com.