[Proposal] Pretty print diff when matching on context fails in ExUnit

[Artur Plysyuk]

Consider the following code:

setup do

%{a: 1}

end

test "test", %{b: b} do

IO.inspect(b)

end

if we run the test, then it fails with the following error:

  1) test test (SandboxTest)
     test/sandbox_test.exs:8
     ** (FunctionClauseError) no function clause matching in SandboxTest."test test"/1

     The following arguments were given to SandboxTest."test test"/1:
     
         # 1
         %{a: 1, async: false, case: SandboxTest, describe: nil, describe_line: nil, file: "/home/artur/projects/sandbox/test/sandbox_test.exs", line: 8, module: SandboxTest, registered: %{}, test: :"test test", test_type: :test}
     
     code: test "test", %{b: b} do
     stacktrace:
       test/sandbox_test.exs:8: (test)

A way better error reporting is present if the test uses assert macro:

setup do

%{a: 1}

end

test "test", context do

assert %{b: b} = context

IO.inspect(b)

end

Error:

  1) test test (SandboxTest)
     test/sandbox_test.exs:9
     match (=) failed
     code:  assert %{b: b} = context
     left:  %{b: b}
     right: %{
              a: 1,
              async: false,
              case: SandboxTest,
              describe: nil,
              describe_line: nil,
              file: "/home/artur/projects/sandbox/test/sandbox_test.exs",
              line: 9,
              module: SandboxTest,
              registered: %{},
              test: :"test test",
              test_type: :test
            }
     stacktrace:
       test/sandbox_test.exs:10: (test)

It would be great if something similar is reported for the first code snippet.

[Ben Wilson]

Agreed. Can the formatter get invoked on output like that I wonder?

[José Valim]

If we are going down this path, then my suggestion would be to add this to all of Elixir itself, starting with function clause errors. :) If someone wants to explore this path, please go ahead!

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/513c8678-e03c-40da-9163-9cfd7e9c943bn%40googlegroups.com.

[Brandon Gillespie]

I may not understand the added suggestion to do it for all of Elixir, but assuming I do, then I feel doing it would be nice, but presents a pretty significant security risk.

If any runtime function failure would log the runtime data... please no! That would make hitting any level of regulatory compliance a nightmare, as one would have to have crazy involved log scrubbing.

Perhaps only show it off log level is debug? And if not, it instead prints the types only.

It definitely would need to be a configurable thing.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Ben Wilson]

That would be lovely as well. My main question there is if there are performance concerns invoking the formatter in such cases? I can't imagine it'd be an issue for test failures, but general pattern match exceptions seem like they might be more sensitive to such things.

And possibly a relevant question in both cases: How well does this work with inspect limits that end up eliding parts of the data?

[José Valim]

We already have an opt-in Exception.blame mechanism that is only used in certain scenarios (like Plug Debugger, ExUnit, and so on) and this would be done as part of Exception.blame, so it won't run all the time, which should alleviate both performance and security concerns.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/5f8ef0b6-0b6f-41d7-90df-73a24c3226f2n%40googlegroups.com.