[Proposal] Shorthand to filter tests by multiple line numbers

[Devon Estes]

Currently, if I want to run only a single test by line number, I can easily do:

    mix test test/potion_web/plugs/persist_request_plug_test.exs:92

This is parsed in ExUnit as:

    Including tags: [line: "92"]

    Excluding tags: [:test]

But if I want to only run two tests in a file, I need to do:

    mix test --include line:74 --include line:92 --exclude test test/potion_web/plugs/persist_request_plug_test.exs

I'm proposing a shorthand for this, along the lines of what we already have for filtering by line number:

    mix test test/potion_web/plugs/persist_request_plug_test.exs:[74, 92]

Which, when parsed, gives us

    Including tags: [line: "74", line: "92"]

   Excluding tags: [:test]

I'd be happy to implement this if y'all think it's a good idea

[José Valim]

*Stream of thoughts below.*

Can you use only instead?

mix test --only line:74 --only line:92 test/potion_web/plugs/persist_request_plug_test.exs

My only concern with your feature proposal is that it may lead to wanting to run multiple tests across multiple files and that's a trickier problem. One option would be change our line filter to be something like file+line. In a way that:

mix test test/potion_web/plugs/persist_request_plug_test.exs:92

becomes:

Including tags: [location: {"test/potion_web/plugs/persist_request_plug_test.exs", 92}]

but that's quite more verbose. Thoughts?

However, if we decide we don't need to support multiple files, then I would do:

mix test test/potion_web/plugs/persist_request_plug_test.exs:92:94:102

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/e04f0e77-3c37-4f0f-b4a0-664e8ce74cea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sven Gehring]

I'd personally prefer the syntax proposed by José.

Also, I would argue that allowing for multiple files introduces quite a bit of unnecessary complexity.

I never had the need to run only specific tests in multiple files but I think if it is really necessary for

someone, you can always still chain the commands (ie. mix test ... && mix test ...) which is not ideal

because you do the setup/teardown twice but I think it would work to cover that edge case

Sven

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JvEOHo565x74FbfCdW6piNwxqwBjToy8kKAN_SQg0ZTQ%40mail.gmail.com.

[Amos King]

The syntax that José proposed is common to other command line tools so I vote for it.

I have wanted to do more than one file. As a user I’m not at all concerned with the verbosity of the parsed version. If it isn’t a big impact on performance. Some setups can be large, and starting of applications. I’d rather save the time there and not have to chain command line calls. Chaining command line calls also has the added issue of one command has a failure and the rest don’t run.

Amos King

Binary Noggin

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAFuVuxyybff9FqXgy3vgmHxayMNnPCboN99hea_0RQeTnahSFg%40mail.gmail.com.

[Devon Estes]

`mix test --only line:74 --only line:92 test/potion_web/plugs/persist_request_plug_test.exs` does work, but don't you think it's a little confusing to allow `--only` more than once?

I'm cool with the `mix test test/potion_web/plugs/persist_request_plug_test.exs:92:94:102` syntax. Whatever syntax is chosen doesn't matter all that much to me, acutally.

I've personally never wanted to run tests in more than one file, but I can imagine that someone might (i.e. running some unit tests & integration tests together).

[Martin Svalin]

Hm. I'll chip in with an opinion based on surprises I got experimenting a bit with the mix test task.

If I wanted to run tests across multiple files identified by line numbers, I would naively write `mix test test/some_test.exs:15 test/some_test.exs:20`.

`mix test test/some_test.exs` will run all tests of a single file.  👍

`mix test test/some_test.exs test/another_test.exs` will run all tests of both files. 👍

`mix test test/some_test.exs:15` will run the test located at line 15 for that file. 👍

`mix test test/some_test.exs test/another_test.exs:20` will run all tests in some_test.exs, but not the test at line 20 in another_test.exs. 🤔

`mix test test/some_test.exs:15 test/another_test.exs` will run all tests in another_test.exs, but not the test at line 15 in some_test.exs 🤔

In both cases, my expectation was to run all tests of one file, and the test at the specific line for the other file.

`mix test test/some_test.exs:15 test/another_test:20` will exit with an error:

"Paths given to `mix test` did not match any directory/file: test/some_test.exs:15, test/another_test.exs:20"  😔

My expectation was that two tests would have been run, the one at line 15 in some_test.exs, and at line 20 in another_test.exs.

Do people agree that this naive expectation is an intuitive one? (I didn't really know any of the internals of the test runner.) Or would other people have other intuitions?

For running multiple tests in a single file identified by line numbers, I'd support `mix test test/some_test.exs:15:20:35`.

I'd also expect to be able to run `mix test test/some_test.exs:15:20:35 test/another_test.exs:40:75:120`.

The question of shorthand for multiple lines within a single file is of course separate from that of running individual tests across multiple files. Simply commenting as the multi-file issue was brought up.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/d5612cd5-4cfc-4952-baad-b75ce3c13934%40googlegroups.com.