Add ex_unit line limit for printing non_failures

[Victor Oliveira Nascimento]

Hi all!

I'd like to ask if a line limit to ex_unit cli_formatter would be welcomed in core. This sounds like a super silly feature but it is one that I think would raise the quality of the tool because:

1. It is easier to know the number of tests run with a quick glance

2. Most shell/console interfaces are vertical and not horizontal

3. Some shells/consoles have issues with very long lines

I have a proof of concept and was about to open a PR as a draft but reading through the contributing guide in README this step here is necessary. In any case, this is not a difficult thing to code :)

The MAIN reason I am doing this is number 3. I use Emacs and work on a project that has, currently, around 3 thousand tests. There is a long history of Emacs not handling this well because, well... let's just say because Emacs...

So, I thought this is pretty simple:

- count consecutive non-failures

- if it reaches line_limit add a "\n"
- any failures will reset the counter too

Here is how it looks for ex_units test suite:

So, is this a welcome addition? Should I go on and open a PR?

In any case, thanks to the core team for the great work on the platform.

Stay safe, stay home!

Regards

[Boris Kuznetsov]

Hi, Victor

I suppose your issue is from trying to run tests using one of Emacs
terminal implementations.

I'm using Emacs as well. I don't think we should integrate workarounds
for the editor into the language. I guess the best way to handle this is
to fix terminal handling in Emacs.

Fow now, Emacs has many implementations for terminal, probably one of
them could work well for your use case.

Other than that, I couldn't find use case for this feature.

[Victor Oliveira Nascimento]

Hi Boris! Thanks for your comments.

I disagree with you that this is simply a workaround for an editor. I think the role of the "dots" in the formatter is to show some visual information and the way things are now it is not optimal visual information as a very long horizontal line adds too little as a UX.

Emacs is just one part of this equation (and has recently added a so-long-mode.el to core distribution but that is not working properly for me). But I see this UX on CI and other places too and I think we can do better.

In any case, I totally understand if this is unwanted in core though I still think it would be just "a better" formatter for all.

Glad there are other Emacsers around!

Regards,

[Josh Adams]

On Tuesday, June 9, 2020 at 7:49:04 AM UTC-5, Victor Oliveira Nascimento wrote:

I think the role of the "dots" in the formatter is to show some visual information and the way things are now it is not optimal visual information as a very long horizontal line adds too little as a UX.

I find this compelling in its own right. Due to variability in output, a long line of dots is not as meaningful visually as a more fixed volumetric output would be.

Having said that, I collect junit output and present this sort of thing a lot more valuably inside CI anyway. Presumably if an emacs plugin wanted to produce information with a nicer user experience, it could parse the junit output similarly.

At any rate, I just wanted to mention that I enjoyed the argument-from-UX

[Jonathan Arnett]

Emacs has visual-line-mode (originally called longlines-mode), which can be enabled in at least eshell and shell (not sure about ansi-term) and will automatically wrap long lines and modify the navigation keybindings to treat those wrapped lines as if they are separate lines, which I think accomplishes your stated goal.

For whatever it's worth, I'm working on making elixir-test-mode, which vastly simplifies running test in Elixir projects, MELPA-ready. MELPA has very exacting standards, as it should. Anyhow, you can hook elixir-test-mode to use visual-lines-mode—or, better yet, hook compilation-mode (its parent) to use visual-lines-mode to get line-wrapping in all compilation-type buffers (as I do 🙂).

[Victor Oliveira Nascimento]

I use exunit.el for tests and it works pretty nice. Also, visual-mode-line do not ease the processing of the long line for emacs.

But again, the main point here is NOT Emacs itself but a nicer CLI formatter for all :)

In any case it is good to know more Emacsers around and contributing with nice projects! Thanks for your work too! I will try it out certainly.

Cheers

[Tonći Galić]

Hi Victor,

As far as I know, it's already possible to generate your own formatting with ExUnit.Formatter.
You basically tell ExUnit which module should receive events and that module can handle the output/format reporting failure etc.

Some docs/examples:
  - https://hexdocs.pm/ex_unit/master/ExUnit.Formatter.html#content
  - example (I'm not affiliated/familiar): https://github.com/victorolinasc/junit-formatter

So I think it's not necessary to add a new feature.

Hope it helps,

Tonći

[Victor Oliveira Nascimento]

Hi Tonći!

Thanks for your comment. I know that this is possible and I've even implemented a formatter that spits out JUnit xml report.

That is an alternative for sure and one I'd probably use until this is built-in. I still think that the current formatter could be better though for the reasons I've mentioned before.

Having a new formatter which only adds this funcionality but is a copy and paste of the rest of the source seems not really good but I'll probably go on with this suggestion if this is not accepted.

Regards,