[Proposal] Access ExUnit context within doctests

[Rosa Richter]

I'm trying to write some good examples for documentation, and I'd like them to be run as tests. However, there's some setup required that would be too verbose and not relevant for a doctest. What are the group's thoughts on allowing the ExUnit setup context to be available in doctests?

I'm not trying to contain all of my tests in doctests; I understand that certain kinds of tests should remain in the test file. I'm only trying to write concise examples that need a little setup. One example: I need a user object as an argument to a business logic module, but creating a user object is not a relevant part of the example.

[José Valim]

Hi Rosa, thanks for the proposal!

My main concern with this approach is that the doctests are no longer "plug and play". I can no longer copy and paste them into IEx and have them work. Passing the context can be even more confusing, because you may end-up referencing things like `context.test`, which are hard to replicate on IEx.

What I have done in these cases is to have an explicit module that I call with a note in the docs:

Assuming your application has a MyExampleApp.client/0 that returns a MyLibrary.Client,

you can do:

iex> client = MyExampleApp.client()

iex> MyLibrary.call(client, :foo)

or:

Assuming your application has a MyExampleApp.setup_some_args/0 that adds some

fixtures to the database:

iex> client = MyExampleApp.setup_some_orgs()

iex> MyLibrary.run_query(some_qiery)

Then in my tests I define the MyExampleApp satisfying the conditions of the tests.

On Mon, Oct 26, 2020 at 4:28 PM Rosa Richter <ro...@prokeep.com> wrote:

I'm trying to write some good examples for documentation, and I'd like them to be run as tests. However, there's some setup required that would be too verbose and not relevant for a doctest. What are the group's thoughts on allowing the ExUnit setup context to be available in doctests?

I'm not trying to contain all of my tests in doctests; I understand that certain kinds of tests should remain in the test file. I'm only trying to write concise examples that need a little setup. One example: I need a user object as an argument to a business logic module, but creating a user object is not a relevant part of the example.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/ed0b6ff5-79aa-4d7e-a951-b802e36976c4n%40googlegroups.com.

[Rosa Richter]

Thanks so much for the reply!

That's a great suggestion, I'm definitely going to do that for now.

I can appreciate the concern with keeping doctests plug-and-play, one of their features is replicating what it's like to use the code in the terminal. But if the doctest uses a special test-only module like that, can they still be used in IEx? It seems easier for an IEx user to create the context map as they need, and then paste in the doctest.

This feature could definitely be abused to make some ugly, unrepeatable doctests, but my hope is that someone writing doctests would be keeping in mind how clear their documentation examples are, and how complicated they've made the context.

[José Valim]

> It seems easier for an IEx user to create the context map as they need, and then paste in the doctest.

My hope is that calling out to a module is a bit more explicit than calling a magic "user" or "context" variable. :)

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/53fb52a5-5655-4c8f-b8c6-84aaeac770e5n%40googlegroups.com.

[Rosa Richter]

That's valid. Thanks for your time José!