Probability controls

[Yoshifumi YAMAGUCHI]

Hi Hypothesis community,

I've been playing with Hypothesis these days and trying to port some long run test written in PropEr. I wanted to confirm if still `one_of` is the recommended way to choose from multiple strategies and Hypothesis is not looking at providing options to let users control the probabilities of multiple choices, which PropEr's `frequency/1` offers.

I looked up Stack Overflow and confirmed the answer from Zac, but I wanted to double check if this direction is the case because the answers are a couple of years ago.

https://stackoverflow.com/a/55329132/515508 https://stackoverflow.com/q/49827010/515508

Best,

Yoshi

[Zac Hatfield Dodds]

Hi Yoshi

This is a very stable core API, so no changes - `st.one_of()` is still the recommended way to take the union of strategies, and there are (still) no options to control the probabilities.  There are a couple of reasons for that second point:

1. Probability options make the API larger and harder to learn, which is a worse user experience.  
2. They tend to focus users on constructing particular kinds of data, rather than thinking about a whole domain of inputs.
3. Bugs tend to hide precisely in the places you didn't think about testing, so we think that Hypothesis (perhaps augmented with HypoFuzz and Crosshair) can actually outperform almost all users at finding error-inducing inputs.

Of course it's true that probability-control tools are sometimes useful for fuzzing experts - but on the whole I think we've made a very good trade, and in those cases I'd suggest using the same effort to instead write an additional more tightly targeted test :-)

Hope that helps,

Zac

--
You received this message because you are subscribed to the Google Groups "Hypothesis users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypothesis-use...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/hypothesis-users/c67575ea-16c7-41f0-9e88-e16c1d589f8cn%40googlegroups.com.

[Yoshifumi YAMAGUCHI]

Thank you, Zac for the detailed answer. I believe the philosophy described in this answer is good to know when we use Hypothesis, and it would be great if we can put this on `st.one_of`'s document. Should I make a pull request for it?