best way to modify `max_examples` for a test method

[Evgeni Burovski]

In array-api-tests, we have a custom pytest mark, `unvectorized`, which makes a decorated test use 1/10th of `--max-examples` [1].

We're not using `@ settings(max_examples=...)` directly because we don't set the number itself, we want to decrease the user-provided number by a factor of ten.

Our implementation works for standalone test functions, but breaks down for test methods grouped into a test class

class TestSomething:

      @pytest.mark.unvectorized

      @given(....)

      def test(self, ....):

           .....

The failure is due to us having to apply `hypothesis.settings` to a pytest "item", so it's clearly on us, and this email is me asking for advice.

AFAICS, the key problem I'm having is that I've to apply `settings` to a function---and modifying a test function is a bit awkward in pytest.

Now, a Copilot-drafted fix [2] uses `inspect` and pytest's `item.cls` to reattach the result of applying `setting(max_examples=...)(test_function)`. Previous copilot iterations included such beauties as `setattr(item.obj.__self.__class__, item.obj.__name__, settings(max_examples=...)(test_func)`. 

This all looks quite backwards to me, so I wonder if there's a better / recommended way to modify a parameter of hypothesis settings for a test, and wold appreciate any pointers.

Either way, thanks for the hypothesis!

Evgeni

[1] https://github.com/data-apis/array-api-tests/blob/master/conftest.py#L253

[2] https://github.com/data-apis/array-api-tests/pull/416/changes#diff-a31c7ed5d35f5ed8233994868c54d625b18e6bacb6794344c4531e62bd9dde59L256

[Liam DeVoe]

Yes! You want `item.obj._hypothesis_internal_use_settings`, where `item` is a pytest node. This is obviously internal-please-only-use-if-you-know-what-you're-doing, but I've used this in a number of research-style projects etc, in HypoFuzz, and it's used in Hypothesis' pytest plugin.

This is both read and write, so updates to it will propagate to when pytest runs the node as expected.

--
You received this message because you are subscribed to the Google Groups "Hypothesis users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypothesis-use...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/hypothesis-users/e7e6c6ba-6aa0-4911-8fb8-d9fc3ecae394n%40googlegroups.com.

[Liam DeVoe]

@zac, perhaps we should (optionally stabilize the name and) document this in our internal docs section and add a how-to?

[Evgeni Burovski]

On Sat, Feb 7, 2026 at 6:28 PM Liam DeVoe <orion...@gmail.com> wrote:

Yes! You want `item.obj._hypothesis_internal_use_settings`, where `item` is a pytest node. This is obviously internal-please-only-use-if-you-know-what-you're-doing, but I've used this in a number of research-style projects etc, in HypoFuzz, and it's used in Hypothesis' pytest plugin.

This is both read and write, so updates to it will propagate to when pytest runs the node as expected.

Ouch, thank you Liam!

I saw this setting in pdb but the name serves its purpose of conveying "don't even look in this direction" very well indeed. Having a how-to would be really great! If you do add a how-to, the `fn = getattr(item.obj, "__func__", item.obj)` trick from your second link would be great to include, too. It's not specific to hypothesis of course, but it's very much not obvious unless you've poked around it these parts recently, and failure modes of not doing it when needed are... obscure, to put it mildly. 

Thanks again,

Evgeni