Organizing utilities for Django's test suite

[Tim Graham]

At present, we have a number of functions in django.test.utils which aren't documented and are meant for use in Django's test suite. Things like:

* extend_sys_path()

* isolate_lru_cache() 

* captured_stdout()/err()/in()

* freeze_time()

* require_jinja2()

I have the need to reuse a WidgetTestCase from forms_tests in postgres_tests so I tentatively proposed putting it in django/test/testcases.py with a comment that it's not a public API [0]. Simon proposed creating a django/tests/tests/utils.py submodule (alongside runtests.py) instead of defining it in django.test since it's not meant to be a public API.

Do you have any feelings about how to best organize things? I agree that mixing public and private things in django.test is not ideal. Another idea I had was to create something like django/test/_utils.py (underscore prefix) for private stuff. That would allow user code to continue using private APIs at their own risk (whereas putting test helpers in tests/ would not). 

[0] https://github.com/django/django/pull/8425

[Markus Holtermann]

Hey Tim,

I think we can make a case for including this in django/tests/testcases.py and in a new module tests/utils/something.py which is then only available within Django's own the test suite. I think we should include that test case as part of Django's own test suite for now. It's IMO easier to move code from tests/utils/something.py to django/tests/something.py than vice versa.

That said, I think we should make capture_stdout()/err()/in() public API. It's super helpful.

/Markus

[Tim Graham]

About the module name for Django's test suite, "tests" doesn't feel intuitive to me. I think it should be more descriptive, like "django_test_utils", though I'm hardly advocating for that. "test_utils" is already taken as a module for testing django.test.utils. Anyone else have suggestions of conventions used for this sort of thing in other projects?

About capture_stdout(), etc. these are copied from cpython's test.support. I guess it could be nice to see if Python could make them public so Django doesn't have to.