switching from nosetests to pytest

[josef...@gmail.com]

(this is a shortened email from my initial draft)

Because nosetests is not developed anymore, every package in the scientific python area is switching to pytest.

Kevin prepared a PR https://github.com/statsmodels/statsmodels/pull/3804 to migrate to pytest. We would like to merge this ASAP.

At the moment we still have some yield based tests in the recently merged VECM. pytest has deprecated yield based tests but currently still supports them. Those tests will be converted to regular tests in the near future.

Also, we currently don't use pytest specific features, so for now nosetests still works on the command line.

As part of the PR we also switch from coveralls to codecov for the test coverage results.

scipy is also currently transitioning to pytest, and there are some questions still open about the support that we will get from numpy.testing for pytest instead of nosetests.

Josef

[Yaroslav Halchenko]

On Fri, 28 Jul 2017, josef...@gmail.com wrote:

> (this is a shortened email from my initial draft)
> Because nosetests is not developed anymore, every package in the
> scientific python area is switching to pytest.

even though every one is moving, and I do use pytest in new projects, I
want first of all to applaud nose developer(s) for providing community
with the super nice and stable (was used in thousands of projects
without any heavy real maintenance for a while) tool. I am still
learning my ways around pytest, which are at times "too magical", and
often look back at the clarity of the explicit nose way of doing things.

> Kevin prepared a PR https://github.com/statsmodels/statsmodels/pull/3804
> to migrate to pytest. We would like to merge this ASAP.
> At the moment we still have some yield based tests in the recently merged
> VECM. pytest has deprecated yield based tests but currently still supports
> them. Those tests will be converted to regular tests in the near future.

some of the "yield" tests are "parametric tests", e.g. test_all in
statsmodels/nonparametric/tests/test_lowess.py
so they should be converted to pytest's way for those, e.g. with

@pytest.mark.parametrize

> Also, we currently don't use pytest specific features, so for now
> nosetests still works on the command line.
> As part of the PR we also switch from coveralls to codecov for the test
> coverage results.

+1
just to make sure -- you are aware of their wonderful browser(s)
extension to annotate PRs code diff view with coverage info, aren't you?


--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

[josef...@gmail.com]

On Sat, Jul 29, 2017 at 2:32 AM, Yaroslav Halchenko <yarik...@gmail.com> wrote:

On Fri, 28 Jul 2017, josef...@gmail.com wrote:

>    (this is a shortened email from my initial draft)
>    Because nosetests is not developed anymore, every package in the
>    scientific python area is switching to pytest.

even though every one is moving, and I do use pytest in new projects, I
want first of all to applaud nose developer(s) for providing community
with the super nice and stable (was used in thousands of projects
without any heavy real maintenance for a while) tool.  I am still
learning my ways around pytest, which are at times "too magical", and
often look back at the clarity of the explicit nose way of doing things.

I fully agree here. I have used nosetests for more than 9 years without major or any problems. Together with the numpy.testing tools it has made writing and running unit tests in python very easy. .

 

>    Kevin prepared a PR https://github.com/statsmodels/statsmodels/pull/3804
>    to migrate to pytest. We would like to merge this ASAP.
>    At the moment we still have some yield based tests in the recently merged
>    VECM. pytest has deprecated yield based tests but currently still supports
>    them. Those tests will be converted to regular tests in the near future.

some of the "yield" tests are "parametric tests", e.g. test_all in
statsmodels/nonparametric/tests/test_lowess.py
so they should be converted to pytest's way for those, e.g. with

 @pytest.mark.parametrize

Right now I prefer to stay in compatibility mode for a while, until we or another package has figured out what works and what doesn't work well.

For example, while trying out the PR yesterday I ran into segfaults with matplotlib again. I managed to get rid of the segfaults for nosetests but now they are back with pytest.

And, I never liked "black magic" and I don't like automatic code rewriting. I worry that it will add to difficult to debug issues. I haven't tried the pdb with pytest yet, but 

nosetests --pdb --pdb-failures <path to test>

has been my core tool for development, maintenance and bug-hunting for a very long time.

 

>    Also, we currently don't use pytest specific features, so for now
>    nosetests still works on the command line.
>    As part of the PR we also switch from coveralls to codecov for the test
>    coverage results.

+1
just to make sure -- you are aware of their wonderful browser(s)
extension to annotate PRs code diff view with coverage info, aren't you?

I only browsed briefly one of their coverage reports, so I'm not aware much of any goodies.

Josef

[josef...@gmail.com]

I just merged Kevin's PR to switch to pytest.

From now on pytest needs to be installed to run the unit tests in master.

Some PR will have merge conflicts, but those should be limited to a few places in the unit tests.

Also, some features that nosetests supports but not pytest will have to be avoided. Unit tests need to be pytest compatible from now on.

e.g. the non-pythonic  setupClass is illegel, use setup_class instead (which looks much prettier)

There will be a few things to watch out for as we write new unit tests, but I hope we learn fast.

Special thanks goes to Kevin who did all the migration work.

Josef