Make Pyramid 1.7 scaffolds ready for naïve usage of py.test and tox in new projects?

Skip to first unread message

Vincent Férotin

May 5, 2016, 12:44:19 PM5/5/16
to pylons-devel
Hi, Pyramid developers!

Here are some thoughts about what I consider a small problem for new users:
while usage of py.test is now promoted in documentation
scaffolds are not ready for naive usage of py.test by new comers to Pyramid,
i.e. running ``py.test [--cov]`` at new project's root collects no tests.

As I don't know if this could be considered a bug, I'd prefer opening discussion here, before opening a bug in the tracker or submitting a pull request.

What I, as new Pyramid user, attempt just after creating project via ``pcreate -s [scaffold] [myproject]``,
and seeing the "Welcome to Pyramid.  Sorry for the convenience." message, is that
project is ready to be tested out of the box by py.test or tox at project's root.
Currently, this seems to not be the case: py.test collects no item (outputs "collected 0 items").

To change that, I tried to modify `alchemy` scaffold so that:
- py.test runs out-of-the-box,
- little effort is required for tox to also work.

To do that, I:
- add a `tox.ini` configuration file for py.test;
- add a `.coveragerc` configuration file for pytest-cov plugin;
- move all tests material in its own directory (``{{pacakge}}/tests/``);
- split test file in `tests/` and `tests/`.
Full change could be found here:

Discuss-able choices I'm aware of are:
* usage of tox.ini in place of setup.cfg:
  as setup.cfg generation was removed from 1.7, and usage of tox could be possible,
  I've preferred putting py.test configuration in tox.ini.
  This allows small change (i.e. adding ``[tox] envlist=py35``) for also having tox running on the new project;
* moving tests on their own sub-directory:
  This avoids py.test scanning all project's files in order to collect tests, which could be perceived as slow,
  by explicitly telling it where tests stand.
  Also, this allow splitting tests into material for testing (`tests/`) to tests them selves (`tests/test_*.py`),
  in same manner than models is splitted between `models/` and `models/`.

So, what do you guys think about that?

-- Vincent

Michael Merickel

May 5, 2016, 1:01:39 PM5/5/16
to pylons-devel
Hey Vincent, thanks for working on this.

It would be very helpful if you could open an issue or a PR on github as it's much easier to discuss things there. I will say up front that a couple of these decisions are made already.

1) no tox. This is an app, not a library and we've already had a couple discussions about this where it was decided we would not promote tox in the scaffolds.
2) no explicit location where tests must live, I'd opt for scanning the project for particular conventions such as files starting with ``test_``.

As far as adding a setup.cfg or a pytest.ini, I'm all for these things if they make running the tests simpler but avoid certain opinions like package layout. I'm not sure what coverage settings you'd need so I'd have to see the PR.

- Michael

You received this message because you are subscribed to the Google Groups "pylons-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To post to this group, send email to
Visit this group at
For more options, visit

Steve Piercy

May 5, 2016, 1:46:06 PM5/5/16
On 5/5/16 at 9:44 AM, (Vincent
Férotin) pronounced:

>scaffolds are not ready for naive usage of py.test by new comers to Pyramid,
>i.e. running ``py.test [--cov]`` at new project's root collects no tests.

You are correct in that one cannot just naïvely run:

$ py.test [--cov]

py.test discovers tests in the following manner:

By following the instructions for the SQLAlchemy + URL dispatch
wiki tutorial to get started:

I can run the tests:

stevepi-mbp:tutorial stevepiercy$ $VENV/bin/py.test
tutorial/ -q
2 passed in 0.54 seconds

stevepi-mbp:tutorial stevepiercy$ $VENV/bin/py.test
--cov=tutorial --cov-report=term-missing tutorial/
test session starts ========================================================================
platform darwin -- Python 3.5.0, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /Users/stevepiercy/pyramidtut/tutorial, inifile:
plugins: cov-2.2.1
collected 2 items

tutorial/ ..
coverage: platform darwin, python 3.5.0-final-0 ----------------------------------------------------------
Name Stmts Miss Cover Missing
tutorial/ 8 6 25% 7-12
tutorial/models/ 22 0 100%
tutorial/models/ 5 0 100%
tutorial/models/ 8 0 100%
tutorial/ 3 3 0% 1-3
tutorial/scripts/ 0 0 100%
tutorial/scripts/ 26 26 0% 1-45
tutorial/ 39 0 100%
tutorial/views/ 0 0 100%
tutorial/views/ 12 0 100%
tutorial/views/ 4 4 0% 1-7
TOTAL 127 39 69%

2 passed in 0.57 seconds =====================================================================
stevepi-mbp:tutorial stevepiercy$


Steve Piercy, Soquel, CA

Vincent Férotin

May 6, 2016, 8:51:56 AM5/6/16
Thank you, Michael and Steve, for the quick responses.

I will open a PR on github with changes to take in consideration what you said.

Sure you can explicitly tell py.test to run a specific test module,
but the whole point of my suggestion is to not pass test module path
to py.test. It's the point of view of a beginner, not familiar with
Pyramid scaffolds nor py.test: isn't it easier/more convenient to only
run ``py.test`` at project's root directory, just after project
creation by scaffold, and get all tests collected by default?

-- Vincent

Steve Piercy

May 6, 2016, 11:52:10 AM5/6/16
In this case, py.test was totally new to me. I had always used
``nose`` or ``tox`` that someone else wrote or'ed for
me. I was a beginner to py.test and test discoverability.

When I tried running tests out of the box following that pattern
via ``py.test``, it didn't work. Hmm, I thought, there *must*
be some way to make it work. So I Google Fu'd, visited the
link[1] to read about test discovery, and it became clear in my
mind how to do so. I then wrote the documentation for how to
run tests for the scaffold. As a beginner, I managed to work it out.

Because I learned something from this process, I think it is
good for beginners to realize that there is more to py.test than
its mere invocation without options. There's a lot to explore.
After having this discussion, I think I should add this
reference to the tutorial installation step.

For what it's worth, when we go deeper into tests, we use the
simple invocation with a single option to suppress output,
``py.test -q``.[2]



On 5/6/16 at 2:51 PM, (Vincent
Férotin) pronounced:

Vincent Férotin

May 6, 2016, 12:41:43 PM5/6/16
I totally agree, Steve, that there is much more to explore with
py.test than simple invocation we describe here. And I better
understand how you come to the pattern ``py.test mymodule/
-q``. My experience took an other path, where I was sure that using
``py.test`` alone should work, googled about it, and found
configuration i purposed here ;-)

Anyway, I rework previous patch to meet Michael preferences, and it is
now simpler: just adding tow new file to each scaffold (one for
`pytest`, the other for `coverage`). I also update narrative doc. to
first use simple ``py.test -q`` call (letting a reference to the more
explicit ``py.test mymodule/ -q`` pattern).

The bad news is that I simply failed to create a PR, in that pushing
new commits to 1.7-branch of my forked repo. aborts (with a
[french/here translated] message like:
" ! [rejected] 1.7-branch -> 1.7-branch (non-fast-forward)
error: unable to push references to
astuce: Update failed due to branch head is behind
astuce: its distant homologue.
astuce: Pull distant changes (e.g. 'git pull ...')
astuce: before pushing. [...]").

I'm not familiar enough with git to resolve this issue.
So I push patches here:
Patches could be imported by StackedGit (`stg`) through ``stg import
--series pyramid-patches/series``).

I'm sorry about that, and hope this will not abort insert of these
changes in Pyramid's repo.

-- Vincent

Steve Piercy

May 6, 2016, 1:21:13 PM5/6/16
We try to follow a good git workflow.[1]

The error provides instructions of what to do.

>astuce: Pull distant changes (e.g. 'git pull ...')
>astuce: before pushing. [...]").

Which implies you did not follow the helpful instructions in under "Building documentation for a Pylons
Project project", steps 6-10[1] which I wrote to prevent such
occurrences. I haz a sad. :-(



On 5/6/16 at 6:41 PM, (Vincent
Férotin) pronounced:

Vincent Férotin

May 6, 2016, 2:08:23 PM5/6/16
Ok, here's an attempt for the PR:

Thank you for your patience, and thank you Steve for pointing me the
right document! I wasn't aware of its existence, as it seems not
integrated to HTML documentation (?). (I've previously found the which is not
so helpful.)
As an aside, following steps GitHub Flow, I create new branch directly
from master, but was confused by the paragraph on other branches
should I have taken this in consideration?

Again, thank you Steve!

-- Vincent

Steve Piercy

May 6, 2016, 3:15:13 PM5/6/16
It's hard to miss When you create a new issue
or PR, a friendly yellow bar appears with the message:

Please review the [guidelines for
contributing]( to
this repository. is limping along. It's on the list of
to-dos, behind the [new marketing site for
Pyramid]( and a new Sphinx theme for documentation.

The branch to which you submit a PR should always be taken into
consideration. 'master' is usually default. We usually
backport changes from master to a specific branch.


On 5/6/16 at 8:08 PM, (Vincent
Férotin) pronounced:

Vincent Férotin

May 8, 2016, 9:38:02 AM5/8/16
Much clear. Thanks!
Reply all
Reply to author
0 new messages