test order

[Mitchell L Model]

I am using nose again for the first time in a few years. I searched for "order" in the archives, and even located a post of my own  — from 2/16/10! I read a number of other messages located by the search

• I understand that tests defined in a test class within a source file are executed in alphabetical order.
• The posts I read showed that, like me, other people were interested in having them executed in the order in which they appear in the source code.
• I understand that tests are supposed to be independent of each other and therefore the order of execution.
• However, I think this limitation misses an important use case, a situation in which I find myself, as follows.

I am developing tests for an unfamiliar API. I wrote some tests that started with the simplest possible use of the API and moving forward into progressively more sophisticated uses. I want my simpler tests run before the more complicated ones because if the way I am using the API in the simpler tests is wrong

1. There is no point running the later tests.
2. The result of the earlier tests get buried in the noise of the massive failure outputs of the other tests.
3. Executing the more complicated tests first in and IDE means the program doesn't reach breakpoints set on the simpler tests if the more complicated tests raise errors.
4. The API may change in a future release and I want to discover that the first place in my test code that runs across a difference that causes a problem. (The API is for rapidly evolving software.)

I know I can handle my situation to some extent with one of the following approaches

• Force alphabetical ordering by including something like "_00_", "_12_", etc. in all my test names. This is real bother and sometimes requires renumbering.
• Use @skip, @expectedFailure, etc. on all the failing tests and remove them progressively as I assure myself that I understand progressively more sophisticated uses of the API. However I don't know of a way (I may just be missing something) to continue testing after an error is raised by a test run from within an IDE using nose.run().
• I can break up my tests into separate classes or files to group together tests of parts of the API that are related by either functionality or simplicity, and I can explicitly execute just one of those test classes or files. This doesn't really solve the problem, but it reduces its affects somewhat.

I did see a note — from 2/7/07 (!!!) to the effect that a plugin would be simple to write to enforce source-ordering. Has anyone done this? I don't see anything that comes with nose that would help.

I also understand that nose has been replaced by nose2. I have not used nose2 because I cannot see any advantages for me at all, and I keep rereading its documentation. I am particularly fond of nose.tools, which disappears, as I understand it, in nose2. (It seems to me that nose2 benefits the developers mostly, and doesn't offer much for users. I would be delighted to hear why I am wrong about this.)

So, is there any reasonable way to execute my tests in source order?