Test coverage: what are the priorities?

[Yannick Gingras]

Fellow Packagers,
tomorrow at the sprint, a few persons are likely to code some
unittests. What should they focus on? Following a list of modules
sorted by inverse order of code coverage? I presume that some of
these are poorly covered because they are on the way to deprecation
but it would be good to get a confirmation from someone familiar with
the roadmap so we don't spend sprinting effort on testing dead code.

Name Stmts Exec Cover
-------------------------------------------------------------------
distutils2/compiler/msvccompiler 362 60 16%
distutils2/compiler/emxccompiler 104 25 24%
distutils2/command/bdist_wininst 176 55 31%
distutils2/log 50 17 34%
distutils2/compiler/cygwinccompiler 139 64 46%
distutils2/compiler/unixccompiler 169 89 52%
distutils2/command/config 176 95 53%
distutils2/command/bdist 60 36 60%
distutils2/converter/refactor 13 8 61%
distutils2/compiler/ccompiler 415 261 62%
distutils2/command/build_ext 331 222 67%
distutils2/dist 428 287 67%
distutils2/util 396 280 70%
distutils2/core 74 54 72%
distutils2/manifest 166 126 75%
distutils2/command/install_scripts 32 25 78%
distutils2/extension 32 25 78%
distutils2/depgraph 96 77 80%
distutils2/command/cmd 180 146 81%
distutils2/command/upload 116 95 81%
distutils2/command/bdist_dumb 68 56 82%
distutils2/command/build_clib 88 73 82%
distutils2/command/register 164 136 82%
distutils2/command/install 244 204 83%
distutils2/fancy_getopt 214 181 84%
distutils2/command/build_scripts 78 67 85%
distutils2/command/install_lib 98 87 88%
distutils2/command/sdist 197 175 88%
distutils2/config 78 69 88%
distutils2/pypi/simple 185 163 88%
distutils2/command/build_py 233 209 89%
distutils2/command/install_egg_info 39 35 89%
distutils2/command/build 59 54 91%
distutils2/command/check 40 37 92%
distutils2/command/upload_docs 91 84 92%
distutils2/pypi/dist 156 144 92%
distutils2/command/clean 38 36 94%
distutils2/command/install_headers 24 23 95%
distutils2/converter/fixers/fix_setup_options 46 44 95%
distutils2/metadata 391 372 95%
distutils2/command/__init__ 2 2 100%
distutils2/command/install_data 42 42 100%
distutils2/compiler/__init__ 1 1 100%
distutils2/converter/__init__ 1 1 100%
distutils2/converter/fixers/__init__ 0 0 100%
distutils2/converter/fixers/fix_imports 30 30 100%
distutils2/errors 26 26 100%
distutils2/pypi/__init__ 1 1 100%
distutils2/pypi/errors 8 8 100%
distutils2/version 192 192 100%
-------------------------------------------------------------------
TOTAL 6349 4599 72%

Cheers,

--
Yannick Gingras
http://ygingras.net
http://montrealpython.org -- lead organizer

[Éric Araujo]

I’m not an expert on all parts, but I think I have some answers, and I
trust other watchful people to correct me eventually.

[I wonder if some of the tests could be added first to distutils and
then to distutils2. Tarek, does that seem like a good idea or do you
prefer to have things happen in d2 and then backports?]

The pypi subpackage has some renaming and improvements from Alexis
waiting, so I advise to ask him before touching it. I want to review his
changes and push them to Tarek’s clone before the sprint.

Modules that are going to change: core (setup and run_setup are, hm,
interesting pieces of code), config, install_egg_info (gone!). Don’t
waste time on those.

log is going to be removed; Tarek, an update on that? There will be need
for some tests, to make sure output is preserved.

Utility modules that are also provided as building blocks for other
projects have to be fully tested: version, metadata, depgraph.
Increasing coverage for commands is also a good thing, especially
torture-testing options values and combinations.

The compiler subpackage would benefit from testing and documentation,
although I have a suspicion than if you find bugs, we’ll need compiler
wizards to fix them. I recommend targeting the base class in ccompiler
first.

fancy_getopt is an interesting target: One of the Montreal hackers has
taken my suggestion to make it use argparse undercover while preserving
the API for defining commands, and good tests are required to make sure
the internal refactoring doesn’t break anything.

Modules I have said nothing about: converter.refactor, dist, util,
manifest, extension. They won’t go away, adding tests for them is safe.

Cheers

[Yannick Gingras]

On July 28, 2010, Éric Araujo wrote:
> I’m not an expert on all parts, but I think I have some answers, and I

> trust other watchful people to correct me eventually. [...]

Thanks! We'll see what we can do.