Coverage's xml command fails upon missing sources

[Arve Knudsen]

Hi Ned

I just discovered that unlike the standard reporter, Coverage's XML reporter fails when it detects a missing source file. If you'd pull from my branch "arve.knudsen" at https://bitbucket.org/arve.knudsen/coverage.py, I have there changed it to behave like the standard reporter (i.e., warn on the commandline). I also added a couple of tests (to test.test_coverage).

Arve

[Ned Batchelder]

Huh, that's odd. I'd pull from your branch, but it seems to be private.

--Ned.

[Arve Knudsen]

Sorry, didn't realize repositories could even be private on github. I tried to make it public, but an internal error (500) occurred :( I'll let you know when it's fixed.

Arve

[Arve Knudsen]

Actually, I think my modification might have gone through. At least in the settings, the repository is no longer marked as private. Please try again.

I see I called the site "github" in my previous mail btw, oops :) Apparently getting confused by this proliferance of project hosting services.

Arve

[Ned Batchelder]

Thanks, now I can see it.

But I'm wondering, why isn't the -i option good enough already?  Or alternately, why shouldn't the other reporters also change from ignoring missing files to warning about them?

--Ned.

[Arve Knudsen]

It's a question of semantics obviously. The standard reporter, which I guess should be considered the reference in this case, interprets "ignore" mode as to not warn of missing files. The XML reporter on the other hand, raises a more or less uncaught exception from what I can recall. So, it isn't correct that the standard reporter at least ignores missing files, it actually does warn about them. This behaviour makes sense to me at least. I can't see it as a hard error that certain files aren't found (in my case, they're temporary files of uncertain origin).

Arve

[Ned Batchelder]

Arve,

Maybe I don't understand what you are saying.  The handling of missing files isn't different for each type of report, it's handled centrally in report.py.  Your branch doesn't patch xmlreport.py, it patches report.py.  The behavior today is that missing files raise an exception which stops the reporting.  Using the -i option makes all report types completely ignore the missing files.  Your change would make it so that the default is to warn on missing files, and leave -i the same, ignore them completely.

You may be right that we should change the code from "stop or ignore" to "warn or ignore".  But this isn't just about the XML report, it applies to all reports.

BTW: recently I've found that when people ask about ignoring certain files during reporting, it's actually better to ignore them during measurement.  Is there a way your temp files could be left out of the measurement altogether, perhaps by modifying coverage.py?

--Ned.

[Arve Knudsen]

Forgot to reply to all..

---------- Forwarded message ----------
From: Arve Knudsen <arve.k...@gmail.com>
Date: Sat, Sep 18, 2010 at 8:12 PM
Subject: Re: Coverage's xml command fails upon missing sources
To: Ned Batchelder <n...@nedbatchelder.com>


But the handling *is* different, what is common is the raising of NoSource, reacting to it is different. As an example, if I run 'coverage report' in a project of mine, it says within the report: 

/private/var/folders/0z/0z0neyduEeu0BDrqWi37zE+++TI/-Tmp-/tmp9xwDiA   NoSource: No source for code: '/Users/arve/Projects/SrlLib/Tests/tmp9xwDiA.py'

However, if I run 'coverage xml', I get the message 'No source for code: '/Users/arve/Projects/SrlLib/Tests/tmp9xwDiA.py'' and the report (coverage.xml) is empty.

When it comes to my fix, I would say I did in fact introduce it in the right place (report.py as opposed to xmlreport.py). The reason being that the function report_files iterates over each code unit; if the exception isn't caught there, it will abort the report! The SummaryReporter isn't affected by this issue in report_files, since it doesn't use it. If there's going to be a standard way to report missing source files, there should also be a common implementation (for now, in report.py?).

I want to leave out the temp files yes, I don't even know how they get in there. But... that's a different issue, probably due to something my testsuite is doing, let's stick to the one at hand :)

Arve

[Arve Knudsen]

Ned, to prove the difference between the summary and XML reporters I've created in my repository a branch 'test-reporters' (off 'default'), which just adds a new test module 'test_reporters.py'. The latter tests whether the summary and XML reporters raise NoSource when missing source is detected (and not in ignore mode). You should be able to see for yourself that the behaviour differs. I decided in this branch to make raising of NoSource the correct behaviour, since it's currently the common implementation (which the summary reporter ignores).

Furthermore, I think these tests should be useful as a foundation for testing all implementations of the reporter interface. Since there is a common interface, there should be a standard set of tests.

Arve