Hi all!
I just managed to get coverage reports for
ceylon.ast tests working, and I thought that this would be useful to others as well, as coverage reports are pretty useful information when writing or improving tests.
(If you don’t know what I’m talking about, the short version is that coverage reports tell you what parts of your program are actually ran when you run your tests.
Bad coverage usually means that you should write more tests to ensure that you’ll catch bugs early.
This is what coverage can look like when integrated into the IDE;
the screenshot indicates that perhaps you should consider testing the
addAll() method with an empty collection argument.)
If you’re using Ant
, you can mostly copy the bottom part of my build.xml. You just need to change the names on lines 316-318 and 353.
See
this commit for usage.
If you want to do it manually, here’s what you have to do:
- Download and extract JaCoCo from here. (You only need the jacocoant.jar and jacocoagent.jar files.)
- Run your tests with JaCoCo attached. If you use ceylon run or ceylon test, simply set the environment variable JAVA_OPTS to -javaagent:[yourpath/]jacocoagent.jar. For example, on Linux:
JAVA_OPTS="-javaagent:lib/jacocoagent.jar" ceylon test test.my.module
If you call java directly, add -javaagent:lib/jacocoagent.jar to the command line.
You can also customize JaCoCo; see here for more information. - Now you have a jacoco.exec file in your project directory, which contains the raw coverage data.
To generate a report, JaCoCo will need the class files, and it can’t get them from the CAR file, so we need to extract them.
For example, on Linux:
find modules -name '*.car' -exec unzip {} -x '*META-INF*' -d classes/ \;
- Now we can generate the report.
Unfortunately, that’s a bit of a pain from the command line; apparently it’s simply not possible without either Ant or Maven.
You can use this sample build.xml file; adapt the file and directory paths in it as necessary, then run ant. - You’re done, look at the file coverage-report/index.html in your browser! If you did everything correctly (and I didn’t mess up the instructions ;-) ), then you should get detailed coverage information, and if you click all the way through to a function and click on that, you even get to see the source code, with green/yellow/red background indicating full/partial/no coverage.
Keep in mind that Ceylon instructions don’t directly correspond to Java (or JVM) instructions. For certain code, the Ceylon compiler even adds code that will never be executed. For example:
- A switch/case statement without an else case is usually compiled to an if/else if chain, with a final else clause that looks like this:
else throw new .com.redhat.ceylon.compiler.java.language.EnumeratedTypeError("Supposedly exhaustive switch was not exhaustive");
- The initializer of a toplevel object contains code to detect and handle cyclic initialization; unless your objects are actually cyclic, it will never run.
- Defaulted function parameters generate an overloaded function (with less parameters) and a function that holds the default value computation; depending on how you call these functions (with or without explicit arguments), some of these default values may never be evaluated.
- The catch block of a try-with-resources can get pretty complicated when multiple resources are used (shuffling around suppressed exceptions), resulting in a disproportionate amount of instructions. (This is true for Java as well.)
So don’t despair because you don’t get 100% coverage!
I hope this helps some people :)
– Lucas