[API] Getting coverage per *unit* Test Case

[José Carlos de Campos]

Hi,

First of all, many thanks for your amazing work on JaCoCo. :)

I'm trying to build a tool on the top of JaCoCo, and one of the features

requested is coverage per *unit* test case (and not the aggregated coverage).

What I've so far: I've created my own TestListener, and every time an

*atomic* test finishes I execute

byte[] b = RT.getAgent().getExecutionData(true);

and then (as suggested here) I create a dump function that performs

the coverage analysis. something like:

final ByteArrayInputStream in = new ByteArrayInputStream(cov); // cov is the byte array from agent.dump()

final ExecutionDataReader reader = new ExecutionDataReader(in);

reader.setSessionInfoVisitor(new ISessionInfoVisitor() {

  public void visitSessionInfo(final SessionInfo info) {

    ExecFileLoader execFileLoader = new ExecFileLoader();

    execFileLoader.load(new ByteArrayInputStream(cov));

    final CoverageBuilder coverageBuilder = new CoverageBuilder();

    final Analyzer analyzer = new Analyzer(execFileLoader.getExecutionDataStore(), coverageBuilder);

    analyzer.analyzeAll(new File("__classes_directory__"));

    for (final IClassCoverage cc : coverageBuilder.getClasses()) {

      // perfom my coverage analysis

    }

  }

});

reader.setExecutionDataVisitor(new IExecutionDataVisitor() {

  public void visitClassExecution(final ExecutionData data) {

    // empty

  }

});

however, and because I'm calling my dump function thousands of times,

my coverage analysis is taking ages to finish. I've measured the CPU time

and, 45% is org.jacoco.core.internal.flow.MethodProbesAdapter.visitLabel(),

13% is org.jacoco.core.internal.analysis.SourceNodeImpl.incrementLine.

is there any better way of doing this? am I not using the right API functions?

in your opinion, how can I improve the performance of my coverage analysis?

--

Cheers,

Jose

[Marc Hoffmann]

Hi,

if you have thousands of test cases and want per test coverage results
you need thousands analysis calls. Maybe you can optimize a bit and e.g.
narrow the scope of the classes analyzed for a specific test.

Anyway, your code looks a bit strange: Why do you put the analysis into
the visitSessionInfo() method? This forces you to parse the cov byte
Array twice. This should be enough:

ExecFileLoader execFileLoader = new ExecFileLoader();
execFileLoader.load(new ByteArrayInputStream(cov));

final CoverageBuilder coverageBuilder = new CoverageBuilder();
final Analyzer analyzer = new
Analyzer(execFileLoader.getExecutionDataStore(), coverageBuilder);

analyzer.analyzeAll(new File("__classes_directory__"));

for (final IClassCoverage cc : coverageBuilder.getClasses()) {

// perfom my coverage analysis

}

If you need Session infos you can retrieve them from the ExecFileLoader:

execFileLoader.getSessionInfoStore();

Regards,
-marc

On 2015-10-15 13:44, José Carlos de Campos wrote:
> Hi,
>
> First of all, many thanks for your amazing work on JaCoCo. :)
>
> I'm trying to build a tool on the top of JaCoCo, and one of the
> features
> requested is coverage per *unit* test case (and not the aggregated
> coverage).
> What I've so far: I've created my own TestListener, and every time an
>
> *atomic* test finishes I execute
>
> byte[] b = RT.getAgent().getExecutionData(true);
>

> and then (as suggested here [1]) I create a _dump_ function that

> performs
> the coverage analysis. something like:
>
> final ByteArrayInputStream in = new ByteArrayInputStream(cov); // cov
> is the byte array from agent.dump()
> final ExecutionDataReader reader = new ExecutionDataReader(in);
>
> reader.setSessionInfoVisitor(new ISessionInfoVisitor() {
> public void visitSessionInfo(final SessionInfo info) {
>
> ExecFileLoader execFileLoader = new ExecFileLoader();
> execFileLoader.load(new ByteArrayInputStream(cov));
>
> final CoverageBuilder coverageBuilder = new CoverageBuilder();
> final Analyzer analyzer = new
> Analyzer(execFileLoader.getExecutionDataStore(), coverageBuilder);
>
> analyzer.analyzeAll(new File("__classes_directory__"));
>
> for (final IClassCoverage cc : coverageBuilder.getClasses()) {
>
> // perfom my coverage analysis
>
> }
> }
> });
> reader.setExecutionDataVisitor(new IExecutionDataVisitor() {
> public void visitClassExecution(final ExecutionData data) {
> // empty
> }
> });
>

> however, and because I'm calling my _dump_ function thousands of

> times,
>
> my coverage analysis is taking ages to finish. I've measured the CPU
> time
> and, 45% is
> org.jacoco.core.internal.flow.MethodProbesAdapter.visitLabel(),
> 13% is org.jacoco.core.internal.analysis.SourceNodeImpl.incrementLine.
>
> is there any better way of doing this? am I not using the right API
> functions?
> in your opinion, how can I improve the performance of my coverage
> analysis?
>
> --
> Cheers,
> Jose
>

> --
> You received this message because you are subscribed to the Google
> Groups "JaCoCo and EclEmma Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jacoco+un...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jacoco/762ac31f-641b-46ed-9d66-61956778e2b8%40googlegroups.com
> [2].
> For more options, visit https://groups.google.com/d/optout [3].
>
>
> Links:
> ------
> [1] http://eclemma.org/jacoco/trunk/doc/examples/java/ExecDump.java
> [2]
> https://groups.google.com/d/msgid/jacoco/762ac31f-641b-46ed-9d66-61956778e2b8%40googlegroups.com?utm_medium=email&utm_source=footer
> [3] https://groups.google.com/d/optout

[José Carlos de Campos]

Hi Marc (and thanks for your help),

On Thursday, October 15, 2015 at 3:25:42 PM UTC+1, Marc R. Hoffmann wrote:

Hi,

if you have thousands of test cases and want per test coverage results
you need thousands analysis calls. Maybe you can optimize a bit and e.g.
narrow the scope of the classes analyzed for a specific test.

yes, good call. checking the API I found that it's possible to know if an

ExecutionDataStore has/contains a classname. I've tried something like:

if (execFileLoader.getExecutionDataStore().contains(classname)) {

    analyzer.analyzeClass( ... );

}

but that contains method is *always* returning false after analyzing

the coverage of the first atomic test case. what am I doing wrong? do I

need to reset any class/variable?

Anyway, your code looks a bit strange: Why do you put the analysis into
the visitSessionInfo() method? This forces you to parse the cov byte
Array twice. This should be enough:


ExecFileLoader execFileLoader = new ExecFileLoader();
execFileLoader.load(new ByteArrayInputStream(cov));

final CoverageBuilder coverageBuilder = new CoverageBuilder();
final Analyzer analyzer = new
Analyzer(execFileLoader.getExecutionDataStore(), coverageBuilder);

analyzer.analyzeAll(new File("__classes_directory__"));

for (final IClassCoverage cc : coverageBuilder.getClasses()) {

    // perfom my coverage analysis

}

thanks. I didn't know that I was in fact parsing the byte array twice.

--

Cheers,

Jose 

[José Carlos de Campos]

Hi,

On Thursday, October 15, 2015 at 11:35:54 PM UTC+1, José Carlos de Campos wrote:

Hi Marc (and thanks for your help),

On Thursday, October 15, 2015 at 3:25:42 PM UTC+1, Marc R. Hoffmann wrote:

Hi,

if you have thousands of test cases and want per test coverage results
you need thousands analysis calls. Maybe you can optimize a bit and e.g.
narrow the scope of the classes analyzed for a specific test.

yes, good call. checking the API I found that it's possible to know if an

ExecutionDataStore has/contains a classname. I've tried something like:

if (execFileLoader.getExecutionDataStore().contains(classname)) {

    analyzer.analyzeClass( ... );

}

but that contains method is *always* returning false after analyzing

the coverage of the first atomic test case. what am I doing wrong? do I

need to reset any class/variable?

apparently, it was an bug in my classloader. it's solved and contains

method of ExecutionDataStore is now working as expected.

--

Cheers,

Jose