EclEmma coverage failure when running a test method with Guava Cache on JUnit 4.13 within Eclipse

62 views
Skip to first unread message

Dávid Karnok

unread,
Feb 16, 2020, 12:56:33 PM2/16/20
to JaCoCo and EclEmma Users
Hi. I've run into an odd coverage failure with JUnit 4.13, Eclipse, EclEmma and test methods referencing Google Guava's Cache. 

I was advised by the JUnit project to ask about the situation with this project (and/or with Eclipse / Google Guava).


#### Environment:

Eclipse: Version: 2019-09 R (4.13.0), Build id: 20190917-1200
EclEmma 3.1.2.201903112331 org.eclipse.eclemma.feature.feature.group Eclipse EclEmma
JUnit 4.13
Windows 10 x64
Java 1.8u241

#### Issue:

I've run into an odd error while running coverage on an unit test having some methods use Google Guava's CacheBuilder in Eclipse. The coverage fails with the error code 5013 and error report indicating java.net.SocketException: Connection reset. If I remove these methods, the coverage succeeds.

If I revert back to JUnit 4.12, both the test file and the individual test succeeds in producing the coverage data.

Example method:

This is where the cache is created:

#### Additional details:

- If I run "gradlew test jacocoTestReport" on the entire project, it succeeds in creating the report HTML apparently.
- If I run "Coverage As > JUnit test" on the entire project from Eclipse, it fails with the same 5013 error.
- If I run any other test class/package/method in the project, the coverage is created successfully.

Please advise.

Evgeny Mandrikov

unread,
Mar 2, 2020, 11:22:29 AM3/2/20
to JaCoCo and EclEmma Users
Hi,

JaCoCo and hence EclEmma rely on the execution of JVM shutdown hook.
The message "java.net.SocketException: Connection reset" might indicate that JVM shutdown hook was not executed.

Using
Eclipse Version: 2019-09 R (4.13.0), Build id: 20190917-1200


Execution of the following test even without EclEmma 

package io.reactivex.rxjava3.core;

import org.junit.Test;

public class ExampleTest extends RxJavaTest {
    @Test
    public void test() {
    Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
        @Override
        public void run() {
            System.err.println("hook executed");
        }
    }));
    }
}


Won't print anything, whereas execution of the same test without "extends RxJavaTest" will print "hook executed".

The same test will also print "hook executed" if you remove "globalTimeout" defined in "RJavaTest".

From the above IMO clear that the issue is not in EclEmma.

Furthermore, problem with execution of JVM shutdown hooks
in presence of org.junit.rules.Timeout from JUnit 4.13 can be demonstrated
without Eclipse
using following "build.gradle"

apply plugin: 'java'

repositories {
  mavenCentral()
}

test {
  testLogging.showStandardStreams = true
}

dependencies {
  testCompile 'junit:junit:4.13'
}

and following "src/test/java/Example.java"

import org.junit.Test;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.Timeout;

import java.util.concurrent.TimeUnit;

public class ExampleTest {
  @Rule
  public Timeout globalTimeout = new Timeout(5, TimeUnit.MINUTES);

  @Test
  public final void test() {
    Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
      @Override
      public void run() {
        System.err.println("hook executed");
      }
    }));
  }
}


And since the problem goes away in all the above cases after downgrade to JUnit 4.12,
then IMO it is caused by changes in JUnit 4.13.


Regards,
Evgeny

Dávid Karnok

unread,
Mar 2, 2020, 12:11:19 PM3/2/20
to JaCoCo and EclEmma Users
Hi and thanks for the detailed analysis.

Evgeny Mandrikov

unread,
Mar 2, 2020, 12:42:33 PM3/2/20
to JaCoCo and EclEmma Users
Hm, however EclEmma and JaCoCo register hook outside of the test,
and the following example seems to work even with JUnit 4.13

import org.junit.Test;

import org.junit.Rule;
import org.junit.rules.Timeout;

import java.util.concurrent.TimeUnit;

public class ExampleTest {

  static {
    Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
      @Override
      public void run() {
        System.err.println("Hook executed");
      }
    }));
  }

  @Rule
  public Timeout globalTimeout = new Timeout(5, TimeUnit.MINUTES);

  @Test
  public final void test() {
  }
}

But not the following reduction of your
FlowableGroupByTest.mapFactoryWithExpiringGuavaCacheDemonstrationCodeForUseInJavadoc

package io.reactivex.rxjava3.internal.operators.flowable;

import java.util.Map;
import java.util.concurrent.TimeUnit;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.Timeout;
import com.google.common.cache.CacheBuilder;
import io.reactivex.rxjava3.core.Flowable;
import io.reactivex.rxjava3.functions.Consumer;
import io.reactivex.rxjava3.functions.Function;
import io.reactivex.rxjava3.internal.functions.Functions;

public class FlowableGroupByTest {

static {
System.err.println("registering hook");
Thread thread = new Thread(new Runnable() {
@Override
public void run() {
System.err.println("hook executed");
}
});
Runtime.getRuntime().addShutdownHook(thread);
System.err.println(thread.getThreadGroup());
}

@Rule
public Timeout globalTimeout = new Timeout(5, TimeUnit.MINUTES);

private static final Function<Integer, Integer> mod5 = new Function<Integer, Integer>() {
@Override
public Integer apply(Integer n) throws Exception {
return n % 5;
}
};

@Test
public void mapFactoryWithExpiringGuavaCacheDemonstrationCodeForUseInJavadoc() {
Function<Consumer<Object>, Map<Integer, Object>> evictingMapFactory = new Function<Consumer<Object>, Map<Integer, Object>>() {
@Override
public Map<Integer, Object> apply(final Consumer<Object> notify) throws Exception {
return CacheBuilder.newBuilder() //
.<Integer, Object>build().asMap();
}
};
Flowable.range(1, 1) //
.groupBy(mod5, Functions.<Integer>identity(), true, 16, evictingMapFactory) //
.test();
}

}


whose execution without EclEmma sometimes prints "hook executed" and sometimes not.

While the root cause is unclear, I hope that the above helps to understand that the problem is not in EclEmma/JaCoCo and pinpoint the root cause.


Regards,
Evgeny

Dávid Karnok

unread,
Mar 2, 2020, 12:51:12 PM3/2/20
to jac...@googlegroups.com
Thanks for the help. I'm glad at least EclEmma/JaCoCo can be ruled out. Now it's back to the JUnit people...

--
You received this message because you are subscribed to a topic in the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jacoco/SHw3R6cvZSs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/5b1f5db4-62f0-42da-8273-b2719f688cd5%40googlegroups.com.


--
Best regards,
David Karnok

Evgeny Mandrikov

unread,
Mar 2, 2020, 1:48:53 PM3/2/20
to JaCoCo and EclEmma Users
And here is further reduction

import java.util.concurrent.TimeUnit;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.Timeout;

import com.google.common.cache.CacheBuilder;

public class ExampleTest {

  static {
    System.err.println("registering hook");
    Thread thread = new Thread(new Runnable() {
      @Override
      public void run() {
        System.err.println("hook executed");
      }
    });
    Runtime.getRuntime().addShutdownHook(thread);
    System.err.println(thread.getThreadGroup());
  }

  @Rule
  public Timeout timeout = new Timeout(5, TimeUnit.MINUTES);

  @Test
  public void test() {
    CacheBuilder.newBuilder() //
      .<Integer, Object>build();
  }
}

both in Eclipse and in IntelliJ IDEA it sometimes prints "hook executed" and sometimes not.
To unsubscribe from this group and all its topics, send an email to jacoco+unsubscribe@googlegroups.com.

Evgeny Mandrikov

unread,
Mar 3, 2020, 8:44:14 AM3/3/20
to JaCoCo and EclEmma Users
Let's close this thread here by posting missing link on thread created in JUnit project ;)
Where I just posted a detailed explanation of observed behavior.


Regards,
Evgeny
Reply all
Reply to author
Forward
This conversation is locked
You cannot reply and perform actions on locked conversations.
0 new messages