Excessive memory usage on test after update to Java 8

[Mads Nielsen]

Hi Jenkins devs

I recently decided to upgrade our windows test slave, which has the tools we need to run our Jenkins Integration Tests. We use JenkinsRule to teardown and setup Jenkins environments before each tests, and it looks like this has started to leak 'processes' after our upgrade to Java 8. 

Has anyone here experienced anything similar? Our builds went from using ~1Gb of memory to more to roughly ~2.5Gb which out of memoried my slave. 

Best regards,

Mads 

Mads Nielsen
Consultant
m...@praqma.net
+45 50 98 18 09
Skype: inkspot

Praqma
www.praqma.com
DK: CPH, Aarhus, Allerod
NO: OSL
SE: STHLM
+45 36 PRAQMA

[Kanstantsin Shautsou]

Hi, Praqma consultant.

Am i right understand that it some closed sources tests similar to ATH?

Please provide what java (oracle, openjdk) with what version was installed before and after. 

Verify that thread count for tests runs didn't changed.

Have you tried to jump between minor java8 versions?

Have you tried some profilers, jmap, etc?

[Mads Nielsen]

Hi Kanstantin...

It is an open source project...and i also figured out what the problem is. 

We have this section in the pom, to limit the resouces used for testing, since we require an expensive tool (ClearCase UCM) to run on the machine as well.

          <plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-failsafe-plugin</artifactId>

                <version>2.19</version>

                <configuration>

                    <forkCount>1</forkCount>

                    <reuseForks>true</reuseForks>

                    <argLine>-Xms512m -Xmx512m -Xss1m</argLine>                    

                </configuration>

                <executions>

                    <execution>

                        <goals>

                            <goal>integration-test</goal>

                            <goal>verify</goal>

                        </goals>

                    </execution>

                </executions>

            </plugin>

The lines i marked with bold seems to be completely ignored in Java 8, it spawns multiple java processes on the slave and eventually starves the slave for memory, which causes the tool (ClearCase UCM) to behave erratically which results in wierd Interrupted exceptions during build:

java.lang.InterruptedException
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:502)
	at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:73)
	at net.praqma.hudson.test.CCUCMRule.buildProject(CCUCMRule.java:380)
	at net.praqma.hudson.test.CCUCMRule.initiateBuild(CCUCMRule.java:281)
	at net.praqma.hudson.test.CCUCMRule.initiateBuild(CCUCMRule.java:264)
	at net.praqma.hudson.test.integration.child.BaselinesFoundIT.initiateBuild(BaselinesFoundIT.java:36)
	at net.praqma.hudson.test.integration.child.BaselinesFoundIT.testRecommended(BaselinesFoundIT.java:59)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at net.praqma.clearcase.test.junit.ClearCaseRule$1.evaluate(ClearCaseRule.java:218)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.jvnet.hudson.test.JenkinsRule$2.evaluate(JenkinsRule.java:486)
	at net.praqma.util.test.junit.LoggingRule$1.evaluate(LoggingRule.java:67)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:344)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:269)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:240)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:184)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:286)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:240)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)

I fixed it by not running the tests using Java 8, 7 is ok since we came from 6. 

Best regards,

Mads

Mads Nielsen
Consultant
m...@praqma.net
+45 50 98 18 09
Skype: inkspot

Praqma
www.praqma.com
DK: CPH, Aarhus, Allerod
NO: OSL
SE: STHLM
+45 36 PRAQMA

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/328506d8-ab77-40e4-a22f-c1d565491142%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.