[JIRA] (JENKINS-60618) Jenkins test harness InjectedTest hangs on machine with many cores since parent 3.54 upgrade

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite created an issue

Jenkins / JENKINS-60618 Jenkins test harness InjectedTest hangs on machine with many cores since parent 3.54 upgrade Issue Type: Bug Assignee: Oliver Gondža Components: jenkins-test-harness Created: 2020-01-01 14:03 Environment: Ubuntu 16.04 computer with 72 cores Priority: Major Reporter: Mark Waite After upgrading to parent pom 3.54 (and still with parent pom 3.55) on a Ubuntu 16.04 server with 72 cores, the test suite in two different plugins (platformlabeler-plugin and git-plugin) hangs and never returns. The last message reported from the CI job using an agent on that computer is "InjectedTest". The same last message is visible when the tests are run with mvn clean verify. The messages from platformlabeler-plugin look like this at the hang: [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskLsbReleaseTest [INFO] Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.292 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskLsbReleaseTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskReleaseTest [INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.097 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskReleaseTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskStaticStringTest [INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskStaticStringTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskTest [INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.893 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskTest [INFO] Running InjectedTest The same problem does not occur on an 8 core FreeBSD 11 machine or on 4 core or fewer machines running Ubuntu 18.04, CentOS 7, CentOS 8, Debian 9, Debian 10, and Windows. Setting the forkCount in pom.xml does not seem to change the behavior, since git-plugin has forkCount set to 1 (one test thread) and platformlabeler-plugin has forkCount set to 1T (one test thread per core on the machine). The only workaround I've found has been to disable that agent during tests of those plugins. That is unfortunate, since it is one of the fastest agents available to me. The Jenkins test harness versions reported by mvn dependency:list are: [INFO] org.jenkins-ci.main:jenkins-test-harness:jar:2.57:test [INFO] org.jenkins-ci.main:jenkins-test-harness-htmlunit:jar:2.31-2:test Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite updated an issue

Jenkins / JENKINS-60618 Jenkins test harness InjectedTest hangs on machine with many cores since parent 3.54 upgrade

Change By: Mark Waite

After upgrading to parent pom 3.54 (and still with parent pom 3.55) on a Ubuntu 16.04 server with 72 cores, the test suite in two different plugins (platformlabeler-plugin and git-plugin) hangs and never returns.  The last message reported from the CI job using an agent on that computer is "InjectedTest".  The same last message is visible when the tests are run with {{mvn clean verify}}. The messages from platformlabeler-plugin look like this at the hang:

{noformat}

[INFO] ------------------------------------------------------- [INFO]  T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskLsbReleaseTest [INFO] Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.292 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskLsbReleaseTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskReleaseTest [INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.097 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskReleaseTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskStaticStringTest [INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskStaticStringTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskTest [INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.893 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskTest [INFO] Running InjectedTest

{noformat} The same problem does not occur on an  8 core FreeBSD 11 machine or on 4 core or fewer machines running Ubuntu 16.04, Ubuntu 18.04, CentOS 7, CentOS 8, Debian 9, Debian 10, and or Windows.

Setting the {{forkCount}} in pom.xml does not seem to change the behavior, since git-plugin has {{forkCount}} set to 1 (one test thread) and platformlabeler-plugin has {{forkCount}} set to 1T (one test thread per core on the machine). The only workaround I've found has been to disable that agent during tests of those plugins.  That is unfortunate, since it is one of the fastest agents available to me. The Jenkins test harness versions reported by {{mvn dependency:list}} are:

{noformat}

[INFO]    org.jenkins-ci.main:jenkins-test-harness:jar:2.57:test [INFO]    org.jenkins-ci.main:jenkins-test-harness-htmlunit:jar:2.31-2:test

{noformat}

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite assigned an issue to Mark Waite

Jenkins / JENKINS-60618 Jenkins test harness InjectedTest hangs on machine with many cores since parent 3.54 upgrade

Change By: Mark Waite Assignee: Oliver Gondža Mark Waite

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite updated an issue

Jenkins / JENKINS-60618 Jenkins test harness InjectedTest hangs on machine with many cores since parent 3.54 upgrade Change By: Mark Waite

After upgrading to parent pom 3.54 (and still with parent pom 3.55) on a Ubuntu 16.04 server with 72 cores, the test suite in two different plugins (platformlabeler-plugin and git-plugin) hangs and never returns.  The last message reported from the CI job using an agent on that computer is "InjectedTest".  The same last message is visible when the tests are run with {{mvn clean verify}}. The messages from platformlabeler-plugin look like this at the hang: {noformat} [INFO] ------------------------------------------------------- [INFO]  T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskLsbReleaseTest [INFO] Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.292 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskLsbReleaseTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskReleaseTest [INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.097 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskReleaseTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskStaticStringTest [INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskStaticStringTest [INFO] Running org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskTest [INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.893 s - in org.jvnet.hudson.plugins.platformlabeler.PlatformDetailsTaskTest [INFO] Running InjectedTest {noformat}

The same problem does not occur on an  8 core FreeBSD 11 machine or on 4 core or fewer machines running Ubuntu 16.04, Ubuntu 18.04, CentOS 7, CentOS 8, Debian 9, Debian 10, or Windows. Setting the {{forkCount}} in pom.xml does not seem to change the behavior, since git-plugin has {{forkCount}} set to 1 (one test thread) and platformlabeler-plugin has {{forkCount}} set to 1T 1C (one test thread per core on the machine).

The only workaround I've found has been to disable that agent during tests of those plugins.  That is unfortunate, since it is one of the fastest agents available to me. The Jenkins test harness versions reported by {{mvn dependency:list}} are: {noformat} [INFO]    org.jenkins-ci.main:jenkins-test-harness:jar:2.57:test [INFO]    org.jenkins-ci.main:jenkins-test-harness-htmlunit:jar:2.31-2:test {noformat}

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite assigned an issue to Oliver Gondža

Jenkins / JENKINS-60618 Jenkins test harness InjectedTest hangs on machine with many cores since parent 3.54 upgrade

Change By: Mark Waite Assignee: Mark Waite Oliver Gondža

Add Comment