How to determine permgen errors on cloudbees

[Gavin Mogan]

So I started to merge all my recent changes, including to upgrade to the new pom formats, and had my latest build fail due to it taking over 30 minutes. 

After a bit of detective work, I found the class in question was spitting out PermGen space errors.

That in itself is kinda worry some, and would explain the timeout, but the earlier error actually has me alarmed:

Aug 14, 2016 9:06:33 PM jenkins.InitReactorRunner$1 onTaskFailed

SEVERE: Failed PluginImpl.init

It doesn't actually say how/why it failed. I'd appreciate some help with this. The ever so useful "it runs fine on my laptop/our own ci"

Gavin

Full log file: https://jenkins.ci.cloudbees.com/job/plugins/job/sauce-ondemand-plugin/ws/target/surefire-reports/TEST-hudson.plugins.sauce_ondemand.SauceOnDemandProjectActionTest.xml

-----

Aug 14, 2016 9:05:43 PM org.eclipse.jetty.util.log.Log initialized

INFO: Logging initialized @74560ms

Picking up existing exploded jenkins.war at /scratch/jenkins/workspace/plugins/sauce-ondemand-plugin/target/jenkins-for-test

Aug 14, 2016 9:05:44 PM org.eclipse.jetty.server.Server doStart

INFO: jetty-9.2.12.v20150709

Aug 14, 2016 9:05:44 PM org.eclipse.jetty.webapp.StandardDescriptorProcessor visitServlet

INFO: NO JSP Support for /jenkins, did not find org.eclipse.jetty.jsp.JettyJspServlet

Aug 14, 2016 9:05:44 PM org.eclipse.jetty.server.handler.ContextHandler doStart

INFO: Started o.e.j.w.WebAppContext@58ee0ea1{/jenkins,file:/scratch/jenkins/workspace/plugins/sauce-ondemand-plugin/target/jenkins-for-test/,AVAILABLE}{/scratch/jenkins/workspace/plugins/sauce-ondemand-plugin/target/jenkins-for-test}

Aug 14, 2016 9:05:44 PM org.eclipse.jetty.server.AbstractConnector doStart

INFO: Started ServerConnector@5832d4fa{HTTP/1.1}{localhost:56098}

Aug 14, 2016 9:05:44 PM org.eclipse.jetty.server.Server doStart

INFO: Started @75682ms

Aug 14, 2016 9:05:44 PM org.jvnet.hudson.test.JenkinsRule createWebServer

INFO: Running on http://localhost:56098/jenkins/

Aug 14, 2016 9:05:45 PM jenkins.InitReactorRunner$1 onAttained

INFO: Started initialization

Aug 14, 2016 9:06:28 PM jenkins.InitReactorRunner$1 onAttained

INFO: Listed all plugins

Aug 14, 2016 9:06:31 PM jenkins.InitReactorRunner$1 onAttained

INFO: Prepared all plugins

Aug 14, 2016 9:06:31 PM jenkins.InitReactorRunner$1 onAttained

INFO: Started all plugins

Aug 14, 2016 9:06:31 PM jenkins.InitReactorRunner$1 onAttained

INFO: Augmented all extensions

Aug 14, 2016 9:06:33 PM jenkins.InitReactorRunner$1 onTaskFailed

SEVERE: Failed PluginImpl.init

java.lang.Error: java.lang.reflect.InvocationTargetException

at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)

at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:176)

at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)

at jenkins.model.Jenkins$7.runTask(Jenkins.java:904)

at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)

at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:106)

... 8 more

Caused by: java.lang.OutOfMemoryError: PermGen space

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:800)

at jenkins.util.AntClassLoader.defineClassFromData(AntClassLoader.java:1138)

at hudson.ClassicPluginStrategy$AntClassLoader2.defineClassFromData(ClassicPluginStrategy.java:799)

at jenkins.util.AntClassLoader.getClassFromStream(AntClassLoader.java:1309)

at jenkins.util.AntClassLoader.findClassInComponents(AntClassLoader.java:1365)

at jenkins.util.AntClassLoader.findClass(AntClassLoader.java:1325)

at jenkins.util.AntClassLoader.loadClass(AntClassLoader.java:1078)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:800)

at jenkins.util.AntClassLoader.defineClassFromData(AntClassLoader.java:1138)

at hudson.ClassicPluginStrategy$AntClassLoader2.defineClassFromData(ClassicPluginStrategy.java:799)

at jenkins.util.AntClassLoader.getClassFromStream(AntClassLoader.java:1309)

at jenkins.util.AntClassLoader.findClassInComponents(AntClassLoader.java:1365)

at jenkins.util.AntClassLoader.findClass(AntClassLoader.java:1325)

at jenkins.util.AntClassLoader.loadClass(AntClassLoader.java:1078)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:274)

at hudson.ExtensionFinder$Sezpoz.scout(ExtensionFinder.java:680)

at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:380)

at hudson.ExtensionList.load(ExtensionList.java:300)

at hudson.ExtensionList.ensureLoaded(ExtensionList.java:253)

at hudson.ExtensionList.iterator(ExtensionList.java:143)

at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:379)

at hudson.ExtensionList.load(ExtensionList.java:300)

at hudson.ExtensionList.ensureLoaded(ExtensionList.java:253)

at hudson.ExtensionList.get(ExtensionList.java:158)

at hudson.PluginManager$PluginUpdateMonitor.getInstance(PluginManager.java:1257)

at hudson.maven.PluginImpl.init(PluginImpl.java:58)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

[Jesse Glick]

On Sun, Aug 14, 2016 at 9:58 PM, Gavin Mogan <ga...@gavinmogan.com> wrote:
> So I started to merge all my recent changes, including to upgrade to the new
> pom formats, and had my latest build fail due to it taking over 30 minutes.
> After a bit of detective work, I found the class in question was spitting
> out PermGen space errors.

Probably the job needs to be switched to JDK 8. This has been
necessary in other cases. I just switched the default for all plugins
including yours, and started

https://jenkins.ci.cloudbees.com/job/plugins/job/sauce-ondemand-plugin/340/consoleFull

Hope it helps.

[Gavin Mogan]

This does indeed fix things.

I thought jenkins was targetting java 7 though.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/g_lisqkNnfM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1r1FbQGyufFcm9iELH%2BVrHJ1WJQ33n5eh1LQsD42g%2BWA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Jesse Glick]

On Mon, Aug 15, 2016 at 1:22 PM, Gavin Mogan <ga...@gavinmogan.com> wrote:
> I thought jenkins was targetting java 7 though.

Yes it *can* run on Java 7, but there is no particular reason to build
& test on 7 when 8 is preferred at runtime.

(The parent POM does compile using `-target 7`, and verify that Java
Platform API calls are defined in Java 7 unless you use
`@IgnoreJRERequirement`.)

[Mark Waite]

On Monday, August 15, 2016 at 11:22:36 AM UTC-6, Gavin Mogan wrote:

This does indeed fix things.

I thought jenkins was targetting java 7 though.

Since you've upgraded to a more recent parent pom, I think you can add an argLine property to the properties section of your pom with the permgen value.  The git plugin did that and permgen test failures stopped (as far as I recall).

Refer to https://github.com/jenkinsci/git-plugin/blob/master/pom.xml#L31-L32 

Jessie's suggestion intrigues me (build with Java 8, rely on it to create Java 7 byte code), but I'm still building with Java 7, unit testing with Java 7, then acceptance testing in a Jenkins running Java 8.

Mark Waite

 

On Mon, Aug 15, 2016 at 6:09 AM, Jesse Glick <jgl...@cloudbees.com> wrote:

On Sun, Aug 14, 2016 at 9:58 PM, Gavin Mogan <ga...@gavinmogan.com> wrote:
> So I started to merge all my recent changes, including to upgrade to the new
> pom formats, and had my latest build fail due to it taking over 30 minutes.
> After a bit of detective work, I found the class in question was spitting
> out PermGen space errors.

Probably the job needs to be switched to JDK 8. This has been
necessary in other cases. I just switched the default for all plugins
including yours, and started

https://jenkins.ci.cloudbees.com/job/plugins/job/sauce-ondemand-plugin/340/consoleFull

Hope it helps.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/g_lisqkNnfM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

[Jesse Glick]

On Mon, Aug 15, 2016 at 5:14 PM, Mark Waite <mark.ea...@gmail.com> wrote:
> I think you can add an
> argLine property to the properties section of your pom with the permgen
> value. The git plugin did that and permgen test failures stopped

Java 8 does not have a permanent generation, so this does not apply
unless you go back to building on 7.

[Mark Waite]

Yes, I should have been clear about that.  The git client plugin is currently based on an old enough Jenkins version that it won't successfully run tests with Java 8.  Thus, it is (temporarily) stuck with compiling with Java 7.  I hope to have solved that problem with the release of git client plugin 2.0.0 when it will no longer claim support for Jenkins versions older than 1.625.

Mark Waite

[Gavin Mogan]

Saucelabs plugin is using 

<jenkins.version>1.609.2</jenkins.version>

With the fix releases, I'm not sure if thats newer or older than 1.625

but we've had no issue compiling on java 8

--

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/g_lisqkNnfM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4a028b4d-fa38-48be-92ae-32f787148eca%40googlegroups.com.

[Mark Waite]

On Mon, Aug 15, 2016 at 4:40 PM Gavin Mogan <ga...@gavinmogan.com> wrote:

Saucelabs plugin is using 

<jenkins.version>1.609.2</jenkins.version>

With the fix releases, I'm not sure if thats newer or older than 1.625

but we've had no issue compiling on java 8

That's older than 1.625, but much newer than the 1.532 which is the required core for the git client plugin.  Yes, it is well past time to move the git client plugin to a newer core version.

Mark Waite

 

On Mon, Aug 15, 2016 at 3:35 PM, Mark Waite <mark.ea...@gmail.com> wrote:

On Monday, August 15, 2016 at 3:36:20 PM UTC-6, Jesse Glick wrote:

On Mon, Aug 15, 2016 at 5:14 PM, Mark Waite <mark.ea...@gmail.com> wrote:
> I think you can add an
> argLine property to the properties section of your pom with the permgen
> value.  The git plugin did that and permgen test failures stopped

Java 8 does not have a permanent generation, so this does not apply
unless you go back to building on 7.

Yes, I should have been clear about that.  The git client plugin is currently based on an old enough Jenkins version that it won't successfully run tests with Java 8.  Thus, it is (temporarily) stuck with compiling with Java 7.  I hope to have solved that problem with the release of git client plugin 2.0.0 when it will no longer claim support for Jenkins versions older than 1.625.

Mark Waite

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/g_lisqkNnfM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4a028b4d-fa38-48be-92ae-32f787148eca%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DutoHdVaAKo9ZUXhuajJ6a3SM8nJqSXobPspuqCo2istOg%40mail.gmail.com.