Jenkins resource usage

36 views
Skip to first unread message

Paulo Matos

unread,
Sep 5, 2014, 10:16:07 AM9/5/14
to jenkins...@googlegroups.com
Hi,

Running Jenkins 1.565.1 here, and during a git fetch of new job I notice the web ui getting sluggish so I open htop and see what's attached.
It's slightly worrying that each java instance is allocating 10.2Gb for a starters but more than that, why are there so many of them?
This is a Jenkins which just 10 jobs and only one is running. The machine has 16cores, there's no reason for it to be sluggish.

Does Jenkins by default run everything on a single machine core? Is it possible to tell the Jenkins server to spread its use to all the cores in the machine?

Thanks,

Paulo Matos


jenkins.png

Shane Turner

unread,
Sep 5, 2014, 11:01:42 AM9/5/14
to jenkins...@googlegroups.com
I believe that you are seeing the threads that Java is running. Try "H"
in htop to hide the threads.

Shane
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ This e-mail is intended only for the named recipient(s) and may contain information that is otherwise privileged, confidential and/or exempt from disclosure under applicable law. No waiver of privilege, confidence, or otherwise is intended by virtue of communication via the internet. Any unauthorized use, dissemination or copying is strictly prohibited. If you have received this e-mail in error, or are not the named as a recipient, please immediately notify the sender and destroy all copies of this e-mail. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Paulo Matos

unread,
Sep 5, 2014, 11:09:12 AM9/5/14
to jenkins...@googlegroups.com

> -----Original Message-----
> From: jenkins...@googlegroups.com [mailto:jenkinsci-
> us...@googlegroups.com] On Behalf Of Shane Turner
> Sent: 05 September 2014 16:01
> To: jenkins...@googlegroups.com
> Subject: Re: Jenkins resource usage
>
> On 05/09/2014 11:15 AM, Paulo Matos wrote:
> > Hi,
> >
> > Running Jenkins 1.565.1 here, and during a git fetch of new job I
> notice the web ui getting sluggish so I open htop and see what's
> attached.
> > It's slightly worrying that each java instance is allocating 10.2Gb
> for a starters but more than that, why are there so many of them?
> > This is a Jenkins which just 10 jobs and only one is running. The
> machine has 16cores, there's no reason for it to be sluggish.
> >
> > Does Jenkins by default run everything on a single machine core? Is
> it possible to tell the Jenkins server to spread its use to all the
> cores in the machine?
> >
> > Thanks,
> >
> > Paulo Matos
> >
> I believe that you are seeing the threads that Java is running. Try
> "H"
> in htop to hide the threads.
>


Thanks,

Also I just noticed from the Jenkins logs that it's full of these:

Sep 05, 2014 3:39:37 PM WARNING org.eclipse.jetty.util.log.JavaUtilLog warn
Error while serving http://jenkins-firepathtools.cam.broadcom.com/job/rall-tests/valgrindResult/graph
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor423.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298)
at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161)
at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96)
at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:120)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858)
at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:390)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858)
at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:248)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:631)
at org.kohsuke.stapler.Stapler.service(Stapler.java:225)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:96)
at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99)
at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:89)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99)
at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:88)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:48)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:74)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

Is there anything I can do to help diagnose this? Is this a case of increasing the memory available to the JVM? My machine has 128Gb and it's a machine dedicated to Jenkins but I don't actually see Jenkins use much of it.

Paulo Matos

Paulo Matos

unread,
Sep 5, 2014, 11:11:17 AM9/5/14
to jenkins...@googlegroups.com


> -----Original Message-----
> From: jenkins...@googlegroups.com [mailto:jenkinsci-
> us...@googlegroups.com] On Behalf Of Shane Turner
> Sent: 05 September 2014 16:01
> To: jenkins...@googlegroups.com
> Subject: Re: Jenkins resource usage
>
> On 05/09/2014 11:15 AM, Paulo Matos wrote:
> > Hi,
> >
> > Running Jenkins 1.565.1 here, and during a git fetch of new job I
> notice the web ui getting sluggish so I open htop and see what's
> attached.
> > It's slightly worrying that each java instance is allocating 10.2Gb
> for a starters but more than that, why are there so many of them?
> > This is a Jenkins which just 10 jobs and only one is running. The
> machine has 16cores, there's no reason for it to be sluggish.
> >
> > Does Jenkins by default run everything on a single machine core? Is
> it possible to tell the Jenkins server to spread its use to all the
> cores in the machine?
> >
> > Thanks,
> >
> > Paulo Matos
> >
> I believe that you are seeing the threads that Java is running. Try
> "H"
> in htop to hide the threads.
>

And by the way, yes they seemed to be just threads. 'H' in htop removed all the duplicate java processes.

Thanks,
Paulo Matos
Reply all
Reply to author
Forward
0 new messages