[cas-user] OutOfMemoryError running CAS 3.5.2 via Jetty
Lazar, Michael E
Hello!
We are in production now with a slightly modified CAS 3.5.2 on Jetty (jdk 1.7): modified with some minor bug fixes and we did add a module to provide “forgot my password” functionality for our users.
We are experiencing OutOfMemoryErrors with CAS running. This error that I am including the trace of occurred within 8 hours of the service startup which we are bouncing nightly at 4AM due to these memory issues.
Our hunch is that it has to do with the EhCache module but this is currently only speculation, which is why I am writing the group for information that could relate to our issue and/or some data as to what we do to resolve it.
Please let me know if you need any additional information and I will reply promptly.
Thanks all for your attention!
-Michael.
2013-06-05 12:53:48.212:WARN:oejs.ServletHandler:qtp1774300666-10037:
org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:972)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.jasig.cas.web.init.SafeDispatcherServlet.service_aroundBody2(SafeDispatcherServlet.java:128)
at org.jasig.cas.web.init.SafeDispatcherServlet.service_aroundBody3$advice(SafeDispatcherServlet.java:57)
at org.jasig.cas.web.init.SafeDispatcherServlet.service(SafeDispatcherServlet.java:1)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:671)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1448)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at com.github.inspektr.common.web.ClientInfoThreadLocalFilter.doFilter(ClientInfoThreadLocalFilter.java:63)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:447)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:449)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:589)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:520)
at java.lang.Thread.run(Thread.java:722)
2013-06-05 12:53:48.212:WARN:oejs.ServletHandler:qtp1774300666-10037:
org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:972)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.jasig.cas.web.init.SafeDispatcherServlet.service_aroundBody2(SafeDispatcherServlet.java:128)
at org.jasig.cas.web.init.SafeDispatcherServlet.service_aroundBody3$advice(SafeDispatcherServlet.java:57)
at org.jasig.cas.web.init.SafeDispatcherServlet.service(SafeDispatcherServlet.java:1)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:671)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1448)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at com.github.inspektr.common.web.ClientInfoThreadLocalFilter.doFilter(ClientInfoThreadLocalFilter.java:63)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:447)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:376)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:449)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:589)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:520)
at java.lang.Thread.run(Thread.java:722)
Caused by:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.String.replace(String.java:2030)
at java.io.ObjectStreamClass.getClassSignature(ObjectStreamClass.java:1486)
at java.io.ObjectStreamClass.getMethodSignature(ObjectStreamClass.java:1500)
at java.io.ObjectStreamClass.access$2500(ObjectStreamClass.java:69)
at java.io.ObjectStreamClass$MemberSignature.<init>(ObjectStreamClass.java:1828)
at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1755)
at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:69)
at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:247)
at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:245)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:244)
at java.io.ObjectStreamClass.writeNonProxy(ObjectStreamClass.java:709)
at java.io.ObjectOutputStream.writeClassDescriptor(ObjectOutputStream.java:664)
at java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1269)
at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1227)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1411)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:346)
at org.springframework.webflow.execution.repository.snapshot.SerializedFlowExecutionSnapshot.serialize(SerializedFlowExecutionSnapshot.java:173)
at org.springframework.webflow.execution.repository.snapshot.SerializedFlowExecutionSnapshot.<init>(SerializedFlowExecutionSnapshot.java:70)
at org.springframework.webflow.execution.repository.snapshot.SerializedFlowExecutionSnapshotFactory.createSnapshot(SerializedFlowExecutionSnapshotFactory.java:70)
at org.springframework.webflow.execution.repository.snapshot.AbstractSnapshottingFlowExecutionRepository.snapshot(AbstractSnapshottingFlowExecutionRepository.java:75)
at org.springframework.webflow.execution.repository.impl.DefaultFlowExecutionRepository.updateFlowExecutionSnapshot(DefaultFlowExecutionRepository.java:142)
at org.springframework.webflow.engine.impl.FlowExecutionImpl.updateCurrentFlowExecutionSnapshot(FlowExecutionImpl.java:425)
at org.springframework.webflow.engine.impl.RequestControlContextImpl.updateCurrentFlowExecutionSnapshot(RequestControlContextImpl.java:222)
at org.springframework.webflow.engine.ViewState.updateHistory(ViewState.java:330)
at org.springframework.webflow.engine.ViewState.exit(ViewState.java:249)
at org.springframework.webflow.engine.Transition.execute(Transition.java:224)
at org.springframework.webflow.engine.impl.FlowExecutionImpl.execute(FlowExecutionImpl.java:393)
at org.springframework.webflow.engine.impl.RequestControlContextImpl.execute(RequestControlContextImpl.java:214)
at org.springframework.webflow.engine.TransitionableState.handleEvent(TransitionableState.java:119)
at org.springframework.webflow.engine.Flow.handleEvent(Flow.java:555)
java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.sun.tools.javac.main.Main.compile(Main.java:475)
at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:132)
at org.apache.jasper.compiler.Jsr199JavaCompiler.compile(Jsr199JavaCompiler.java:248)
at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:384)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:453)
at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:625)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:374)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:492)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:378)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:671)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:449)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:376)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:261)
at org.eclipse.jetty.server.Dispatcher.error(Dispatcher.java:110)
at org.eclipse.jetty.servlet.ErrorPageErrorHandler.handle(ErrorPageErrorHandler.java:142)
at org.eclipse.jetty.server.Response.sendError(Response.java:360)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:526)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:376)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:449)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:589)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:520)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.String.substring(String.java:1913)
at com.sun.tools.javac.file.RelativePath$RelativeDirectory.dirname(RelativePath.java:120)
at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.readEntry(ZipFileIndex.java:657)
at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.buildIndex(ZipFileIndex.java:580)
at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.access$000(ZipFileIndex.java:484)
at com.sun.tools.javac.file.ZipFileIndex.checkIndex(ZipFileIndex.java:192)
at com.sun.tools.javac.file.ZipFileIndex.<init>(ZipFileIndex.java:137)
at com.sun.tools.javac.file.ZipFileIndexCache.getZipFileIndex(ZipFileIndexCache.java:100)
at com.sun.tools.javac.file.JavacFileManager.openArchive(JavacFileManager.java:548)
at com.sun.tools.javac.file.JavacFileManager.openArchive(JavacFileManager.java:482)
at com.sun.tools.javac.file.JavacFileManager.listContainer(JavacFileManager.java:368)
at com.sun.tools.javac.file.JavacFileManager.list(JavacFileManager.java:644)
at javax.tools.ForwardingJavaFileManager.list(ForwardingJavaFileManager.java:77)
at org.apache.jasper.compiler.Jsr199JavaCompiler$2.list(Jsr199JavaCompiler.java:377)
at com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager.list(ClientCodeWrapper.java:212)
at com.sun.tools.javac.jvm.ClassReader.fillIn(ClassReader.java:2492)
at com.sun.tools.javac.jvm.ClassReader.complete(ClassReader.java:2143)
at com.sun.tools.javac.code.Symbol.complete(Symbol.java:421)
at com.sun.tools.javac.comp.Enter.visitTopLevel(Enter.java:298)
at com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:459)
at com.sun.tools.javac.comp.Enter.classEnter(Enter.java:258)
at com.sun.tools.javac.comp.Enter.classEnter(Enter.java:272)
at com.sun.tools.javac.comp.Enter.complete(Enter.java:484)
at com.sun.tools.javac.comp.Enter.main(Enter.java:469)
at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:929)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:824)
at com.sun.tools.javac.main.Main.compile(Main.java:439)
at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:132)
at org.apache.jasper.compiler.Jsr199JavaCompiler.compile(Jsr199JavaCompiler.java:248)
at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:384)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:453)
at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:625)
2013-06-05 12:53:48.842:WARN:oejs.ErrorPageErrorHandler:qtp1774300666-9960: EXCEPTION
org.apache.jasper.JasperException: PWC6033: Error in Javac compilation for JSP
at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:634)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:374)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:492)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:378)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:671)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:449)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:376)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:261)
at org.eclipse.jetty.server.Dispatcher.error(Dispatcher.java:110)
at org.eclipse.jetty.servlet.ErrorPageErrorHandler.handle(ErrorPageErrorHandler.java:142)
at org.eclipse.jetty.server.Response.sendError(Response.java:360)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:526)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:376)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:449)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:589)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:520)
at java.lang.Thread.run(Thread.java:722)
Caused by:
java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.sun.tools.javac.main.Main.compile(Main.java:475)
at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:132)
at org.apache.jasper.compiler.Jsr199JavaCompiler.compile(Jsr199JavaCompiler.java:248)
at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:384)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:453)
at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:625)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:374)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:492)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:378)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:671)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:449)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:376)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:261)
at org.eclipse.jetty.server.Dispatcher.error(Dispatcher.java:110)
at org.eclipse.jetty.servlet.ErrorPageErrorHandler.handle(ErrorPageErrorHandler.java:142)
at org.eclipse.jetty.server.Response.sendError(Response.java:360)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:526)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:376)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:258)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:449)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:589)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:520)
at java.lang.Thread.run(Thread.java:722)
Caused by:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.String.substring(String.java:1913)
at com.sun.tools.javac.file.RelativePath$RelativeDirectory.dirname(RelativePath.java:120)
at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.readEntry(ZipFileIndex.java:657)
at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.buildIndex(ZipFileIndex.java:580)
at com.sun.tools.javac.file.ZipFileIndex$ZipDirectory.access$000(ZipFileIndex.java:484)
at com.sun.tools.javac.file.ZipFileIndex.checkIndex(ZipFileIndex.java:192)
at com.sun.tools.javac.file.ZipFileIndex.<init>(ZipFileIndex.java:137)
at com.sun.tools.javac.file.ZipFileIndexCache.getZipFileIndex(ZipFileIndexCache.java:100)
at com.sun.tools.javac.file.JavacFileManager.openArchive(JavacFileManager.java:548)
at com.sun.tools.javac.file.JavacFileManager.openArchive(JavacFileManager.java:482)
at com.sun.tools.javac.file.JavacFileManager.listContainer(JavacFileManager.java:368)
at com.sun.tools.javac.file.JavacFileManager.list(JavacFileManager.java:644)
at javax.tools.ForwardingJavaFileManager.list(ForwardingJavaFileManager.java:77)
at org.apache.jasper.compiler.Jsr199JavaCompiler$2.list(Jsr199JavaCompiler.java:377)
at com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager.list(ClientCodeWrapper.java:212)
at com.sun.tools.javac.jvm.ClassReader.fillIn(ClassReader.java:2492)
at com.sun.tools.javac.jvm.ClassReader.complete(ClassReader.java:2143)
at com.sun.tools.javac.code.Symbol.complete(Symbol.java:421)
at com.sun.tools.javac.comp.Enter.visitTopLevel(Enter.java:298)
at com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:459)
at com.sun.tools.javac.comp.Enter.classEnter(Enter.java:258)
at com.sun.tools.javac.comp.Enter.classEnter(Enter.java:272)
at com.sun.tools.javac.comp.Enter.complete(Enter.java:484)
at com.sun.tools.javac.comp.Enter.main(Enter.java:469)
at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:929)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:824)
at com.sun.tools.javac.main.Main.compile(Main.java:439)
at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:132)
at org.apache.jasper.compiler.Jsr199JavaCompiler.compile(Jsr199JavaCompiler.java:248)
at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:384)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:453)
at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:625)
2013-06-05 12:53:48.767:WARN:oejut.QueuedThreadPool:qtp1774300666-10037:
--
You are currently subscribed to cas-...@lists.jasig.org as: jasig-cas-user...@googlegroups.com
To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-user
Scott Battaglia
--
You are currently subscribed to cas-...@lists.jasig.org as: scott.b...@gmail.com
To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-user
Marvin S. Addison
> I am including the trace of occurred within 8 hours of the service
> startup which we are bouncing nightly at 4AM due to these memory issues.
>
> Our hunch is that it has to do with the EhCache module but this is
> currently only speculation
thread dump that suggests a problem there. In cases like these we
typically enable the YourKit agent and capture periodic snapshots and
then analyze them offline. If you've got a resource leak, that should
help you find it. If you don't have access to YourKit, then the built-in
JVM tool jmap can be used as a poor man's alternative.
M
Lazar, Michael E
We are running under red hat 5.9 ala vmware, the servers are 4GB with -Xms and -Xms at 2048. One thought has been to increase the system's memory although those thoughts are also coupled with thoughts that the heap will just take longer to fill up.
I will look into the yourkit solution -- I have been attempting at profiling with jvisualvm, but perhaps yourkit is a bit more helpful since I'm not sure exactly what we're looking for.
Anything else that comes to mind will be helpful, thanks again.
-Michael.
-----Original Message-----
From: Marvin S. Addison [mailto:marvin....@gmail.com]
Sent: Thursday, June 06, 2013 1:02 PM
To: cas-...@lists.jasig.org
Subject: Re: [cas-user] OutOfMemoryError running CAS 3.5.2 via Jetty
> We are experiencing OutOfMemoryErrors with CAS running. This error
> that I am including the trace of occurred within 8 hours of the
> service startup which we are bouncing nightly at 4AM due to these memory issues.
>
> Our hunch is that it has to do with the EhCache module but this is
> currently only speculation
What evidence do you have for Ehcache? I don't see anything in the thread dump that suggests a problem there. In cases like these we typically enable the YourKit agent and capture periodic snapshots and then analyze them offline. If you've got a resource leak, that should help you find it. If you don't have access to YourKit, then the built-in JVM tool jmap can be used as a poor man's alternative.
M
--
Scott Battaglia
To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-user
Lazar, Michael E
Hi,
Thanks for the attention
These settings in ticketExpirationPolicies.xml?
We have the ST set at 60 seconds, and the TGT set at 1800 seconds, this is both values, time to live and time to kill.
Correct?
${tgt.timeToKillInSeconds:1800}
Your suggestions are welcome.
Thanks,
-Michael.
Scott Battaglia
Lazar, Michael E
Now that is a great question. I’m not sure where I would find this value – in the ehcache-replicated.xml? I don’t think I have anything defined there.
Let me know where to look, thanks!
Misagh Moayyed
They would be in ticketRegistry.xml, which is typically the place you have defined the instance of EhCacheTicketRegistry. Caches for each ticket type should have their own eviction policy.
--
You are currently subscribed to cas-...@lists.jasig.org as: mmoa...@unicon.net
To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-user
Lazar, Michael E
Sorry, we are working on a couple things currently.
I’ve found our memoryStoreEvictionPolicy is set to LRU, and I do notice that the ST has a time to idle of 0, and time to live is 300. The TGT is set to 1800 idle and live.
So my first thought is to change the ST TTI to 1800 like the other values, is LRU preferred?
Thanks again for the advice.