Skip to first unread message
Joshua
Nov 1, 2018, 5:09:07 PM11/1/18
to Payara Forum
I haven't been able to get the TimerService which is injected by @Resource to work on a clustered Payara server. It works just fine as a standalone server, but attempting to do a deployment group with the same application fails with RuntimeException: EJB Timer Service is not available. This is on Payara 5 183, Windows OpenJDK 8 181.
Any idea's or help would be great.
HASUNUMA Kenji
Nov 27, 2018, 6:33:16 PM11/27/18
to Payara Forum
Hi Joshua,
2018年11月2日金曜日 6時09分07秒 UTC+9 Joshua:
If the exception is caused by java.sql.SQLException, you should use a separate RDBMS system (Oracle, PostgreSQL, MySQL, etc.) to handle the data for EJB timers instead of embedded database. Especially, embedded Derby doesn't work such a scenario.
Thanks,
Thanks,
Kenji
2018年11月2日金曜日 6時09分07秒 UTC+9 Joshua:
Joshua
Nov 27, 2018, 6:42:14 PM11/27/18
to Payara Forum
I put in a separate RDBMS system to handle the Timer service and installed the database scripts from the Payara folder. It would have been nice to see that in the documentation. Found some old GlassFish 3.0 documentation that stated to change the EJB Timer to a different database and use the scripts in the folder to add the correct tables, for clustering environments.
There are still times that this is a problem, seems random for now. I get a 404 page inside of Payara trying to view EJB Timers. The server log file only state that timer service is unavailable and does not state a reason more than that. Database and tables are good and working during those times.
Joshua
Dec 5, 2018, 5:54:30 PM12/5/18
to Payara Forum
Updated to 184 and the EJB List timers are giving the HTTP Status 500 error, the same JDBC tables that use to work are still setup. It also provide the same message and error when the the deployment group is stopped, shouldn't that provide a different result? The stack trace is the following from the server.log.
[2018-12-05T16:38:39.329-0600] [Payara 5.184] [WARNING] [] [javax.enterprise.web] [tid: _ThreadID=87 _ThreadName=admin-thread-pool::admin-listener(1)] [timeMillis: 1544049519329] [levelValue: 900] [[
StandardWrapperValve[FacesServlet]: Servlet.service() for servlet FacesServlet threw exception
java.lang.IllegalStateException: getOutputStream() has already been called for this response
at org.apache.catalina.connector.Response.getWriter(Response.java:737)
at org.apache.catalina.connector.ResponseFacade.getWriter(ResponseFacade.java:221)
at com.sun.faces.context.ExternalContextImpl.getResponseOutputWriter(ExternalContextImpl.java:936)
at com.sun.faces.context.PartialViewContextImpl.createPartialResponseWriter(PartialViewContextImpl.java:561)
at com.sun.faces.context.PartialViewContextImpl.access$300(PartialViewContextImpl.java:87)
at com.sun.faces.context.PartialViewContextImpl$DelayedInitPartialResponseWriter.getWrapped(PartialViewContextImpl.java:702)
at javax.faces.context.PartialResponseWriter.startDocument(PartialResponseWriter.java:115)
at com.sun.faces.context.AjaxExceptionHandlerImpl.handlePartialResponseError(AjaxExceptionHandlerImpl.java:202)
at com.sun.faces.context.AjaxExceptionHandlerImpl.handle(AjaxExceptionHandlerImpl.java:124)
at javax.faces.context.ExceptionHandlerWrapper.handle(ExceptionHandlerWrapper.java:117)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:118)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:223)
at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:732)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:475)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1628)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:339)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:209)
at com.sun.webui.jsf.util.UploadFilter.doFilter(UploadFilter.java:233)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:251)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:209)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:755)
at org.apache.catalina.core.StandardPipeline.doChainInvoke(StandardPipeline.java:579)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:159)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.lang.Thread.run(Thread.java:748)
]]
StandardWrapperValve[FacesServlet]: Servlet.service() for servlet FacesServlet threw exception
java.lang.IllegalStateException: getOutputStream() has already been called for this response
at org.apache.catalina.connector.Response.getWriter(Response.java:737)
at org.apache.catalina.connector.ResponseFacade.getWriter(ResponseFacade.java:221)
at com.sun.faces.context.ExternalContextImpl.getResponseOutputWriter(ExternalContextImpl.java:936)
at com.sun.faces.context.PartialViewContextImpl.createPartialResponseWriter(PartialViewContextImpl.java:561)
at com.sun.faces.context.PartialViewContextImpl.access$300(PartialViewContextImpl.java:87)
at com.sun.faces.context.PartialViewContextImpl$DelayedInitPartialResponseWriter.getWrapped(PartialViewContextImpl.java:702)
at javax.faces.context.PartialResponseWriter.startDocument(PartialResponseWriter.java:115)
at com.sun.faces.context.AjaxExceptionHandlerImpl.handlePartialResponseError(AjaxExceptionHandlerImpl.java:202)
at com.sun.faces.context.AjaxExceptionHandlerImpl.handle(AjaxExceptionHandlerImpl.java:124)
at javax.faces.context.ExceptionHandlerWrapper.handle(ExceptionHandlerWrapper.java:117)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:118)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:223)
at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:732)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:475)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1628)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:339)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:209)
at com.sun.webui.jsf.util.UploadFilter.doFilter(UploadFilter.java:233)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:251)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:209)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:755)
at org.apache.catalina.core.StandardPipeline.doChainInvoke(StandardPipeline.java:579)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:159)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.lang.Thread.run(Thread.java:748)
]]
Joshua
Dec 5, 2018, 6:11:38 PM12/5/18
to Payara Forum
Also noticing one of theses errors in the server.log file as well for every instance.
[2018-12-05T17:10:22.856-0600] [Payara 5.184] [SEVERE] [] [javax.enterprise.system.tools.deployment.common] [tid: _ThreadID=86 _ThreadName=payara-executor-service-task] [timeMillis: 1544051422856] [levelValue: 1000] [[
Exception while parsing file file:/C:/Program%20Files/Payara%205/glassfish/domains/production/applications/application/
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@76daa11b rejected from java.util.concurrent.ThreadPoolExecutor@4bfda67f[Running, pool size = 20, active threads = 20, queued tasks = 30, completed tasks = 13]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at org.glassfish.hk2.classmodel.reflect.Parser.parse(Parser.java:293)
at com.sun.enterprise.v3.server.ReadableArchiveScannerAdapter.handleJar(ReadableArchiveScannerAdapter.java:197)
at com.sun.enterprise.v3.server.ReadableArchiveScannerAdapter.onSelectedEntries(ReadableArchiveScannerAdapter.java:131)
at org.glassfish.hk2.classmodel.reflect.Parser.doJob(Parser.java:345)
at org.glassfish.hk2.classmodel.reflect.Parser.access$300(Parser.java:68)
at org.glassfish.hk2.classmodel.reflect.Parser$3.call(Parser.java:304)
at org.glassfish.hk2.classmodel.reflect.Parser$3.call(Parser.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
]]
Exception while parsing file file:/C:/Program%20Files/Payara%205/glassfish/domains/production/applications/application/
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@76daa11b rejected from java.util.concurrent.ThreadPoolExecutor@4bfda67f[Running, pool size = 20, active threads = 20, queued tasks = 30, completed tasks = 13]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at org.glassfish.hk2.classmodel.reflect.Parser.parse(Parser.java:293)
at com.sun.enterprise.v3.server.ReadableArchiveScannerAdapter.handleJar(ReadableArchiveScannerAdapter.java:197)
at com.sun.enterprise.v3.server.ReadableArchiveScannerAdapter.onSelectedEntries(ReadableArchiveScannerAdapter.java:131)
at org.glassfish.hk2.classmodel.reflect.Parser.doJob(Parser.java:345)
at org.glassfish.hk2.classmodel.reflect.Parser.access$300(Parser.java:68)
at org.glassfish.hk2.classmodel.reflect.Parser$3.call(Parser.java:304)
at org.glassfish.hk2.classmodel.reflect.Parser$3.call(Parser.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
]]
Rudy De Busscher
Dec 6, 2018, 4:23:53 AM12/6/18
to Payara Forum
Hi Joshua,
We are aware of this issue and working on a solution/workaround.
Grtz
Rudy
Joshua
Dec 6, 2018, 5:20:01 PM12/6/18
to Payara Forum
Thanks, It does appear that running the command "asadmin set-payara-executor-service-configuration --threadpoolexecutorqueuesize 1000" solved the timer service unavailable problem.
Rudy De Busscher
Dec 7, 2018, 2:28:09 AM12/7/18
to Payara Forum
Hi Joshua,
Setting the thread pool executor queue size should solve the issue of deploying larger applications and is a fix for this Exception you saw
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@76daa11b rejected from java.util.concurrent.ThreadPoolExecutor@4bfda67f[Running, pool size = 20, active threads = 20, queued tasks = 30, completed tasks = 13]
We don't know how this is related to your EJB timer issue.
Rudy
Reply all
Reply to author
Forward
0 new messages