Task rejected - Thread pool too small?

452 views
Skip to first unread message

Alvaro G [Andor]

unread,
Nov 15, 2019, 6:14:13 AM11/15/19
to dcm...@googlegroups.com

Hi,

I'm testing the migration for a relatively big PACS (receiving 5 to 10 instances per second 24x7) with v5.19.0, and, after some days of testing I always get this kind of errors where it begins rejecting all associations (log below).

At the same time it receives the studies, it's doing delayed compression of hundreds of thousands old studies.

I'm not well versed on this Java/Jboss stuff, but it seems like it's filling a pool of workers or jobs and it won't take any more.

I could reduce the delayed compression jobs, but the server is quite big, the use is very low, and it has ample room for processing which is not being used. Right now it sits 90% idle.

So:

  • Can I enlarge that pool and or the jobs done at the same time somewhere? I have 16 cores waiting for work! ;)
  • Shouldn't an "Exception on accepted connection" where we reject jobs be an ERR and not a WARN?


2019-11-11 15:32:19,989 WARN  [org.dcm4che3.net.Connection] (EE-ManagedExecutorService-default-Thread-2) Exception on accepted connection Socket[addr=/10.10.10.10,port=44451,localport=11112]:: java.util.concurrent.RejectedExecutionException: Task org.glassfish.enterprise.concurrent.internal.ManagedFutureTask@3612af83[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@8f5897b[Wrapped task = org.jboss.as.ee.concurrent.ControlPointUtils$ControlledRunnable@2122cb3c]] rejected from org.glassfish.enterprise.concurrent.internal.ManagedThreadPoolExecutor@18ff67ed[Running, pool size = 100, active threads = 100, queued tasks = 0, completed tasks = 122563]
    at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055)
    at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825)
    at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355)
    at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedExecutorServiceImpl.execute(ManagedExecutorServiceImpl.java:138)
    at org.jbo...@18.0.0.Final//org.jboss.as.ee.concurrent.ManagedExecutorServiceImpl.execute(ManagedExecutorServiceImpl.java:70)
    at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedExecutorServiceAdapter.execute(ManagedExecutorServiceAdapter.java:97)
    at org.dcm...@5.19.0//org.dcm4che3.net.Device.execute(Device.java:1202)
    at org.dcm...@5.19.0//org.dcm4che3.net.Association.activate(Association.java:526)
    at org.dcm...@5.19.0//org.dcm4che3.net.Association.<init>(Association.java:130)
    at org.dcm...@5.19.0//org.dcm4che3.net.DicomProtocolHandler.onAccept(DicomProtocolHandler.java:53)
    at org.dcm...@5.19.0//org.dcm4che3.net.TCPListener.listen(TCPListener.java:126)
    at org.dcm...@5.19.0//org.dcm4che3.net.TCPListener.access$000(TCPListener.java:56)
    at org.dcm...@5.19.0//org.dcm4che3.net.TCPListener$1.run(TCPListener.java:74)
    at org.jbo...@18.0.0.Final//org.jboss.as.ee.concurrent.ControlPointUtils$ControlledRunnable.run(ControlPointUtils.java:105)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedFutureTask.run(ManagedFutureTask.java:141)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
    at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)

Regards

Gunter Zeilinger

unread,
Nov 15, 2019, 6:25:45 AM11/15/19
to dcm...@googlegroups.com
You may verify the the number of threads in the WIldfly Admin console and increase the default value (100) for the maximal number by Docker ENV WILDFLY_EXECUTER_MAX_THREADS or directly in the WIldfly Admin console or in dcm4chee-arc.xml.


Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
--
You received this message because you are subscribed to the Google Groups "dcm4che" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dcm4che+u...@googlegroups.com.

Alvaro G [Andor]

unread,
Nov 15, 2019, 7:38:45 AM11/15/19
to dcm...@googlegroups.com

Great thank you,

I think I should make a list of "performance/tuning" parameters to observe and monitor. I'll try to add that value to our monitoring.

Thanks!

Ankit Gupta

unread,
Jan 3, 2024, 12:29:35 AM1/3/24
to dcm4che
Hi Alvaro,

I'm facing a similar issue. Have you managed to solve the problem? I've installed dcm4chee v5 using the Docker version and configured storage for Amazon S3. I can upload DICOM objects up to 400 MB without any errors, but when attempting to upload DICOM objects of 1 GB, I encounter data loss in my S3 bucket. I suspect it might be related to the threads and pools of dcm4chee.

Could you please provide guidance on this issue? Your help would be greatly appreciated.

Thanks in advance.
Reply all
Reply to author
Forward
0 new messages