[19.04] PROXYCAS - java.io.IOException: Too many open files

B D

unread,

Sep 17, 2020, 4:29:00 AM9/17/20

to georchestra

Bonjour à toutes et tous,

Le service tomcat-proxyas a planté laissant un message d'erreur dans le navigateur :

17-Sep-2020 08:20:35.412 SEVERE [http-nio-8180-exec-109] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [proxy] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalStateException: org.apache.http.nio.reactor.IOReactorException: Failure opening selector] with root cause

java.io.IOException: Too many open files

Ne sachant quoi faire pour y remédier, j'ai redémarré le service et tout est rentré dans l'ordre.

Sachant que le serveur est largement sous-utilisé, auriez-vous une idée sur ce qu'il s'est passé ?

D'autres erreurs semblent associées au problème :

java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED

java.io.FileNotFoundException: /var/lib/tomcat-proxycas/webapps/ROOT/WEB-INF/lib/json-20180813.jar (Too many open files)

java.io.FileNotFoundException: /var/lib/tomcat-proxycas/webapps/ROOT/WEB-INF/lib/postgresql-42.2.5.jar (Too many open files)

java.lang.IllegalStateException: java.io.FileNotFoundException: /var/lib/tomcat-proxycas/webapps/cas/WEB-INF/lib/commons-collections-3.2.2.jar (Too many open files)

java.lang.IllegalStateException: java.io.FileNotFoundException: /var/lib/tomcat-proxycas/webapps/ROOT/WEB-INF/lib/spring-beans-4.3.22.RELEASE.jar (Too many open files)

B.D.

=================STACK TRACE COMPLET=============

17-Sep-2020 08:20:35.412 SEVERE [http-nio-8180-exec-109] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [proxy] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalStateException: org.apache.http.nio.reactor.IOReactorException: Failure opening selector] with root cause

    java.io.IOException: Too many open files
        at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
        at sun.nio.ch.EPollArrayWrapper.<init>(EPollArrayWrapper.java:130)
        at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:69)
        at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36)
        at java.nio.channels.Selector.open(Selector.java:227)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.<init>(AbstractMultiworkerIOReactor.java:142)
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.<init>(DefaultConnectingIOReactor.java:79)
        at org.apache.http.impl.nio.client.IOReactorUtils.create(IOReactorUtils.java:43)
        at org.apache.http.impl.nio.client.HttpAsyncClientBuilder.build(HttpAsyncClientBuilder.java:667)
        at org.georchestra.security.Proxy.handleRequest(Proxy.java:595)
        at org.georchestra.security.Proxy.handlePathEncodedRequests(Proxy.java:513)
        at org.georchestra.security.Proxy.handleRequest(Proxy.java:349)
        at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:181)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:440)
        at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:428)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.georchestra.security.BasicAuthChallengeByUserAgent.doFilterInternal(BasicAuthChallengeByUserAgent.java:85)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:121)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.georchestra.security.UTF8Filter.doFilter(UTF8Filter.java:32)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:615)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1627)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

Remi Desgrange

unread,

Sep 21, 2020, 5:01:13 AM9/21/20

to georc...@googlegroups.com, georchestra

Bonjour,

Le problème n'est pas un soucis de ressources CPU/mémoire mais une limite du système d'exploitation

=> limite de nombre de fichier ouvert sur le système: cat /proc/sys/fs/file-max

=> limite pour l'utilisateur tomcat (exemple): su - tomcat -c 'ulimit -Sn' -s '/bin/sh'

Ensuite pour changer il faut passer par sysctl. dans sysctl.conf, je crois que sur certain système ça se règle aussi dans /etc/security/limits.conf.

Cordialement/Best Regards, Rémi Desgrange

--
--
Vous avez reçu ce message, car vous êtes abonné au groupe
Groupe "georchestra" georc...@googlegroups.com
voir http://groups.google.fr/group/georchestra

Site web : http://www.georchestra.org

---
Vous recevez ce message, car vous êtes abonné au groupe Google Groupes "georchestra".
Pour vous désabonner de ce groupe et ne plus recevoir d'e-mails le concernant, envoyez un e-mail à l'adresse georchestra...@googlegroups.com.
Cette discussion peut être lue sur le Web à l'adresse https://groups.google.com/d/msgid/georchestra/5e422aeb-28dd-4fe3-8632-3480076a0299o%40googlegroups.com.

B D

unread,

Sep 22, 2020, 5:52:38 AM9/22/20

to georchestra

Merci Rémi pour l'info.

cat /proc/sys/fs/file-max --------------> 26.277.038

su - tomcat -c 'ulimit -Sn' -s '/bin/sh' -----------------> No passwd entry for user 'tomcat'

Avant de modifier quoi que ce soit dans /etc/sysctl.conf ou /etc/security/limits.conf (tous deux présents dans le système), peut-il y avoir des "contradictions" entre la configuration de ces deux fichier et les valeurs se trouvant dans geoserver_data_dir/controlflow.properties (modifié car le serveur permet des valeurs plus grandes, ws.wms.getmap=512 et ows.gwc=1024) ?

Bonne journée,

B.D.

François Van Der Biest

unread,

Sep 23, 2020, 1:59:13 PM9/23/20

to georchestra

On Tue, Sep 22, 2020 at 11:52 AM B D <beno...@gmail.com> wrote:

Merci Rémi pour l'info.

cat /proc/sys/fs/file-max --------------> 26.277.038
su - tomcat -c 'ulimit -Sn' -s '/bin/sh' -----------------> No passwd entry for user 'tomcat'

Avant de modifier quoi que ce soit dans /etc/sysctl.conf ou /etc/security/limits.conf (tous deux présents dans le système), peut-il y avoir des "contradictions" entre la configuration de ces deux fichier et les valeurs se trouvant dans geoserver_data_dir/controlflow.properties (modifié car le serveur permet des valeurs plus grandes, ws.wms.getmap=512 et ows.gwc=1024) ?

Oh oh ces chiffres sont bien trop élevés ...

Usuellement ws.wms.getmap et ows.gwc < 100 (cf les commentaires de https://github.com/georchestra/geoserver_minimal_datadir/blob/master/controlflow.properties )

F.

B D

unread,

Sep 29, 2020, 3:06:10 AM9/29/20

to georchestra

Connaissant les spécifications du serveur, la personne qui l'a installé m'a dit que je pouvais y aller franco.

B.

~$ lscpu

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
Thread(s) per core:    2
Core(s) per socket:    8
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
Stepping:              4
CPU MHz:               801.086
CPU max MHz:           3000.0000
CPU min MHz:           800.0000
BogoMIPS:              4200.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              11264K
NUMA node0 CPU(s):     0-7,16-23
NUMA node1 CPU(s):     8-15,24-31
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke

Reply all

Reply to author

Forward