WF 21 hangs, draws 100% CPU

[Nicklas Karlsson]

I know that this probably isn't a WF issue but I would greatly appreciate help in debugging an issue I'm having. The application works fine until it suddenly starts drawing 100% CPU. 

I managed to grab a thread dump and it appears to be in the middle of doing heavy I/O from the database. Which is really strange because the methods called returns smallish JSON texts. I would assume that if I would have some DB-lock, it would just halt and wait. The database doesn't appear to be under CPU load.

Help would be appreciated since it's a production system and it's getting embarrassing having to restart it :-/

Thanks in advance,

  Nik

[Alkeshkumar Barot iGTB]

Hello, 

It looks clear If you see heavy load due to DB operations, Check DB reports from particular timeframe to understand whats kind of operation Server is doing. 

From Logs, I can see findOrCreateVaadinSession()  method trying to acquire lock on some object and its waiting. There are 51 threads trying to acquire lock. 

You may validate with some Vaadin expert

...

 at com.vaadin.flow.server.VaadinService.lockSession(VaadinService.java:800)
at com.vaadin.flow.server.VaadinService.findOrCreateVaadinSession(VaadinService.java:836)
at com.vaadin.flow.server.VaadinService.findVaadinSession(VaadinService.java:701)
at com.vaadin.flow.server.VaadinService.handleRequest(VaadinService.java:1539)
at com.vaadin.flow.server.VaadinServlet.service(VaadinServlet.java:247)
at com.vaadin.cdi.CdiVaadinServlet.service(CdiVaadinServlet.java:67)

..

Regards,

Alkesh Barot

--
You received this message because you are subscribed to the Google Groups "WildFly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wildfly+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/ff81a11f-43ac-40a1-b5be-f7b477f8029an%40googlegroups.com.

This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at https://www.intellectdesign.com

[Nicklas Karlsson]

(oops, previous reply didn't cc:to group)

The postgresql-server is on the same host and works very fast in normal cases.

I will try to look at DB locks when it happens the next time. Although I think it should just block waiting for the release of the lock in that case.

On Thu, Mar 18, 2021 at 3:20 PM Alkeshkumar Barot iGTB <alkeshku...@intellectdesign.com> wrote:

Hi Nicklas, 

Thats right, WAITING thread will not consume CPU. 

What are below threads? Trying to read from DB at same time, might be due to slow connection or its long read which is increasing CPU utilization.

17- Thread trying to read from DB

   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:143)
at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:112)
at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:70)
at org.postgresql.core.PGStream.receiveChar(PGStream.java:283)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1919)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:291)
- locked <564a9d11> (a org.postgresql.core.v3.QueryExecutorImpl)

Below thread are 8-Thread, these does not looks problem but you can check

   java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <7f73e942> (a sun.nio.ch.Util$3)

Regards,

Alkesh Barot

Adv leave notification: 

On Thu, Mar 18, 2021 at 6:09 PM Nicklas Karlsson <nick...@gmail.com> wrote:

Thanks for the reply. On closer inspection, I'm not actually sure if I interpreted the graphs correctly. There is no noticeable DB load, they might just be waiting but I'm not sure on what. Shouldn't the threads just block if they're waiting for a lock? I can't see any reasonable explanation for the 100% CPU load

--

Nicklas Karlsson, +358 40 5062266

Vaakunatie 10 as 7, 20780 Kaarina

This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at https://www.intellectdesign.com

--

Nicklas Karlsson, +358 40 5062266

Vaakunatie 10 as 7, 20780 Kaarina