Wildfly 18 Server Randomly Shutsdown

858 views
Skip to first unread message

Mat

unread,
Oct 11, 2023, 3:47:09 PM10/11/23
to WildFly
I have an app hosted on a Wildfly 18 Server that will randomly shutdown periodically. (the Wildfly service that is), It's hosted on a Ubuntu 22.0 VM.

 Checking the wildfly logs just shows the message: "Server shutdown has been requested via an OS Signal." Checking syslog just shows Session-{#}.scope: Deactivated successfully. Not sure how I can trackdown the reason for this happening, haven't detected any patterns.

Ken Wills

unread,
Oct 11, 2023, 3:51:24 PM10/11/23
to Mat, WildFly
Hi Matt,


On Wed, Oct 11, 2023, 14:47 Mat <matsoa...@gmail.com> wrote:
I have an app hosted on a Wildfly 18 Server that will randomly shutdown periodically. (the Wildfly service that is), It's hosted on a Ubuntu 22.0 VM.

 Checking the wildfly logs just shows the message: "Server shutdown has been requested via an OS Signal." Checking syslog just shows Session-{#}.scope: Deactivated successfully. Not sure how I can trackdown the reason for this happening, haven't detected any patterns.

The most likely case is the VM is running out of memory and the out-of-memory (OOM) reaper is terminating it, you can check dmesg for clues as to if that is happening here.

Aside from that not sure what else would cause a shutdown like this, how is the server being started, systemd?

Ken


--
You received this message because you are subscribed to the Google Groups "WildFly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wildfly+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/be7e0ec3-1427-41fc-89b8-c32699ffb4a8n%40googlegroups.com.

Mat

unread,
Oct 12, 2023, 11:51:05 AM10/12/23
to WildFly
Hey Ken,

I don't think it's a OOM issue. The VM currently has 25GB ram free and I haven't seen where it has come close to maxing it out or using any of the swap memory.

The server is started using the start command in the /bin/ folder (.standalone.sh)


Ken Wills

unread,
Oct 12, 2023, 11:56:41 AM10/12/23
to Mat, WildFly
Hi Matt,



On Thu, Oct 12, 2023, 10:51 Mat <matsoa...@gmail.com> wrote:
Hey Ken,

I don't think it's a OOM issue. The VM currently has 25GB ram free and I haven't seen where it has come close to maxing it out or using any of the swap memory.

The server is started using the start command in the /bin/ folder (.standalone.sh)

When it does happen do you see anything in the dmesg output? OOM aside you could be getting other errors that would result in the termination of the process, like memory corruption issues etc.

Ken


On Wednesday, 11 October 2023 at 14:51:24 UTC-5 Ken Wills wrote:
Hi Matt,


On Wed, Oct 11, 2023, 14:47 Mat <matsoa...@gmail.com> wrote:
I have an app hosted on a Wildfly 18 Server that will randomly shutdown periodically. (the Wildfly service that is), It's hosted on a Ubuntu 22.0 VM.

 Checking the wildfly logs just shows the message: "Server shutdown has been requested via an OS Signal." Checking syslog just shows Session-{#}.scope: Deactivated successfully. Not sure how I can trackdown the reason for this happening, haven't detected any patterns.

The most likely case is the VM is running out of memory and the out-of-memory (OOM) reaper is terminating it, you can check dmesg for clues as to if that is happening here.

Aside from that not sure what else would cause a shutdown like this, how is the server being started, systemd?

Ken


--
You received this message because you are subscribed to the Google Groups "WildFly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wildfly+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/be7e0ec3-1427-41fc-89b8-c32699ffb4a8n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "WildFly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wildfly+u...@googlegroups.com.

Mat

unread,
Oct 12, 2023, 12:29:52 PM10/12/23
to WildFly
The last time it had happened was last week Thursday, my dmesg command starts at Oct 6 so I'm currently unable to verify. Skimming through the dmesg log file itself though didn't show any OOM messages for me.

Ken Wills

unread,
Oct 12, 2023, 2:08:25 PM10/12/23
to Mat, WildFly
On Thu, Oct 12, 2023 at 11:29 AM Mat <matsoa...@gmail.com> wrote:
The last time it had happened was last week Thursday, my dmesg command starts at Oct 6 so I'm currently unable to verify. Skimming through the dmesg log file itself though didn't show any OOM messages for me.

Well, just to be clear, it may not just be OOM, you could have crashes etc showing up in there as well. I'd take a look the next time it happens and see if you can get some more information on whats happening.

Ken
 

Mat

unread,
Oct 12, 2023, 3:56:50 PM10/12/23
to WildFly
Gotcha. Any example of what error messages I could look out for? I'm hoping it doesn't happen again but it in the event it does I'd like to know what I can look for in whichever logs.

Ken Wills

unread,
Oct 12, 2023, 4:15:03 PM10/12/23
to Mat, WildFly


On Thu, Oct 12, 2023, 14:56 Mat <matsoa...@gmail.com> wrote:
Gotcha. Any example of what error messages I could look out for? I'm hoping it doesn't happen again but it in the event it does I'd like to know what I can look for in whichever logs.

Well, for a crash an example might be a default, and you'll see a stack trace in the dmesg output etc. It's somewhat difficult to be specific as it could be any number of things. 

Some other random thoughts: 

This may not apply to you at all but worth remembering: it's not unheard for automation etc to kill off processes in some environments, so I'd also check the output from the 'last' command too see if you have something you're not expecting logging in and terminating things.

Also if you're starting Wildfly in a shell and logging out, the default will be to terminate the process on logout I believe, so you can try using 'nohup ./standalone.sh'  and see if that helps (or run it in tmux or screen.)

Sorry I can't be more specific -- if you get any more info perhaps I could be more helpful.

Ken


Mat

unread,
Oct 13, 2023, 9:51:50 AM10/13/23
to WildFly
Thank you for the heads up. I'll make sure to take note if (hopefully not) it happens again.
Reply all
Reply to author
Forward
0 new messages