Payara server stops at night

[Roberto Marchena]

I have a customer that has an application running on Payara 4.1.2.172 (build 235) and a Linux Ubuntu 14.04.3. The application has been working fine for months, but recently they start to have the problem that everyday when they get to the office at morning the payara server is not running, and they need to start it again to run the application. When they leave the office at night always payara was working fine and next morning they find it stopped again, like while night something kill the process.

Reading the Payara log files it seems as nothing happens between the end of the day and in linux files I can't find anything reated with the problem

Any ideas?

[Mike Croft]

Do you know what time it stops each night? It would be suspicious if it was the same time. Have you got any monitoring set up on Ubuntu to see what the memory situation is like? Sometimes the kernel may kill a process if the server is extremely starved of resources. It may also be worth looking at your JVM heap size settings and comparing that to the available RAM.

[Roberto Marchena]

Hi Mike,

I don't know the time when the process stops, I know that everyday when they close the office at 19 PM the service is working fine, and at 9 AM the Payara server is stopped. I haven't got any monitoring set up on memory situation. Have you any recommendation how to set up the JVM heap size? and what should I compare with available RAM?

I went back to previous versión of Payara 4.1.1.171.1 (build 139), and yesterday was working when they got to the office at 9 AM, but today they had the problem again, so is not a problem related to the last version of Payara.

All the best,
Roberto

[Christoph John]

Hi Roberto,

in my opinion it is either as Mike suggested or the process itself encountered a SIGSEG or similar signal which caused it to crash.

If it is as Mike suggested, then you could check this post to see if the OOM killer killed the Payara process: https://unix.stackexchange.com/questions/354319/how-to-make-oom-killer-log-into-var-log-messages-when-it-kills-any-process

If the process got a signal which caused it to stop you might either find a "core" or "hs_err" file in the payara directory or the directory where your domain is located (if different from payara directory).
Just check if find . -name core or find . -name "hs_err*" are yielding any results.

Cheers,
Chris.