booting WildFly 10.1 on Ubuntu 20.04 fails

[Stijn Bousard]

I'm trying to install Wildfly 10.1 on a VPS with Ubuntu 20.04.
Whatever I try, wildfly doesn't want to boot.

The errors being thrown are:
ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.remoting.endpoint.management.management.operation.handler: org.jboss.msc.service.StartException in service jboss.remoting.endpoint.management.management.operation.handler: Failed to start service

and

12:50:19,626 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("parallel-subsystem-boot") failed - address: ([]): java.lang.OutOfMemoryError: unable to create new native thread

Full log:
admin@myserver:/opt/wildfly/bin$ sudo ./standalone.sh
=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /opt/wildfly

  JAVA: java

  JAVA_OPTS:  -server -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

12:50:14,618 INFO  [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final
12:50:15,336 INFO  [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
12:50:15,512 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049: WildFly Full 10.1.0.Final (WildFly Core 2.2.0.Final) starting
12:50:19,400 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
12:50:19,432 INFO  [org.xnio] (MSC service thread 1-2) XNIO version 3.4.0.Final
12:50:19,442 INFO  [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.4.0.Final
12:50:19,497 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.remoting.endpoint.management.management.operation.handler: org.jboss.msc.service.StartException in service jboss.remoting.endpoint.management.management.operation.handler: Failed to start service
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_312]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_312]
        at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_312]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method) [rt.jar:1.8.0_312]
        at java.lang.Thread.start(Thread.java:717) [rt.jar:1.8.0_312]
        at org.jboss.threads.JBossThread.start(JBossThread.java:342)
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957) [rt.jar:1.8.0_312]
        at java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603) [rt.jar:1.8.0_312]
        at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334) [rt.jar:1.8.0_312]
        at java.util.concurrent.ScheduledThreadPoolExecutor.scheduleWithFixedDelay(ScheduledThreadPoolExecutor.java:597) [rt.jar:1.8.0_312]
        at org.jboss.as.controller.remote.ResponseAttachmentInputStreamSupport.<init>(ResponseAttachmentInputStreamSupport.java:123)
        at org.jboss.as.controller.remote.ResponseAttachmentInputStreamSupport.<init>(ResponseAttachmentInputStreamSupport.java:117)
        at org.jboss.as.controller.remote.AbstractModelControllerOperationHandlerFactoryService.start(AbstractModelControllerOperationHandlerFactoryService.java:90)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
        ... 3 more

12:50:19,552 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-2) MSC000001: Failed to start service jboss.serverManagement.controller.management.worker: org.jboss.msc.service.StartException in service jboss.serverManagement.controller.management.worker: Failed to start service
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_312]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_312]
        at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_312]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method) [rt.jar:1.8.0_312]
        at java.lang.Thread.start(Thread.java:717) [rt.jar:1.8.0_312]
        at org.xnio.nio.NioXnioWorker.start(NioXnioWorker.java:151)
        at org.xnio.nio.NioXnio.createWorker(NioXnio.java:213)
        at org.jboss.as.server.mgmt.ManagementWorkerService.start(ManagementWorkerService.java:53) [wildfly-server-2.2.0.Final.jar:2.2.0.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.6.Final.jar:1.2.6.Final]
        ... 3 more

12:50:19,626 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("parallel-subsystem-boot") failed - address: ([]): java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717)
        at org.jboss.threads.JBossThread.start(JBossThread.java:342)
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
        at org.jboss.as.controller.ParallelBootOperationStepHandler$2.execute(ParallelBootOperationStepHandler.java:274)
        at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:890)
        at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:659)
        at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
        at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1329)
        at org.jboss.as.controller.ModelControllerImpl.boot(ModelControllerImpl.java:493)
        at org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:387)
        at org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:349)
        at org.jboss.as.server.ServerService.boot(ServerService.java:397)
        at org.jboss.as.server.ServerService.boot(ServerService.java:366)
        at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:299)
        at java.lang.Thread.run(Thread.java:748)

12:50:19,662 WARN  [org.jboss.msc.service.fail] (MSC service thread 1-2) MSC000004: Failure during stop of service jboss.as.external-mgmt-executor: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717)
        at org.jboss.threads.JBossThread.start(JBossThread.java:342)
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
        at org.jboss.as.server.ExternalManagementRequestExecutor.stop(ExternalManagementRequestExecutor.java:133)
        at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
        at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

12:50:19,700 FATAL [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0056: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.
admin@myserver:/opt/wildfly/bin$


Any help would be much appreciated!

[James Perkins]

Have you tried a newer version of WildFly? The error though seems pretty clear that there is not enough memory.

[Stijn Bousard]

SOLVED

Your suggestion opened my eyes. Although I saw the 'java.lang.OutOfMemoryError', I thought wrongly that WildFly would start with the default values if no applications were deployed. 

Now I've changed the JAVA_OPTS:

   # default: JAVA_OPTS="-Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true"
  JAVA_OPTS:  -server -Xms384m -Xmx512m -XX:MetaspaceSize=128M -XX:MaxMetaspaceSize=96m -Djava.net.preferIPv4Stack=true 

This way, I was able to launch WildFly manually:

/opt/wildfly/bin/launch.sh standalone standalone.xml 12.345.678.901

When I tried to launch it as a service, it always stopped without ERROR in the server.log:

2022-03-29 11:32:35,829 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0006: Undertow HTTPS listener https listening on 12.345.678.901:8443
2022-03-29 11:32:35,910 INFO  [org.jboss.ws.common.management] (MSC service thread 1-1) JBWS022052: Starting JBossWS 5.1.5.Final (Apache CXF 3.1.6)
2022-03-29 11:32:36,130 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://12.345.678.901:9990/management
2022-03-29 11:32:36,131 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://12.345.678.901:9990
2022-03-29 11:32:36,132 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 10.1.0.Final (WildFly Core 2.2.0.Final) started in 5562ms - Started 331 of 577 services (393 services are lazy, passive or on-demand)
2022-03-29 11:33:37,857 INFO  [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal
2022-03-29 11:33:37,862 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow HTTPS listener https suspending
2022-03-29 11:33:37,863 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 12.345.678.901:8443
2022-03-29 11:33:37,874 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
2022-03-29 11:33:37,983 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0019: Host default-host stopping
2022-03-29 11:33:37,984 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2
2022-03-29 11:33:37,986 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow HTTP listener default suspending
2022-03-29 11:33:37,988 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 12.345.678.901:8080
2022-03-29 11:33:37,989 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0004: Undertow 1.4.0.Final stopping
2022-03-29 11:33:37,996 INFO  [org.jboss.as] (MSC service thread 1-1) WFLYSRV0050: WildFly Full 10.1.0.Final (WildFly Core 2.2.0.Final) stopped in 62ms

I then googled for "[org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal", and found this thread: https://knowledge.broadcom.com/external/article/196276/wildfly-will-not-start.html

(Re-)Applying the ownership of the wildfly tree solved my problem:

chown -R wildfly:wildfly "/opt/wildfly"

Thanks for the tip!

[Darran Lofthouse]

FYI that version of WildFly was released in August 2016 - you are missing out on almost 6 years of bug and security fixes.

[Stijn Bousard]

-- EDIT --
Looking back on my changes, I noticed that the following setting also needs to be changed for WildFly to run on my system (don't know if it makes any difference for running it manually and/or as a service):

In /etc/systemd/system.conf, add the following:

DefaultTasksMax=infinity

(source: https://stackoverflow.com/questions/16789288/java-lang-outofmemoryerror-unable-to-create-new-native-thread)

Op dinsdag 29 maart 2022 om 14:22:42 UTC+2 schreef Darran Lofthouse:

[emran sayed]

Hi

In error file getting out of memory pls check space once in your system once

Regards

Emran sayed

--
You received this message because you are subscribed to the Google Groups "WildFly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wildfly+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/4ce1ec61-3883-40a0-a215-2fbdfafad5f5n%40googlegroups.com.

[emran sayed]

And increase JVM size also

On Fri, 25 Mar, 2022, 7:31 pm Stijn Bousard, <gm...@boossy.be> wrote:

[Stijn Bousard]

-- EDIT2 --

I don't know where I copied my JAVA_OPTS line from, but the correct one is the following:

   JAVA_OPTS="-Xms384m -Xmx512m -XX:MetaspaceSize=128M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true"

Op dinsdag 29 maart 2022 om 14:51:54 UTC+2 schreef sayede...@gmail.com:

[emran sayed]

You can try xmx and xms 512m and restart once

To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/f858db44-3374-4e49-a738-7e67423b4b63n%40googlegroups.com.