Unable to deploy Webservices War to Wildfly 29 Final

[Samson Palkar]

We are migrating from JBOSS-4.3.2.GA to Wildfly 29 Final and are encountering a myriad of issues, most of them self inflicted quite possibly, however any help in the transition would be helpful. Below are the details.

• Using Java 17
• Using Wildfly 29
• Using Jakarta and not Javax
• Using CXF version 4.0.3 (Using the default provided jars, and upgraded cxf*-http*4.0.3.jar)
• Using Spring version 6.0.12
• Using jboss-structure.xml 
• All other existing default modules and jars from wildfly version 29

We have a war which contains webservices (cannot disclose those) however when we deploy to the Wildfly container we see the following exception.

C:\wildfly-29.0.1.Final\modules,C:\wildfly-29.0.1.Final\modules\system\layers\base))
2023-10-12 14:42:12,231 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-7) MSC000001: Failed to start service jboss.deployment.unit."analytics-ws.war".INSTALL: org.jboss.msc.service.StartException in service jboss.deployment.unit."analytics-ws.war".INSTALL: WFLYSRV0153: Failed to process phase INSTALL of deployment "analytics-ws.war"
at org.jboss...@21.1.1.Final//org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:189)
at org.jb...@1.5.1.Final//org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1617)
at org.jb...@1.5.1.Final//org.jboss.msc.service.ServiceControllerImpl$StartTask.execute(ServiceControllerImpl.java:1580)
at org.jb...@1.5.1.Final//org.jboss.msc.service.ServiceControllerImpl$ControllerTask.run(ServiceControllerImpl.java:1438)
at org.jbos...@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1363)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.ServiceConfigurationError: jakarta.servlet.ServletContainerInitializer: com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer not a subtype
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1244)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
at org.wildfly.ext...@29.0.1.Final//org.wildfly.extension.undertow.deployment.ServletContainerInitializerDeploymentProcessor.deploy(ServletContainerInitializerDeploymentProcessor.java:117)
at org.jboss...@21.1.1.Final//org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:182)
... 8 more


2023-10-12 14:42:12,245 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "analytics-ws.war")]) - failure description: {
    "WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"analytics-ws.war\".INSTALL" => "WFLYSRV0153: Failed to process phase INSTALL of deployment \"analytics-ws.war\"
    Caused by: java.util.ServiceConfigurationError: jakarta.servlet.ServletContainerInitializer: com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer not a subtype"},
    "WFLYCTL0412: Required services that are not installed:" => [
        "jboss.deployment.unit.\"analytics-ws.war\".WeldStartService",
        "jboss.deployment.unit.\"analytics-ws.war\".beanmanager"
    ],
    "WFLYCTL0180: Services with missing/unavailable dependencies" => [
        "jboss.deployment.unit.\"analytics-ws.war\".batch.artifact.factory is missing [jboss.deployment.unit.\"analytics-ws.war\".beanmanager]",
        "jboss.deployment.unit.\"analytics-ws.war\".weld.weldClassIntrospector is missing [jboss.deployment.unit.\"analytics-ws.war\".WeldStartService, jboss.deployment.unit.\"analytics-ws.war\".beanmanager]"
    ]
}

Contents on the spring module.xml as follows.

<module name="org.springframework.spring" xmlns="urn:jboss:module:1.9">
<resources>
<resource-root path="spring-jdbc-6.0.12.jar" />
<resource-root path="spring-aop-6.0.12.jar" />
<resource-root path="spring-jms-6.0.12.jar" />
<resource-root path="spring-beans-6.0.12.jar" />
<resource-root path="spring-ldap-core-3.1.1.jar" />
<resource-root path="spring-context-6.0.12.jar" />
<resource-root path="spring-messaging-6.0.12.jar" />
<resource-root path="spring-core-6.0.12.jar" />
<resource-root path="spring-security-core-6.1.4.jar" />
<resource-root path="spring-security-crypto-6.1.4.jar" />
<resource-root path="spring-expression-6.0.12.jar" />
<resource-root path="spring-tx-6.0.12.jar" />
<resource-root path="spring-jcl-6.0.12.jar" />
<resource-root path="spring-web-6.0.12.jar" />
</resources>
<dependencies>
<module name="jakarta.servlet.api" export="true" />
<module name="javaee.api" export="true"/>
<module name="org.jboss.vfs" export="true"/>
<module name="jakarta.el.api" export="true"/>
<module name="com.sun.xml.bind" export="true"/>
<module name="sun.jdk" export="true"/>
</dependencies>
</module>

Thank you all very much for all the help in advance!!!

[James Perkins]

Are you by chance including the jaxws-rt library in your deployment by chance? Asking because com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer is not in a WildFly dependency that I can see.

[Samson P]

Hello,

Thank you for your response. I am including the resource in a module I created as such 

<resource-root path="jaxws-rt-4.0.1.jar" />

However that is the only place it is referenced.

[James Perkins]

My guess is that's the issue. Do you have a hard dependency on that library? What I mean is, does your application use any types from it? If not, I would try removing that dependency.

[Samson P]

Hello,

We do not have a hard dependency on  jaxws-rt, however regardless of if I include it or not the exception still persists, which is 

Caused by: java.util.ServiceConfigurationError: jakarta.servlet.ServletContainerInitializer: com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer not a subtype

I believe the origin of the exception to be the jakarta.servlet-api-6.0.0.jar, but cannot narrow it down. 

[James Perkins]

Do you include a different servlet-api dependency by chance? Possibly in the module that includes the jaxws-rt library? I know in WildFly Apache CXF is used, however I'm not an expert in this area. From a grep and search of the source/dependencies I don't see com.sun.xml.ws.transport.http.servlet.WSServletContainerInitializer anywhere in WildFly.

[Samson P]

Here are the other jars included in the module I provided for the war. This is our custom module.

<resources>
<resource-root path="FastInfoset-2.1.0.jar" />
<resource-root path="HikariCP-java7-2.4.13.jar" />
<resource-root path="activation-1.1.jar" />
<resource-root path="activemq-broker-5.16.0.jar" />
<resource-root path="activemq-client-5.16.0.jar" />
<resource-root path="activemq-jms-pool-5.16.0.jar" />
<resource-root path="activemq-openwire-legacy-5.16.0.jar" />
<resource-root path="activemq-pool-5.16.0.jar" />
<resource-root path="activemq-spring-5.16.0.jar" />
<resource-root path="angus-activation-2.0.1.jar" />
<resource-root path="angus-mail-2.0.1.jar" />
<resource-root path="ant-1.8.2.jar" />
<resource-root path="ant-launcher-1.8.2.jar" />
<resource-root path="antisamy-1.5.8.jar" />
<resource-root path="antlr-2.7.7.jar" />
<resource-root path="antlr-runtime-3.5.2.jar" />
<resource-root path="asm-9.4.jar" />
<resource-root path="asm-debug-all-5.0.1.jar" />
<resource-root path="avalon-framework-api-4.3.1.jar" />
<resource-root path="avalon-framework-impl-4.3.1.jar" />
<resource-root path="batik-constants-1.11.jar" />
<resource-root path="batik-css-1.11.jar" />
<resource-root path="batik-i18n-1.11.jar" />
<resource-root path="batik-util-1.11.jar" />
<resource-root path="bcprov-jdk15on-1.67.jar" />
<resource-root path="bsh-2.0b6.jar" />
<resource-root path="c3p0-0.9.5.4.jar" />
<resource-root path="cewolf-1.1.2.jar" />
<resource-root path="checker-qual-3.33.0.jar" />
<resource-root path="commons-beanutils-1.9.4.jar" />
<resource-root path="commons-cli-1.1.jar" />
<resource-root path="commons-codec-1.12.jar" />
<resource-root path="commons-collections-3.2.2.jar" />
<resource-root path="commons-collections4-4.4.jar" />
<resource-root path="commons-compress-1.18.jar" />
<resource-root path="commons-configuration-1.10.jar" />
<resource-root path="commons-fileupload-1.3.3.jar" />
<resource-root path="commons-io-2.8.0.jar" />
<resource-root path="commons-lang-2.6.jar" />
<resource-root path="commons-lang3-3.11.jar" />
<resource-root path="commons-logging-1.2.jar" />
<resource-root path="commons-math3-3.6.1.jar" />
<resource-root path="commons-pool2-2.8.0.jar" />
<resource-root path="commons-text-1.8.jar" />
<resource-root path="cryptacular-1.1.1.jar" />
<resource-root path="curvesapi-1.06.jar" />
<resource-root path="docx4j-6.1.2.jar" />
<resource-root path="docx4j-ImportXHTML-8.2.1.jar" />
<resource-root path="docx4j-JAXB-ReferenceImpl-8.2.9.jar" />
<resource-root path="docx4j-core-8.2.9.jar" />
<resource-root path="docx4j-openxml-objects-8.2.9.jar" />
<resource-root path="docx4j-openxml-objects-pml-8.2.9.jar" />
<resource-root path="docx4j-openxml-objects-sml-8.2.9.jar" />
<resource-root path="ehcache-2.10.6.jar" />
<resource-root path="error_prone_annotations-2.18.0.jar" />
<resource-root path="esapi-2.2.0.0.jar" />
<resource-root path="failureaccess-1.0.1.jar" />
<resource-root path="fluent-hc-4.2.1.jar" />
<resource-root path="fontbox-2.0.23.jar" />
<resource-root path="geronimo-j2ee-management_1.1_spec-1.0.1.jar" />
<resource-root path="geronimo-javamail_1.4_mail-1.8.4.jar" />
<resource-root path="geronimo-jms_1.1_spec-1.1.1.jar" />
<resource-root path="geronimo-jta_1.1_spec-1.1.1.jar" />
<resource-root path="gmbal-api-only-4.0.3.jar" />
<resource-root path="graphics2d-0.30.jar" />
<resource-root path="guava-32.1.2-jre.jar" />
<resource-root path="hawtbuf-1.11.jar" />
<resource-root path="hazelcast-4.0.1.jar" />
<resource-root path="http2-client-9.4.27.v20200227.jar" />
<resource-root path="http2-common-9.4.27.v20200227.jar" />
<resource-root path="http2-hpack-9.4.27.v20200227.jar" />
<resource-root path="http2-http-client-transport-9.4.27.v20200227.jar" />
<resource-root path="httpasyncclient-4.1.2.jar" />
<resource-root path="httpclient-4.5.12.jar" />
<resource-root path="httpclient-cache-4.5.6.jar" />
<resource-root path="httpcore-4.4.13.jar" />
<resource-root path="httpcore-nio-4.4.5.jar" />
<resource-root path="httpmime-4.5.12.jar" />
<resource-root path="istack-commons-runtime-4.0.1.jar" />
<resource-root path="istack-commons-tools-2.7.jar" />
<resource-root path="itext-2.1.7.jar" />
<resource-root path="itextpdf-5.5.13.2.jar" />
<resource-root path="jackson-annotations-2.11.3.jar" />
<resource-root path="jackson-core-2.11.3.jar" />
<resource-root path="jackson-databind-2.11.3.jar" />
<resource-root path="jakarta.activation-2.0.1.jar" />
<resource-root path="jakarta.activation-api-2.1.2.jar" />
<resource-root path="jakarta.annotation-api-2.1.1.jar" />
<resource-root path="jakarta.jws-api-3.0.0.jar" />
<resource-root path="jakarta.mail-api-2.1.1.jar" />
<resource-root path="jakarta.servlet-api-6.0.0.jar" />
<resource-root path="jakarta.xml.bind-api-4.0.1.jar" />
<resource-root path="jakarta.xml.soap-api-3.0.0.jar" />
<resource-root path="jakarta.xml.ws-api-4.0.0.jar" />
<resource-root path="jasypt-1.9.3.jar" />
<resource-root path="java-http-client-4.3.6.jar" />
<resource-root path="java-support-7.3.0.jar" />
<resource-root path="java-uuid-generator-4.0.1.jar" />
<resource-root path="javax.activation-1.2.0.jar" />
<resource-root path="jaxb-core-4.0.3.jar" />
<resource-root path="jaxb-impl-4.0.3.jar" />
<resource-root path="jaxb-jxc-4.0.2.jar" />
<resource-root path="jaxb-runtime-3.0.2.jar" />
<resource-root path="jaxb-svg11-1.0.2.jar" />
<resource-root path="jaxb-xjc-4.0.2.jar" />
<resource-root path="jaxws-rt-4.0.1.jar" />
<resource-root path="jaxws-tools-4.0.1.jar" />
<resource-root path="jboss-common-core-2.5.0.Final.jar" />
<resource-root path="jboss-logging-3.4.1.Final.jar" />
<resource-root path="jboss-logging-spi-2.1.2.GA.jar" />
<resource-root path="jcl-over-slf4j-1.7.26.jar" />
<resource-root path="jcommon-1.0.16.jar" />
<resource-root path="jdom-1.1.jar" />
<resource-root path="jetty-alpn-client-9.4.27.v20200227.jar" />
<resource-root path="jetty-alpn-java-client-9.4.27.v20200227.jar" />
<resource-root path="jetty-client-9.4.27.v20200227.jar" />
<resource-root path="jetty-http-9.4.27.v20200227.jar" />
<resource-root path="jetty-io-9.4.27.v20200227.jar" />
<resource-root path="jetty-util-9.4.27.v20200227.jar" />
<resource-root path="jfreechart-1.0.8a.jar" />
<resource-root path="jjwt-api-0.10.5.jar" />
<resource-root path="jjwt-impl-0.10.5.jar" />
<resource-root path="jjwt-jackson-0.10.5.jar" />
<resource-root path="jms-1.1.jar" />
<resource-root path="jnpserver-4.2.2.GA.jar" />
<resource-root path="joda-time-2.10.4.jar" />
<resource-root path="jpwgen-1.0.3.jar" />
<resource-root path="jsoup-1.16.1.jar" />
<resource-root path="jsr305-3.0.2.jar" />
<resource-root path="listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar" />
<resource-root path="log4j-1.2.17.jar" />
<resource-root path="log4j-api-2.17.1.jar" />
<resource-root path="log4j-core-2.17.1.jar" />
<resource-root path="log4j-slf4j-impl-2.17.1.jar" />
<resource-root path="logback-classic-1.2.3.jar" />
<resource-root path="logback-core-1.2.3.jar" />
<resource-root path="lorem-2.1.jar" />
<resource-root path="mail-1.4.3.jar" />
<resource-root path="management-api-3.2.3.jar" />
<resource-root path="mbassador-1.3.2.jar" />
<resource-root path="mchange-commons-java-0.2.15.jar" />
<resource-root path="metrics-core-3.1.2.jar" />
<resource-root path="micrometer-commons-1.10.11.jar" />
<resource-root path="micrometer-observation-1.10.11.jar" />
<resource-root path="mimepull-1.10.0.jar" />
<resource-root path="neethi-3.2.0.jar" />
<resource-root path="nekohtml-1.9.22.jar" />
<resource-root path="netty-buffer-4.1.54.Final.jar" />
<resource-root path="netty-codec-4.1.54.Final.jar" />
<resource-root path="netty-codec-dns-4.1.54.Final.jar" />
<resource-root path="netty-common-4.1.54.Final.jar" />
<resource-root path="netty-handler-4.1.54.Final.jar" />
<resource-root path="netty-resolver-4.1.54.Final.jar" />
<resource-root path="netty-resolver-dns-4.1.54.Final.jar" />
<resource-root path="netty-transport-4.1.54.Final.jar" />
<resource-root path="netty-transport-native-epoll-4.1.50.Final.jar" />
<resource-root path="netty-transport-native-unix-common-4.1.50.Final.jar" />
<resource-root path="okhttp-3.11.0.jar" />
<resource-root path="okio-1.14.0.jar" />
<resource-root path="okta-authn-sdk-api-1.0.0.jar" />
<resource-root path="okta-authn-sdk-impl-1.0.0.jar" />
<resource-root path="okta-commons-lang-1.1.1.jar" />
<resource-root path="okta-config-check-1.1.1.jar" />
<resource-root path="okta-jwt-verifier-0.4.0.jar" />
<resource-root path="okta-jwt-verifier-impl-0.4.0.jar" />
<resource-root path="okta-sdk-api-1.5.2.jar" />
<resource-root path="okta-sdk-httpclient-1.5.2.jar" />
<resource-root path="okta-sdk-impl-1.5.2.jar" />
<resource-root path="opencsv-5.2.jar" />
<resource-root path="openhtmltopdf-core-1.0.8.jar" />
<resource-root path="openhtmltopdf-pdfbox-1.0.8.jar" />
<resource-root path="opensaml-2.5.1-1.jar" />
<resource-root path="opensaml-core-3.3.0.jar" />
<resource-root path="opensaml-messaging-api-3.3.0.jar" />
<resource-root path="opensaml-profile-api-3.3.0.jar" />
<resource-root path="opensaml-saml-api-3.3.0.jar" />
<resource-root path="opensaml-saml-impl-3.3.0.jar" />
<resource-root path="opensaml-security-api-3.3.0.jar" />
<resource-root path="opensaml-security-impl-3.3.0.jar" />
<resource-root path="opensaml-soap-api-3.3.0.jar" />
<resource-root path="opensaml-soap-impl-3.3.0.jar" />
<resource-root path="opensaml-storage-api-3.3.0.jar" />
<resource-root path="opensaml-xacml-api-3.3.0.jar" />
<resource-root path="opensaml-xacml-impl-3.3.0.jar" />
<resource-root path="opensaml-xacml-saml-api-3.3.0.jar" />
<resource-root path="opensaml-xacml-saml-impl-3.3.0.jar" />
<resource-root path="opensaml-xmlsec-api-3.3.0.jar" />
<resource-root path="opensaml-xmlsec-impl-3.3.0.jar" />
<resource-root path="openws-1.4.2-1.jar" />
<resource-root path="org.jacoco.agent-0.7.1.201405082137.jar" />
<resource-root path="org.jacoco.ant-0.7.1.201405082137.jar" />
<resource-root path="org.jacoco.core-0.7.1.201405082137.jar" />
<resource-root path="org.jacoco.report-0.7.1.201405082137.jar" />
<resource-root path="pdfbox-2.0.23.jar" />
<resource-root path="poi-4.1.0.jar" />
<resource-root path="poi-excelant-4.1.0.jar" />
<resource-root path="poi-ooxml-4.1.0.jar" />
<resource-root path="poi-ooxml-schemas-4.1.0.jar" />
<resource-root path="poi-scratchpad-4.1.0.jar" />
<resource-root path="postgresql-42.3.4.jar" />
<resource-root path="quartz-1.8.6.jar" />
<resource-root path="quartz-2.3.2.jar" />
<resource-root path="saaj-impl-3.0.0.jar" />
<resource-root path="sendgrid-java-4.7.1.jar" />
<resource-root path="serializer-2.7.2.jar" />
<resource-root path="slf4j-api-1.7.36.jar" />
<resource-root path="slf4j-log4j12-1.7.25.jar" />
<resource-root path="snakeyaml-1.17.jar" />
<resource-root path="snappy-java-1.1.7.6.jar" />
<resource-root path="solr-solrj-8.7.0.jar" />
<resource-root path="stax-ex-2.1.0.jar" />
<resource-root path="stax2-api-4.2.1.jar" />
<resource-root path="streambuffer-2.1.0.jar" />
<resource-root path="stringtemplate-3.2.1.jar" />
<resource-root path="txw2-3.0.2.jar" />
<resource-root path="velocity-1.7.jar" />
<resource-root path="wmf2svg-0.9.8.jar" />
<resource-root path="woodstox-core-6.5.1.jar" />
<resource-root path="woodstox-core-asl-4.4.1.jar" />
<resource-root path="wsdl4j-1.6.3.jar" />
<resource-root path="wss4j-1.6.19.jar" />
<resource-root path="wss4j-ws-security-common-2.2.4.jar" />
<resource-root path="wss4j-ws-security-dom-2.2.4.jar" />
<resource-root path="xalan-2.7.2.jar" />
<resource-root path="xalan-interpretive-8.0.0.jar" />
<resource-root path="xalan-metainf-8.0.0.jar" />
<resource-root path="xalan-serializer-8.0.0.jar" />
<resource-root path="xbean-spring-4.17.jar" />
<resource-root path="xercesImpl-2.12.0.jar" />
<resource-root path="xhtmlrenderer-3.0.0.jar" />
<resource-root path="xml-apis-1.4.01.jar" />
<resource-root path="xml-apis-ext-1.3.04.jar" />
<resource-root path="xml-resolver-1.2.jar" />
<resource-root path="xmlbeans-3.1.0.jar" />
<resource-root path="xmlgraphics-commons-2.3.jar" />
<resource-root path="xmlschema-core-2.3.1.jar" />
<resource-root path="xmlsec-2.1.4.jar" />
<resource-root path="xmltooling-1.3.2-1.jar" />
<resource-root path="xmpbox-2.0.23.jar" />
<resource-root path="xom-1.2.10.jar" />
<resource-root path="zookeeper-3.6.2.jar" />
<resource-root path="zookeeper-jute-3.6.2.jar" />
<resource-root path="servlet-4.0.1.jar" />
</resources>
<dependencies>
<module name="java.logging"/>
<module name="org.springframework.spring" services="import">
<imports>
                <include path="META-INF"/>
            </imports>
</module>
<module name="org.postgresql" />
</dependencies>

So these jars, the spring jars and default jars by Wildfly.

[James Perkins]

Okay, that is the problem then. You're include several libraries the server provides. In this specific case the jakarta.servlet-api-6.0.0.jar which is resulting in jakarta.servlet.ServletContainerInitializer being loaded by two different class loaders effectively not making it the same type. There are likely several libraries there hat should be removed and replaced with module dependencies.

[Samson P]

James, 

Thank you!! that worked. Thank you for the help!!!!!

[James Perkins]

Excellent! Not a problem at all.