Session replication only on demand?

[David Baddeley]

I have a JSF based app deployed to Wildfly 26 standalone-ha cluster, I am noticing that the HTTP session changes are constantly being written to the distributed cache causing a performance hit.

I would like to implement the session transfer from one Wildfly node to another to allow seamless shutdown/startup of a new node. However I do not want the memory/performance overhead of constantly updating a distributed cache.

Is it possible to configure Wildfly/Infinispan to only perform session replication when a server shutdown is detected, or perhaps trigger it programatically from within the App?

Thanks

[Paul Ferraro]

The default behavior of the distributed session manager is to be able to support failover of an HttpSession in the event of a JVM crash.  It sounds like your application has much more relaxed availability requirements than this.  Perhaps you can elaborate on what those requirements are?

If you only require that HttpSession state become available to another cluster member following shutdown, then I would suggest using an invalidation-cache with a passivation-only shared store.  This avoids any persistence of HttpSession state until the JVM is shutdown, or if the max-active-session threshold was reached.  Because the passivation store is shared, its state is available to any newly started cluster member.  Does that better fit your use case?

[David Baddeley]

We don't have any HA requirements for the application per se, however since its a stateful App we do not want users to be kicked out during server change, this is our main requirement.

The HTTP session footprint can be quite large for some specific users so I believe maintaining a live distributed cache will be a large memory burden, it also seems that, during initial testing, serialisation of HTTP session attributes 'on the fly' can slow things down so I was wondering if its possible to do this lazily? Although I have not done any actual benchmarking.

I will look at your suggested solution today thanks, is it possible to operate this as a disk-cache to avoid additional JVM memory consumption?

[Paul Ferraro]

In that case, just use the default configuration from standalone.xml (at least for the configuration of the distributable-web and infinispan subsystems).  This will do exactly what you want.  HttpSession data will only passivate to disk on shutdown/undeploy and will restore from disk on startup/redeploy.

Additionally, older sessions in excess of the configured limit can also passivate to disk, which can mitigate your memory burden.

[David Baddeley]

Hi Paul,

Sorry for the delayed reply, I am now testing your suggested configuration but I am getting some new exceptions in the application during failover, I am running two WF 26 nodes, 1 and 2. 

If login on Node 1 to create some session data, then shutdown Node 1, when I refresh the page I get the follow exception at Node 2....

Seems to be a CDI issue with serialization?

[0m [31m11:31:47,479 ERROR [io.undertow.servlet.request] (default task-2) UT015005: Error invoking method requestInitialized on listener class org.jboss.weld.module.web.servlet.WeldInitialListener: org.jboss.weld.exceptions.UnproxyableResolutionException: WELD-001435: Normal scoped bean class com.arjuna.ats.jta.cdi.DelegatingTransactionManager is not proxyable because it has no no-args constructor - Managed Bean [class com.arjuna.ats.jta.cdi.NarayanaTransactionManager] with qualifiers [@Any @Default].

   at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.DefaultProxyInstantiator.validateNoargConstructor(DefaultProxyInstantiator.java:50)

   at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableClassException(Proxies.java:249)

   at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypeException(Proxies.java:199)

   at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypesExceptionInt(Proxies.java:211)

   at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypesException(Proxies.java:190)

   at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.ClientProxyProvider.getClientProxy(ClientProxyProvider.java:213)

   at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.util.SerializableClientProxy.readResolve(SerializableClientProxy.java:62)

   at org.jboss....@2.0.12.Final//org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadResolve(JDKSpecific.java:145)

   at org.jboss....@2.0.12.Final//org.jboss.marshalling.reflect.SerializableClass.callReadResolve(SerializableClass.java:279)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1422)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:246)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1879)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1793)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1741)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1421)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:246)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1879)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1793)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1421)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:231)

   at org.jboss....@2.0.12.Final//org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.util.CollectionExternalizer.readObject(CollectionExternalizer.java:76)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.util.CollectionExternalizer.readObject(CollectionExternalizer.java:41)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.ExternalizerProvider.readObject(ExternalizerProvider.java:46)

   at org.wildfly.clusteri...@26.1.1.Final//org.wildfly.clustering.marshalling.jboss.ExternalizerObjectTable.readObject(ExternalizerObjectTable.java:125)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:376)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:231)

   at org.jboss....@2.0.12.Final//org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.DecoratorExternalizer.readObject(DecoratorExternalizer.java:94)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.ExternalizerProvider.readObject(ExternalizerProvider.java:46)

   at org.wildfly.clusteri...@26.1.1.Final//org.wildfly.clustering.marshalling.jboss.ExternalizerObjectTable.readObject(ExternalizerObjectTable.java:125)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:376)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:246)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1879)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1793)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1421)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:246)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1879)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1793)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1421)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:231)

   at org.jboss....@2.0.12.Final//org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.util.MapExternalizer.readObject(MapExternalizer.java:76)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.util.MapExternalizer.readObject(MapExternalizer.java:40)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.ExternalizerProvider.readObject(ExternalizerProvider.java:46)

   at org.wildfly.clusteri...@26.1.1.Final//org.wildfly.clustering.marshalling.jboss.ExternalizerObjectTable.readObject(ExternalizerObjectTable.java:125)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:376)

   at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:231)

   at org.jboss....@2.0.12.Final//org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)

   at org.wildfly.clusteri...@26.1.1.Final//org.wildfly.clustering.marshalling.jboss.JBossByteBufferMarshaller.readFrom(JBossByteBufferMarshaller.java:71)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.ByteBufferMarshaller.read(ByteBufferMarshaller.java:47)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.ByteBufferMarshalledValue.get(ByteBufferMarshalledValue.java:77)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.ByteBufferMarshalledValue.get(ByteBufferMarshalledValue.java:35)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.MarshalledValueMarshaller.read(MarshalledValueMarshaller.java:40)

   at org.wildfly.cluster...@26.1.1.Final//org.wildfly.clustering.marshalling.spi.MarshalledValueMarshaller.read(MarshalledValueMarshaller.java:30)

   at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.getValue(CoarseSessionAttributesFactory.java:135)

   at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.findValue(CoarseSessionAttributesFactory.java:123)

   at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.findValue(CoarseSessionAttributesFactory.java:61)

   at org.wildfly.clus...@26.1.1.Final//org.wildfly.clustering.web.cache.session.CompositeSessionFactory.findValue(CompositeSessionFactory.java:65)

   at org.wildfly.clus...@26.1.1.Final//org.wildfly.clustering.web.cache.session.CompositeSessionFactory.findValue(CompositeSessionFactory.java:40)

   at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.findSession(InfinispanSessionManager.java:176)

   at org.wildfly.clus...@26.1.1.Final//org.wildfly.clustering.web.cache.session.ConcurrentSessionManager$2.apply(ConcurrentSessionManager.java:67)

   at org.wildfly.clus...@26.1.1.Final//org.wildfly.clustering.web.cache.session.ConcurrentSessionManager$2.apply(ConcurrentSessionManager.java:64)

   at org.wildfly.clu...@26.1.1.Final//org.wildfly.clustering.ee.cache.SimpleManager.apply(SimpleManager.java:52)

   at org.wildfly.clus...@26.1.1.Final//org.wildfly.clustering.web.cache.session.ConcurrentSessionManager.findSession(ConcurrentSessionManager.java:72)

   at org.wildfly.clust...@26.1.1.Final//org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:236)

   at io.undert...@2.2.17.Final//io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:903)

   at io.undert...@2.2.17.Final//io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:425)

   at org.jboss...@3.1.9.Final//org.jboss.weld.module.web.servlet.SessionHolder.requestInitialized(SessionHolder.java:47)

   at org.jboss...@3.1.9.Final//org.jboss.weld.module.web.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:247)

   at org.jboss...@3.1.9.Final//org.jboss.weld.module.web.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:146)

   at io.undert...@2.2.17.Final//io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:263)

   at io.undert...@2.2.17.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:274)

   at io.undert...@2.2.17.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79)

   at io.undert...@2.2.17.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134)

   at io.undert...@2.2.17.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131)

   at io.undert...@2.2.17.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)

   at io.undert...@2.2.17.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)

   at org.wildfly.ext...@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)

   at org.wildfly.ext...@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)

   at org.wildfly.ext...@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)

   at org.wildfly.ext...@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)

   at org.wildfly.ext...@26.1.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)

   at io.undert...@2.2.17.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255)

   at io.undert...@2.2.17.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79)

   at io.undert...@2.2.17.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100)

   at io.under...@2.2.17.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)

   at io.under...@2.2.17.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852)

   at org.jbos...@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)

   at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)

   at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)

   at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)

   at org.jbo...@3.8.7.Final//org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)

   at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: an exception which occurred:

   in object of type org.jboss.weld.bean.proxy.util.SerializableClientProxy

   in field com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorBase.transactionManager

   in object com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired@d18421

   in object of type com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired

   in field org.jboss.weld.contexts.SerializableContextualInstanceImpl.instance

   in object org.jboss.weld.contexts.SerializableContextualInstanceImpl@3ff68b3

   in object of type org.jboss.weld.contexts.SerializableContextualInstanceImpl

   in field org.jboss.weld.contexts.CreationalContextImpl.dependentInstances

   in object org.jboss.weld.contexts.CreationalContextImpl@7f069388

   in object of type org.jboss.weld.contexts.CreationalContextImpl

   in field org.jboss.weld.contexts.SerializableContextualInstanceImpl.creationalContext

   in object org.jboss.weld.contexts.SerializableContextualInstanceImpl@737af0c5

   in object of type org.jboss.weld.contexts.SerializableContextualInstanceImpl

[Paul Ferraro]

I assume that the bean in question injects a TransactionManager?  This is probably a bug, since the TransactionManager client proxy ought to be passivation capable.

You can probably workaround this by making your transaction manager transient, and manually looking it up during deserialization.

[David Baddeley]

The error stacktrace doesn't reference any of my own application code so I don't see how I can make these changes? As far as i am aware we are not dealing with a transaction manager anywhere within the app, we only inject the JPA entityManager (request scoped), which utilises container managed transactions.

Also going back to the original question, I tried your suggestion of using the standalone.xml config for session replication however this does not send session state to the other nodes, I would like sessions to be transferred to other nodes in the event of a server shutdown....can you let me know what the best setup would be? Invalidaton cache?

Thanks

D

[David Baddeley]

Hi Paul,

Looking at this further, I made a few changes to the application, notably lazy-loading the SecurityContext in a few places due to a bug in Soteria (Jakarta EE security API implementation) that made it non-serialisable.

This seems to have changed the exception I am getting during session de-serialisation (when the server starts up following a shutdown), the stack traces now references one of my @ViewScoped beans which has the @Transactional(Transactional.TxType.REQUIRED) annotation, could this be causing the TransactionManager client proxy to be added to the session data?

Can you provide more details on how I would go about making the Transaction Manager transient?

I have included the class below as well as the error stacktrace

Thanks

@Named
@ViewScoped
@Transactional(Transactional.TxType.REQUIRED)
public class UserDeviceManagement implements java.io.Serializable {

    /**
     *
     */
    private static final long serialVersionUID = 1L;

    @Inject private Logger log;
   

    @Inject private SessionDeviceDAO sessionDeviceDAO;
    @Inject private ApiClientUserDAO apiClientUserDAO;
   
    private List<Map<String,Object>> importDevices;
    private List<Map<String,Object>> mobileApDevicesStats;
   
    @RolesAllowed({"admin"})
    public void searchDevices() {
        importDevices = sessionDeviceDAO.getSessionDeviceModelUsage();
        mobileApDevicesStats = apiClientUserDAO.getMobileAppDeviceTypeCount();
    }
   
    @RolesAllowed({"clientUser"})
    public void searchDevicesForPool(Pool pool) {
        importDevices = sessionDeviceDAO.getSessionDeviceModelUsageForPool(pool);
    }

    public List<Map<String,Object>> getImportDevices() {
        return importDevices;
    }

    public void setImportDevices(List<Map<String,Object>> importDevices) {
        this.importDevices = importDevices;
    }

    public List<Map<String,Object>> getMobileApDevicesStats() {
        return mobileApDevicesStats;
    }

    public void setMobileApDevicesStats(List<Map<String,Object>> mobileApDevicesStats) {
        this.mobileApDevicesStats = mobileApDevicesStats;
    }
}

2022-09-13 11:54:35,026 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
2022-09-13 11:54:35,028 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 26.1.1.Final (WildFly Core 18.1.1.Final) started in 52010ms - Started 1857 of 2040 services (466 services are lazy, passive or on-demand) - Server configuration file in use: standalone-ha.xml
2022-09-13 11:54:35,031 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
2022-09-13 11:54:35,031 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
2022-09-13 11:54:41,120 INFO  [com.myapp.actions.server.CliActions] (EJB default - 1) Initialising CLI command context for localhost
2022-09-13 11:54:41,445 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (non-blocking-thread--p7-t10) ISPN000136: Error executing command GetKeyValueCommand on Cache 'MyAppWeb-web.war', writing keys []: org.jboss.weld.exceptions.UnproxyableResolutionException: WELD-001435: Normal scoped bean class com.arjuna.ats.jta.cdi.DelegatingTransactionManager is not proxyable because it has no no-args constructor - Managed Bean [class com.arjuna.ats.jta.cdi.NarayanaTransactionManager] with qualifiers [@Any @Default].

    at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.DefaultProxyInstantiator.validateNoargConstructor(DefaultProxyInstantiator.java:50)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableClassException(Proxies.java:249)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypeException(Proxies.java:199)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypesExceptionInt(Proxies.java:211)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypesException(Proxies.java:190)
    at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.ClientProxyProvider.getClientProxy(ClientProxyProvider.java:213)
    at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.util.SerializableClientProxy.readResolve(SerializableClientProxy.java:62)
    at org.jboss....@2.0.12.Final//org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadResolve(JDKSpecific.java:145)
    at org.jboss....@2.0.12.Final//org.jboss.marshalling.reflect.SerializableClass.callReadResolve(SerializableClass.java:279)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1422)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:246)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1879)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1793)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1741)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1421)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)

    at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.notify(CoarseSessionAttributesFactory.java:192)
    at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.postActivate(CoarseSessionAttributesFactory.java:186)
    at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.infinispan.spi.listener.PostActivateListener.lambda$activated$0(PostActivateListener.java:54)
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at org.jboss.as.cl...@26.1.1.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
    at org.jboss.as.cl...@26.1.1.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
    at java.base/java.lang.Thread.run(Thread.java:833)
    at org.jbos...@2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)

Caused by: an exception which occurred:
    in object of type org.jboss.weld.bean.proxy.util.SerializableClientProxy
    in field com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorBase.transactionManager

    in object com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired@317ab2ea

    in object of type com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired

    in field org.jboss.weld.interceptor.proxy.InterceptionContext.interceptorInstances
    in object org.jboss.weld.interceptor.proxy.InterceptionContext@63368b0d
    in object of type org.jboss.weld.interceptor.proxy.InterceptionContext
    in field org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.ctx
    in object org.jboss.weld.interceptor.proxy.InterceptorMethodHandler@7f25715c
    in object of type org.jboss.weld.interceptor.proxy.InterceptorMethodHandler
    in field org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.interceptorMethodHandler
    in object org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler@1bac71b0
    in object of type org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler
    in field com.myapp.actions.user.UserDeviceManagement$Proxy$_$$_WeldSubclass.methodHandler
    in object com.myapp.actions.user.UserDeviceManagement$Proxy$_$$_WeldSubclass@8553853
    in object of type com.myapp.actions.user.UserDeviceManagement$Proxy$_$$_WeldSubclass
    in field org.omnifaces.cdi.BeanStorage.beans
    in object org.omnifaces.cdi.BeanStorage@5f6cb4b7
    in object of type org.omnifaces.cdi.BeanStorage
    in field org.omnifaces.util.concurrentlinkedhashmap.ConcurrentLinkedHashMap$SerializationProxy.data
    in object org.omnifaces.util.concurrentlinkedhashmap.ConcurrentLinkedHashMap$SerializationProxy@b978197
    in object of type org.omnifaces.util.concurrentlinkedhashmap.ConcurrentLinkedHashMap$SerializationProxy
    in field org.omnifaces.cdi.viewscope.ViewScopeStorageInSession.activeViewScopes
    in object org.omnifaces.cdi.viewscope.ViewScopeStorageInSession@73ce50a4
    in object of type org.omnifaces.cdi.viewscope.ViewScopeStorageInSession
    in field org.jboss.weld.contexts.SerializableContextualInstanceImpl.instance
    in object org.jboss.weld.contexts.SerializableContextualInstanceImpl@52f8692e

    in object of type org.jboss.weld.contexts.SerializableContextualInstanceImpl

2022-09-13 11:54:41,451 ERROR [io.undertow.servlet.request] (default task-1) UT015005: Error invoking method requestInitialized on listener class org.jboss.weld.module.web.servlet.WeldInitialListener: org.infinispan.commons.CacheException: org.jboss.weld.exceptions.UnproxyableResolutionException: WELD-001435: Normal scoped bean class com.arjuna.ats.jta.cdi.DelegatingTransactionManager is not proxyable because it has no no-args constructor - Managed Bean [class com.arjuna.ats.jta.cdi.NarayanaTransactionManager] with qualifiers [@Any @Default].
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:135)
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.InvocationContextInterceptor.lambda$new$0(InvocationContextInterceptor.java:61)
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:25)
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:124)
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:87)
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:33)
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
    at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.CacheLoaderInterceptor.finishLoadInContext(CacheLoaderInterceptor.java:391)
    at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.CacheLoaderInterceptor.lambda$loadInContext$7(CacheLoaderInterceptor.java:382)
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
    at java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:614)
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:844)
    at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at org.jboss.as.cl...@26.1.1.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
    at org.jboss.as.cl...@26.1.1.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
    at java.base/java.lang.Thread.run(Thread.java:833)
    Suppressed: org.infinispan.commons.util.logging.TraceException
        at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:39)
        at org.inf...@13.0.10.Final//org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:249)
        at org.inf...@13.0.10.Final//org.infinispan.cache.impl.InvocationHelper.doInvoke(InvocationHelper.java:297)
        at org.inf...@13.0.10.Final//org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:101)
        at org.inf...@13.0.10.Final//org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:542)
        at org.inf...@13.0.10.Final//org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:536)
        at org.inf...@13.0.10.Final//org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:439)
        at org.inf...@13.0.10.Final//org.infinispan.cache.impl.EncoderCache.get(EncoderCache.java:683)
        at org.inf...@13.0.10.Final//org.infinispan.cache.impl.AbstractDelegatingCache.get(AbstractDelegatingCache.java:439)
        at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.getValue(CoarseSessionAttributesFactory.java:132)

        ... 1 more
Caused by: org.jboss.weld.exceptions.UnproxyableResolutionException: WELD-001435: Normal scoped bean class com.arjuna.ats.jta.cdi.DelegatingTransactionManager is not proxyable because it has no no-args constructor - Managed Bean [class com.arjuna.ats.jta.cdi.NarayanaTransactionManager] with qualifiers [@Any @Default].

    at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.DefaultProxyInstantiator.validateNoargConstructor(DefaultProxyInstantiator.java:50)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableClassException(Proxies.java:249)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypeException(Proxies.java:199)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypesExceptionInt(Proxies.java:211)
    at org.jboss...@3.1.9.Final//org.jboss.weld.util.Proxies.getUnproxyableTypesException(Proxies.java:190)
    at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.ClientProxyProvider.getClientProxy(ClientProxyProvider.java:213)
    at org.jboss...@3.1.9.Final//org.jboss.weld.bean.proxy.util.SerializableClientProxy.readResolve(SerializableClientProxy.java:62)
    at org.jboss....@2.0.12.Final//org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadResolve(JDKSpecific.java:145)
    at org.jboss....@2.0.12.Final//org.jboss.marshalling.reflect.SerializableClass.callReadResolve(SerializableClass.java:279)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1422)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:246)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1879)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1793)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1741)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1421)
    at org.jboss.mar...@2.0.12.Final//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:298)

    at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.notify(CoarseSessionAttributesFactory.java:192)
    at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionAttributesFactory.postActivate(CoarseSessionAttributesFactory.java:186)
    at org.wildfly.cluste...@26.1.1.Final//org.wildfly.clustering.infinispan.spi.listener.PostActivateListener.lambda$activated$0(PostActivateListener.java:54)
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at org.jboss.as.cl...@26.1.1.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
    at org.jboss.as.cl...@26.1.1.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
    at java.base/java.lang.Thread.run(Thread.java:833)
    at org.jbos...@2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)

Caused by: an exception which occurred:
    in object of type org.jboss.weld.bean.proxy.util.SerializableClientProxy
    in field com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorBase.transactionManager

    in object com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired@317ab2ea

    in object of type com.arjuna.ats.jta.cdi.transactional.TransactionalInterceptorRequired

    in field org.jboss.weld.interceptor.proxy.InterceptionContext.interceptorInstances
    in object org.jboss.weld.interceptor.proxy.InterceptionContext@63368b0d
    in object of type org.jboss.weld.interceptor.proxy.InterceptionContext
    in field org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.ctx
    in object org.jboss.weld.interceptor.proxy.InterceptorMethodHandler@7f25715c
    in object of type org.jboss.weld.interceptor.proxy.InterceptorMethodHandler
    in field org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.interceptorMethodHandler
    in object org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler@1bac71b0
    in object of type org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler
    in field com.myapp.actions.user.UserDeviceManagement$Proxy$_$$_WeldSubclass.methodHandler
    in object com.myapp.actions.user.UserDeviceManagement$Proxy$_$$_WeldSubclass@8553853
    in object of type com.myapp.actions.user.UserDeviceManagement$Proxy$_$$_WeldSubclass
    in field org.omnifaces.cdi.BeanStorage.beans
    in object org.omnifaces.cdi.BeanStorage@5f6cb4b7
    in object of type org.omnifaces.cdi.BeanStorage
    in field org.omnifaces.util.concurrentlinkedhashmap.ConcurrentLinkedHashMap$SerializationProxy.data
    in object org.omnifaces.util.concurrentlinkedhashmap.ConcurrentLinkedHashMap$SerializationProxy@b978197
    in object of type org.omnifaces.util.concurrentlinkedhashmap.ConcurrentLinkedHashMap$SerializationProxy
    in field org.omnifaces.cdi.viewscope.ViewScopeStorageInSession.activeViewScopes
    in object org.omnifaces.cdi.viewscope.ViewScopeStorageInSession@73ce50a4
    in object of type org.omnifaces.cdi.viewscope.ViewScopeStorageInSession
    in field org.jboss.weld.contexts.SerializableContextualInstanceImpl.instance
    in object org.jboss.weld.contexts.SerializableContextualInstanceImpl@52f8692e

    in object of type org.jboss.weld.contexts.SerializableContextualInstanceImpl

2022-09-13 11:54:41,453 WARN  [org.jboss.weld.Servlet] (default task-1) WELD-000717: Unable to deactivate context org.jboss.weld.module.web.context.http.HttpRequestContextImpl@681cb3c when destroying request HttpServletRequestImpl [ GET /client/index.xhtml ]

[Paul Ferraro]

On Saturday, September 10, 2022 at 5:20:33 PM UTC-4 da...@222sports.co.uk wrote:

The error stacktrace doesn't reference any of my own application code so I don't see how I can make these changes? As far as i am aware we are not dealing with a transaction manager anywhere within the app, we only inject the JPA entityManager (request scoped), which utilises container managed transactions.

 

Also going back to the original question, I tried your suggestion of using the standalone.xml config for session replication however this does not send session state to the other nodes, I would like sessions to be transferred to other nodes in the event of a server shutdown....can you let me know what the best setup would be? Invalidaton cache?

I suggested using standalone.xml to *prevent* session replication, since you indicated that your application did not want to incur the cost of replication, and since you only needed availability of sessions across server restarts.

The default behavior for <distributable/> web applications in standalone.xml persists sessions to disk on shutdown so that they can be loaded back into memory on startup.

[Paul Ferraro]

Are you using javax.faces.bean.ViewScoped or javax.faces.view.ViewScoped?  The former was deprecated in favor of the latter in JSF 2.2, which is implemented as a custom CDI scope with explicit serializable/proxyable requirements.

Otherwise, now that I see your bean code, I don't see any way to manipulate the serialization of the transaction manager client proxy without switching to bean managed transactions (using an injected TransactionManager).  Hopefully, that won't be necessary.

[David Baddeley]

Ok I see, what I really meant was that I want session state to be transferred to other node(s) in the cluster when a node is shutdown....what I DONT need was serialisation 'on-the-fly' eg. if a node crashes or hardware fails the state would be lost because serialisation is only triggered during graceful shutdown...hopefully that makes sense

I am currently testing the following config which is what I believe you suggested originally, does this seem to fit the requirement?

<cache-container name="session-replication" default-cache="invcache" marshaller="PROTOSTREAM" modules="org.wildfly.clustering.web.infinispan">
                <transport lock-timeout="60000"/>
                <invalidation-cache name="invcache">
                    <locking isolation="REPEATABLE_READ"/>
                    <transaction mode="BATCH"/>
                    <file-store fetch-state="false" passivation="true" purge="false" shared="false"/>
                </invalidation-cache>
            </cache-container>

[David Baddeley]

We're actually using Omnifaces @ViewScoped (org.omnifaces.cdi.ViewScoped) which is the equivalent of the CDI scope with a few extra features

I don't think using BMT's would be an option (would require way too much rework)...I assume there is something in our config that is causing the JTA TransactionManager to creep into the session somewhere

For what its worth, we are using @ApplicationScoped DAO beans that inject a JPA entity manager (using a request scoped CDI @Producer)

[Paul Ferraro]

OK - I understand now.  What you should use instead is a distributed-cache with owners=1.

Cache entries will not survive a crash, but they will be transferred to another cluster member on shutdown, which is what I think you want.

[David Baddeley]

Ok that makes sense, I am trying this setup now:

<cache-container name="session-replication" default-cache="sessions-dist" marshaller="PROTOSTREAM" modules="org.wildfly.clustering.web.infinispan">
                <transport lock-timeout="60000"/>
                <distributed-cache name="sessions-dist" owners="1">
                    <locking isolation="REPEATABLE_READ"/>
                    <transaction mode="NONE"/>
                    <expiration interval="0" lifespan="86400000" max-idle="7200000"/>
                    <file-store purge="false"/>
                </distributed-cache>
</cache-container>

with the following distributable-web config

<subsystem xmlns="urn:jboss:domain:distributable-web:2.0" default-session-management="default" default-single-sign-on-management="default">
            <infinispan-session-management name="default" cache-container="session-replication" cache="sessions-dist" granularity="SESSION">
                <primary-owner-affinity/>
            </infinispan-session-management>
            <infinispan-single-sign-on-management name="default" cache-container="web" cache="sso"/>
            <infinispan-routing cache-container="web" cache="routing"/>
        </subsystem>

But I get the following exception during startup:

09:04:13,048 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "MyAppWeb-web.war")]) - failure description: {
    "WFLYCTL0412: Required services that are not installed:" => [
        "org.wildfly.clustering.cache.group.session-replication.default-server",
        "org.wildfly.clustering.cache.registry.session-replication.default-server"
    ],
    "WFLYCTL0180: Services with missing/unavailable dependencies" => ["org.wildfly.clustering.web.route-locator.\"MyAppWeb-web.war\" is missing [org.wildfly.clustering.cache.group.session-replication.default-server, org.wildfly.clustering.cache.registry.session-replication.default-server]"]
}

Am I missing some additional config?

[Paul Ferraro]

Does your application contain a distributable-web.xml (or distributable-web namespace within jboss-all.xml)?  If so, can you attach?  You seem to have routing configuration somewhere that references a non-existent Infinispan cache.

[David Baddeley]

No I don't have any distributable-web.xml, only have the <distributable/> tag in web.xml and the following in jboss-web.xml

<?xml version="1.0" encoding="UTF-8"?>
<jboss-web>
    <virtual-host>myapp-webapp</virtual-host>
    <context-root>/</context-root>
    <max-active-sessions>200</max-active-sessions>
    <security-domain>myapp</security-domain>
</jboss-web>

[Paul Ferraro]

Hmm.  Actually "default-server" refers to the name of the Undertow server - for which <infinispan-routing/> *should* have installed the services that your server log indicated are missing.

Are you certain that this error was triggered while using infinispan-routing?  It sounds to me as if you had tried using primary-owner-affinity w/local routing, which would fail at runtime with a similar error.

Really, we should have the proper constraints in place to detect this at server startup (rather than at deployment time).
 

[David Baddeley]

Hmm ok, I haven't changed <primary-owner-affinity/> at all it's has always been there, I may have previously started the server with a <local-cache> configured if that's what you mean? But I'm running the server in a Docker container so it should be starting from a fresh image each time....

[Paul Ferraro]

I mean <infinispan-routing/> vs <local-routing/>.  If you attempt to use <primary-owner-affinity/> w/<local-routing/> you will get exceptions during deploy relating to missing services, installed on behalf of the Undertow server (named "default-server").

[David Baddeley]

Oh right, no I haven't changed the <infinispan-routing> at all, it still references the default 'web' cache

[Paul Ferraro]

Ah - I see the problem.

Make sure that your <infinispan-routing/> uses the same cache-container as your infinispan-session-management.

Unfortunately, I don't think there is currently a constraint in place that enforces this.

[David Baddeley]

Great thanks, i thought it would be something like that :-)

Could you spare some time to look at my remaining issue, with this exception after server failover...

com.arjuna.ats.jta.cdi.DelegatingTransactionManager is not proxyable because it has no no-args constructor

I found that it is caused by @Transactional(Transactional.TxType.REQUIRED) annotation in my CDI beans. I can see the the DelegatingTransactionManager does indeed not have a no args constructor, should I raise this as a bug? I feel like I am missing some configuration that allows the TransactionManager to be recreated successfully

[Paul Ferraro]

I suspect that this is a bug in Weld. There is no reason that
DelegatingTransactionManager should require a no-arg constructor,
since this is an abstract superclass of NarayanaTransactionManager,
which does have a no-arg constructor (and therefore, should be
proxyable).

> --
> You received this message because you are subscribed to a topic in the Google Groups "WildFly" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/wildfly/NWApe8dcvzQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to wildfly+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/3cf45b50-08c4-4a0e-ab6f-576014ef6291n%40googlegroups.com.

[David Baddeley]

Hmm ok, trying to find where com.arjuna.ats.jta.cdi is located in the wildly distribution, I can't see it in the modules anywhere?

I can try adding a no-args constructor to the DelegatingTransactionManager

[Paul Ferraro]

https://github.com/jbosstm/narayana/blob/5.12.5.Final/ArjunaJTA/cdi/classes/com/arjuna/ats/jta/cdi/NarayanaTransactionManager.java
Module is here: https://github.com/wildfly/wildfly/blob/26.1.2.Final/ee-feature-pack/common/src/main/resources/modules/system/layers/base/org/jboss/jts/main/module.xml

[David Baddeley]

Thanks Paul, I have built a new jar and replaced the provided one, the error is now no longer thrown, great, I will raise the issue with WELD.

However I am still not seeing the replicated sessions being used, when I refresh the browser following failover there is a long pause and I am redirected back to the login page due to no authenticated user.

I am currently testing this config for the replicated cache:

<distributed-cache name="dist" owners="1">
          <locking isolation="REPEATABLE_READ"/>
          <state-transfer timeout="0" />

          <transaction mode="NONE"/>
          <expiration interval="0" lifespan="86400000" max-idle="7200000"/>

          <file-store purge="false" passivation="true"/>
 </distributed-cache>

After failover, I see this in the new servers logs that shows the old server has left the cluster, it seems the the hibernate entity cache is being distributed, but no mention of the web session cache?

10:50:27,662 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p6-t5) [Context=MyApp-web.war] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,663 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p7-t3) [Context=http-remoting-connector] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,665 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p6-t6) [Context=default-server] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,665 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p6-t3) [Context=org.infinispan.CONFIG] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,666 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p5-t2) [Context=org.infinispan.CONFIG] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,662 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t5) [Context=MyApp-api.war#ProcessorDatabase.com.myapp.entities.Competition] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 6
10:50:27,683 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p7-t1) [Context=org.infinispan.CONFIG] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,689 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t5) [Context=MyApp-api.war#ProcessorDatabase.com.myapp.entities.Competition] ISPN100008: Updating cache members list [node-172.19.0.3], topology id 7
10:50:27,690 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t4) [Context=MyApp-web.war#ProcessorDatabase.com.myapp.entities.Competition] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 6
10:50:27,690 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t2) [Context=MyApp-web.war#MyAppDatabase.com.myapp.entities.Competition] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 6
10:50:27,692 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t5) [Context=MyApp-api.war#MyAppReadOnlyDatabase.com.myapp.entities.Competition] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 6
10:50:27,694 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t4) [Context=MyApp-web.war#ProcessorDatabase.com.myapp.entities.Competition] ISPN100008: Updating cache members list [node-172.19.0.3], topology id 7
10:50:27,694 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t4) [Context=MyApp-web.war#MyAppReadOnlyDatabase.default-update-timestamps-region] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,697 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t2) [Context=MyApp-web.war#MyAppDatabase.com.myapp.entities.Competition] ISPN100008: Updating cache members list [node-172.19.0.3], topology id 7
10:50:27,698 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t2) [Context=org.infinispan.CONFIG] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,700 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t4) [Context=MyApp-web.war#MyAppDatabase.default-update-timestamps-region] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,710 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t2) [Context=MyApp-api.war#MyAppDatabase.com.myapp.entities.Competition] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 6
10:50:27,720 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t4) [Context=MyApp-web.war#MyAppReadOnlyDatabase.com.myapp.entities.Competition] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 6
10:50:27,721 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t5) [Context=MyApp-api.war#MyAppReadOnlyDatabase.com.myapp.entities.Competition] ISPN100008: Updating cache members list [node-172.19.0.3], topology id 7
10:50:27,722 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t5) [Context=MyApp-api.war#MyAppReadOnlyDatabase.default-update-timestamps-region] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,737 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t5) [Context=MyApp-api.war#MyAppDatabase.default-update-timestamps-region] ISPN100007: After merge (or coordinator change), recovered members [node-172.19.0.3] with topology id 7
10:50:27,738 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t2) [Context=MyApp-api.war#MyAppDatabase.com.myapp.entities.Competition] ISPN100008: Updating cache members list [node-172.19.0.3], topology id 7
10:50:27,742 INFO  [org.infinispan.CLUSTER] (non-blocking-thread--p11-t4) [Context=MyApp-web.war#MyAppReadOnlyDatabase.com.myapp.entities.Competition] ISPN100008: Updating cache members list [node-172.19.0.3], topology id 7

[Paul Ferraro]

The first line in your log refers to the cache used for HttpSession
attributes/metadata. Unfortunately, Infinispan did not log the name
of the cache container along with the cache name, which makes these
log messages difficult to differentiate.

Re: your cache configuration - I have a few recommendations:
1. Remove your locking isolation. This would only be useful to
prevent concurrent access by multiple cluster members (and would
require transactions), and the distributed session manager ensures
consistent reads by multiple concurrent threads.
2. You can remove your expiration configuration, as this is not used.
Session expiration is configured via <session-timeout/> within web.xml
(or per session via HttpSession.setMaxInactiveInterval(...)).

Re: not seeing replicated sessions being used - are you certain this
is the case? How does your application configure authentication?

Paul

> To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/216d1d6e-b3b6-4d72-a6fa-0b4dc6ea4a92n%40googlegroups.com.

[David Baddeley]

Ok noted, I will update the cache settings

Re: not seeing replicated sessions being used - are you certain this 
is the case? How does your application configure authentication? 

We are using the standard Java EE 8 security API - implementing our own HttpAuthenticationMechanism for authentication, I have included the implementation below

When a page is reloaded this class checks if request.getUserPrincipal() is null, if it is then you are redirected to the login page, which is what is happening, so it seems that the UserPrincipal is not being restored.

I assumed that since we are using standard Jarkata EE API's that Wildfly would restore the SecurityContext etc. following session replication....maybe I am wrong?

/**
 * Custom Authentication class for validating requests
 *
 * @LoginToContinue is not implemented as it doesnt work well with auto-logins
 *
 * @author David
 *
 */
@RequestScoped
@AutoApplySession
public class MyAppAuthentication implements Serializable, HttpAuthenticationMechanism {

   
    /**
     *
     */
    private static final long serialVersionUID = 1L;
   

    @Inject private PasswordEncryptorEntities passwordEncryptor;
    @Inject private Logger log;

    /**
     * Note: this method is called for all requests (including public) to determine if authentication is required
     */
    @Override
    public AuthenticationStatus validateRequest(HttpServletRequest request, HttpServletResponse response,
            HttpMessageContext httpMessageContext) throws AuthenticationException {
       
        log.trace("Validating request {}",request.getRequestURI());
       
        if(httpMessageContext.isAuthenticationRequest()) {
           
            AppUser loginPrincipal = (AppUser) request.getAttribute("com.myapp.login.app.user");
            Boolean programmaticAuth = (Boolean) request.getAttribute("com.myapp.login.programmatic");
            String password = (String) request.getAttribute("com.myapp.login.password");
            WebModuleType module = (WebModuleType) request.getAttribute("com.myapp.login.module");
            Boolean app = (Boolean) request.getAttribute("com.myapp.login.app");
           
            Set<String> roles = new HashSet<String>();
            if (loginPrincipal!=null) {
                if(programmaticAuth || passwordEncryptor.isPasswordCorrect(password, loginPrincipal.getPassword())){
                   
                    if(module == null || !WebModuleType.API.equals(module)) {
                        //Don't add loggedIn role for api request.. just add roles
                        roles.add(Role.loggedIn.getRole());
                    }
                    roles.add(loginPrincipal.coreRole().getRole());
                   
                     if(loginPrincipal.coreRole().equals(Role.user)) {
                        User user = (User) loginPrincipal;
                    }
                    UserPrincipal up = new UserPrincipal(loginPrincipal);
                    httpMessageContext.getClientSubject()
                        .getPrincipals()
                        .add(up);
                    httpMessageContext.setRegisterSession(up.getName(), roles);
                    log.debug("Login successful for {} with roles {}",loginPrincipal.getFullname(),roles.toString());
                    return httpMessageContext.notifyContainerAboutLogin(up,roles);
                }else {
                    return AuthenticationStatus.SEND_FAILURE;
                }
               
            }else {
                log.error("Request to authenticate with no 'com.myapp.login.app.user' set");
            }
        }else if(httpMessageContext.isProtected() && request.getUserPrincipal()==null) {
            //Protected resource requested with no logged in user, forward to login page
            for(Role r:Role.values()) {
                if(r.getPathContext()!=null && r.getLoginPage()!=null) {
                    if(request.getRequestURI().startsWith(r.getPathContext())) {
                        log.debug("Resource protected and not logged in, forwarding to {} login page",r.toString());
                        return httpMessageContext.forward(r.getLoginPage().getPath());
                    }
                }
            }
           
            log.error("Protected resource requested does not have corresponding Role: {}",request.getRequestURI());
           
        }else if(httpMessageContext.isProtected() && request.getUserPrincipal()!=null) {
            //Protected resource requested with wrong role set, forward to login page
            UserPrincipal up = (UserPrincipal) request.getUserPrincipal();
            if(!request.getRequestURI().startsWith(up.getAppUser().coreRole().getPathContext())) {
                log.warn("Resource {} requested with logged in {}, redirecting to {}",request.getRequestURI(),up.getAppUser().coreRole().toString(),up.getAppUser().coreRole().getLoginPage().getPath());
                return httpMessageContext.forward(up.getAppUser().coreRole().getLoginPage().getPath());
            }
        }
       
        return httpMessageContext.doNothing();
       
    }
   

}

[Paul Ferraro]

OK - that's what I suspected.  @AutoApplySession is "broken" for distributed web applications in WF26, though a fix is coming for WF27.
See: https://github.com/wildfly/wildfly/pull/16028

[David Baddeley]

hmm ok that could be a problem for us as we are stuck with some libraries that are not *jakarta compatible so I believe a move to WF27 also means upgrading to EE 9/10??

Do you know if there is a patch for this issue? Or should I look at downgrading to WF25 or lower?

[Paul Ferraro]

Unfortunately, this has never worked correctly for distributable web applications.  Patching this on WF 26.x will be cumbersome, since the fix involves 4 different modules (i.e. wildfly-elytron, wildfly-elytron-ee, elytron-web, and wildfly-clustering-web-undertow), though it is feasible.

That said, you can still deploy an EE8 application on WF27 using runtime deployment transformation, which should be released before the end of the month.

[David Baddeley]

Ok good, do you know if the fix is included in the WF 27 Alpha 4 release currently available?

Reading the release notes for that is states "A major difference between WildFly Preview and standard WildFly is Preview retains the feature that will bytecode transform deployments that use the EE 8 javax.* APIs so that they instead use the analogous jakarta.* APIs. We do not intend to include that feature in standard WildFly, but we’re retaining it in WildFly Preview, at least for now.

That implies that the standard WF 27 release will not include bytecode transformation?

We do intend on moving to EE9/10 however at the moment we are still supporting a legacy .war which uses an old release of Primefaces Mobile that is no longer under development.

[Paul Ferraro]

On Tue, Sep 20, 2022 at 3:12 AM David Baddeley <da...@222sports.co.uk> wrote:
>
> Ok good, do you know if the fix is included in the WF 27 Alpha 4 release currently available?

The fix has not been merged yet. It is waiting on a new wildfly-core
beta/final release that contains updates to the requisite elytron
components.

> Reading the release notes for that is states "A major difference between WildFly Preview and standard WildFly is Preview retains the feature that will bytecode transform deployments that use the EE 8 javax.* APIs so that they instead use the analogous jakarta.* APIs. We do not intend to include that feature in standard WildFly, but we’re retaining it in WildFly Preview, at least for now.
>
> That implies that the standard WF 27 release will not include bytecode transformation?

Correct.

> You received this message because you are subscribed to the Google Groups "WildFly" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wildfly+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wildfly/3181d67e-c23e-4bfe-acff-02ff5bf4cf90n%40googlegroups.com.

[Paul Ferraro]

FYI, this fix was just recently merged.  Look out for the 27.0.0.Beta1 release coming very soon.

[David Baddeley]

Hi Paul,

Thanks for notifying, I have just got around to testing the new release, unfortunately I am getting an exception during deployment, see below.

I am using the 'preview' distribution which I though would take care of *.javax references?

After some further reading on the wildfly site I can see it stated that WF27 will no longer support EE8 applications, you stated previously we could deploy our app on WF27?

14:21:30,957 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 77) WFLYCTL0013: Operation ("add") failed - address: ([("subsystem" => "transactions")]): java.lang.RuntimeException: java.lang.NoClassDefFoundError: Ljavax/transaction/TransactionManager;
        at org.jboss.jts//com.arjuna.common.internal.util.propertyservice.BeanPopulator.getNamedInstance(BeanPopulator.java:90)
        at org.jboss.jts//com.arjuna.common.internal.util.propertyservice.BeanPopulator.getDefaultInstance(BeanPopulator.java:53)
        at org.jboss.jts//com.arjuna.ats.jta.common.jtaPropertyManager.getJTAEnvironmentBean(jtaPropertyManager.java:42)
        at org.jboss.as...@27.0.0.Beta1//org.jboss.as.txn.service.JTAEnvironmentBeanService.getValue(JTAEnvironmentBeanService.java:91)
        at org.jboss.as...@27.0.0.Beta1//org.jboss.as.txn.subsystem.TransactionSubsystemAdd.performRecoveryEnvBoottime(TransactionSubsystemAdd.java:484)
        at org.jboss.as...@27.0.0.Beta1//org.jboss.as.txn.subsystem.TransactionSubsystemAdd.performBoottime(TransactionSubsystemAdd.java:247)
        at org.jboss.a...@19.0.0.Beta18//org.jboss.as.controller.AbstractBoottimeAddStepHandler.performBoottime(AbstractBoottimeAddStepHandler.java:144)
        at org.jboss.a...@19.0.0.Beta18//org.jboss.as.controller.AbstractBoottimeAddStepHandler.performRuntime(AbstractBoottimeAddStepHandler.java:119)
        at org.jboss.a...@19.0.0.Beta18//org.jboss.as.controller.AbstractAddStepHandler$1.execute(AbstractAddStepHandler.java:164)
        at org.jboss.a...@19.0.0.Beta18//org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:1045)
        at org.jboss.a...@19.0.0.Beta18//org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:777)
        at org.jboss.a...@19.0.0.Beta18//org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:466)
        at org.jboss.a...@19.0.0.Beta18//org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:384)

        at org.jbos...@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
        at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)

        at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.jbos...@2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)
Caused by: java.lang.NoClassDefFoundError: Ljavax/transaction/TransactionManager;
        at java.base/java.lang.Class.getDeclaredFields0(Native Method)
        at java.base/java.lang.Class.privateGetDeclaredFields(Class.java:3061)
        at java.base/java.lang.Class.getDeclaredFields(Class.java:2248)
        at org.jboss.jts//com.arjuna.common.internal.util.propertyservice.BeanPopulator.configureFromProperties(BeanPopulator.java:144)
        at org.jboss.jts//com.arjuna.common.internal.util.propertyservice.BeanPopulator.getNamedInstance(BeanPopulator.java:87)
        ... 18 more
Caused by: java.lang.ClassNotFoundException: javax.transaction.TransactionManager from [Module "org.jboss.jts" from local module loader @16746061 (finder: local module finder @57fd91c9 (roots: /opt/jboss/wildfly/modules,/opt/jboss/wildfly/modules/system/layers/base))]
        at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:200)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:410)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
        at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:116)
        ... 23 more

[vijen reddy]

Hi David,

Did you resolve this issue??

I am also having same issue...

If login on Node 1 to create some session data, then shutdown Node 1, when I refresh the page I get the follow exception at Node 2....... but with different message saying:

 2023-03-09 17:15:21,790 [ALANTAI,QpeoJ4i-PzcnDpRIik-UY61iZ_XH_0obUWfuJUbU,10.70.5.7] INFO  [com.converge.framework.listners.SessionListener] Current Session created : KPq9A0xstt1tPQbzlEtnoe2iMHhTOY8lXcaGNAyS at Thu Mar 09 17:15:21 EST 2023
2023-03-09 17:15:21,790 [ALANTAI,QpeoJ4i-PzcnDpRIik-UY61iZ_XH_0obUWfuJUbU,10.70.5.7] INFO  [com.converge.framework.listners.SessionTimeOutFilter] Session is invalid! redirecting to timeoutpage : /QTrade/secure/login

Thanks,

VJ

[David Baddeley]

Hi Vijen,

No unfortunately we could not upgrade to WF27 so we could not fix it so we have disabled session replication for the application

David