WildFly 40.0.0 Final session access during requestDestroyed() leaks a ThreadContextBatch, poisoning worker threads and causing IllegalStateException in DistributableSession.requestDone (Intermittent)

[sergio loza]

Hello All,  

We have an app running WildFly37 and Java 21.  When upgrading to Wildfly40 we began to see ) org.wildfly.clustering.cache.infinispan.batch.ThreadContextBatch$2@10325ff5: java.lang.IllegalStateException with Undertow

On a Multinode HA cluster with `<replicable/>` web sessions and any CDI-enabled deployment,
ordinary traffic intermittently produces:

```
WARN  [org.wildfly.clustering.web.undertow] (default task-N)
org.wildfly.clustering.cache.infinispan.batch.ThreadContextBatch$2@...:
java.lang.IllegalStateException: org.wildfly.clustering.cache.infinispan.batch.ThreadContextBatch$2@...
at org.wildfly.clustering.cache.infinispan.batch.ThreadContextBatch$2.resume(ThreadContextBatch.java:82)
at org.wildfly.clustering.session.cache.CachedSession.resume(CachedSession.java:47)
at org.wildfly.clustering.session.cache.CachedSessionManager$1.accept(CachedSessionManager.java:66)
...
at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:52)
Caused by: org.wildfly.clustering.cache.infinispan.batch.ContextualException: <N>
```

It took me a good while to reproduce  in my local  but I have a sample app with a hammer  script that  reproduces this on any multi node local setup. In our dev environment  is much easier to reproduce but I do want to acknowledge that it is intermittent.

My  minimal reproducible  app looks like this ( happy to share it)

The WAR File  I created is  minimal: `<replicable/>`, one Struts login/logout flow, one
`@Named @SessionScoped` CDI bean touched on every session-carrying request (via a filter),
and a servlet filter that, on an `iv-user` request header, invalidates the current session
and populates a replacement (mimicking SSO proxy-login filters).

Hammer script: N concurrent workers; per session: login, then bursts of parallel static-asset requests
sharing the session cookie; mid-burst, one request with an `iv-user` header triggers the
invalidate-and-replace, and the remaining burst traffic adopts the replacement cookie
(browser behavior). With 12 workers x 200 sessions this produced 29 warnings in one run

Beyond the warning, the same leaked batch can surface as a `CompletionException` in
application code during login/session-invalidation flows (caught by application exception
handlers, failing user logins).

Here is a sample  log signature found in dev node

```
(task-A) Created batch 4,258                 <- SessionReferenceReader.get(), Weld teardown
(task-A) Created child context 4258[1]       <- SessionManagerFunction extends it (cache miss)
(task-A) Locating session ...                <- the find that created the new shared entry
         [no "Closed batch 4,258" anywhere in the log - reader's close was a no-op]
(task-B) Closed child context 4258[1]        <- another thread last-closes the entry; batch
                                                stays attached to task-B (poisoned)
(task-B) WARN ... IllegalStateException ... Caused by: ContextualException: 4258
(task-B) Created child context 4258[1]       <- post-failure ops keep nesting into the zombie

I have attached  a sample of our log ( sanitized) with task 888 showing the behavior.

It contains the complete lifecycle of leaked
batch `5042`: the `Created child context 5042[1]` events, the WARN with
`ContextualException: 5042` carrying the full creation stack (Weld
`requestDestroyed` → session bean store → `SessionReferenceReader.get`), and the post-failure
child-context churn on the poisoned thread

I have read other 

IllegalStateException

 issues but it seems like those were resolved fixed with 40.0.0 Final.

[Viktoriia Teliuk]

Probably, it is related to https://redhat.atlassian.net/issues?jql=textfields%20~%20%22org.wildfly.clustering.cache.infinispan.batch.ContextualException%22&selectedIssue=WFLY-21965