Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

allow postgres XA transactions to recover

42 views
Skip to first unread message

Tom Eicher

unread,
Jan 13, 2025, 12:26:10 PMJan 13
to WildFly

Hello,

with our clustered wildfly (26.1.3), after some hazard, e.g. database pools exhausted,
after restarting we regularly see errors like

2025-01-13 17:41:09,978 {/} WARN com.arjuna.ats.jta ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff054bea62:-5cd8a387:6784db2a:2b95d, node_name=1, branch_uid=0:ffff054bea62:-5cd8a387:6784db2a:2b964, subordinatenodename=null, eis_name=java:/jboss/datasources/xxxportalDS/crowdXXXXX >, heuristic: TwoPhaseOutcome.HEURISTIC_HAZARD, product: PostgreSQL/14.15, jndiName: java:/jboss/datasources/xxxportalDS/crowXXXXXX com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5a8da963 >

The system uses XA Datasources since all requests need to access a "common" database (for user authentication)
and one of many tenant-specific databases (with the business data of one customer),
those are looked up from wildfly using JNDI in my AbstractMultiTenantConnectionProvider.

database is postgres 14.15 with driver postgresql-42.5.4

defined, like,

<xa-datasource jndi-name="java:/jboss/datasources/xxxportalDS/common" pool-name="xxxportalPoolCommon" enabled="true" use-java-context="true" statistics-enabled="true">
<xa-datasource-property name="Url">...</xa-datasource-property>
<driver>postgresql</driver>
...
</xa-datasource>

<xa-datasource jndi-name="java:/jboss/datasources/xxxportalDS/tenant2" pool-name="xxxportalPoolTenant2" enabled="true" use-java-context="true" statistics-enabled="true">
<xa-datasource-property name="Url">...</xa-datasource-property>
<driver>postgresql</driver>
...
</xa-datasource>

Reading upon the issue, it seems PGXADataSource is not Serializable, but I can't change that ;-)
and some old posts suggest configuring stuff like
<property name="com.arjuna.ats.jta.recovery.XAResourceRecoveryJDBC"
value="com.arjuna.ats.internal.jdbc.recovery.JDBCXARecovery"/>
in a jbossjta.xml or something.

But I have nothing like that, not in the standard distribution (standalone)
and not in my standalone.xml

How could I make XA transactions with postgres jdbc work so they can recover after a hazard?

kind regards
Thanks Tom.


PS I have, like, 
<socket-binding name="txn-recovery-environment" port="4712"/
but I don't know what this means, or if anybody is listening there...

Michael Musgrove

unread,
Jan 14, 2025, 8:49:16 AMJan 14
to WildFly
Heuristic outcomes are generally unrecoverable, in your case the definition of HEURISTIC_HAZARD means that "the work done on behalf of the specified transaction branch may have been heuristically completed" and the disposition of some of the updates is unknown.

The recovery procedure [1,2] could be:

- use the WildFly CLI to find the transaction and try moving it back to the prepared state using the recover command and the transaction manager will reattempt the commit
- if you know that all the participants have finished in the same state (either rolled back or committed) then you could manually delete the log (again using the CLI)

And if neither of these options are appropriate then you can figure out via the CLI and/or the vendor tooling what the actual disposition of the resources are and if they are all in the same state then you could manually finish the transaction using the vendor's tooling.
Reply all
Reply to author
Forward
0 new messages