Problem with Oracle XA data source on JBoss/ATG9

[Lambert Torres]

Hi again,

Just continuing our migration from atg7 to atg9 and have come up with an issue on JBoss with an Oracle XA datasource.

We've configured the oracle-xa-ds.xml as per documentation.  The system starts up and then hangs unresponsive to requests until 3 or 4 minutes later where this gets spit out.

I have included the oracle-xa-ds.xml file below for reference; though the host names and ports have been changed, obviously. Any help would be much appreciated.

Versions:
ATG 9.0
JBoss AS 4.2.0GA
Oracle 10.2.0.1
Oracle ojdbc14.jar for 10.2.0.1
Java 1.5.0_17
Windows XP SP3

Thanks,
Lambert


2009-02-11 17:23:14,915 WARN  [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.transaction.arjunacore.xastart] [com.arjuna.ats.internal.jta.transaction.arjunacore.xastart] TransactionImple.enlistResource - xa_start  - caught: XAException.XA_RETRY for < 131075, 26, 24, 1-ac1b82d:9b3:49934e5c:1c5ac1b82d:9b3:49934e5c:36f
2009-02-11 17:23:14,915 ERROR [STDERR] oracle.jdbc.xa.OracleXAException
2009-02-11 17:23:14,915 ERROR [STDERR]     at oracle.jdbc.xa.OracleXAResource.checkError(OracleXAResource.java:1101)
2009-02-11 17:23:14,915 ERROR [STDERR]     at oracle.jdbc.xa.client.OracleXAResource.start(OracleXAResource.java:237)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.adapter.jdbc.xa.XAManagedConnection.start(XAManagedConnection.java:121)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.start(JcaXAResourceWrapper.java:113)
2009-02-11 17:23:14,915 ERROR [STDERR]     at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:775)
2009-02-11 17:23:14,915 ERROR [STDERR]     at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:446)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener$TransactionSynchronization.enlist(TxConnectionManager.java:773)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener.enlist(TxConnectionManager.java:564)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.connectionmanager.TxConnectionManager.managedConnectionReconnected(TxConnectionManager.java:337)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.connectionmanager.BaseConnectionManager2.reconnectManagedConnection(BaseConnectionManager2.java:518)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:399)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:842)
2009-02-11 17:23:14,915 ERROR [STDERR]     at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:88)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.adapter.gsa.GSATransaction.getConnection(GSATransaction.java:725)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.adapter.gsa.GSAItemDescriptor.getConnection(GSAItemDescriptor.java:2314)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.adapter.gsa.GSAItemDescriptor.executeQuery(GSAItemDescriptor.java:7261)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.adapter.gsa.GSAView.executeUncachedQuery(GSAView.java:317)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.adapter.gsa.GSAView.executeUncachedQuery(GSAView.java:417)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.repository.RepositoryViewImpl.executeQuery(RepositoryViewImpl.java:346)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.adapter.gsa.GSAView.executeQuery(GSAView.java:266)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.repository.RepositoryViewImpl.executeQuery(RepositoryViewImpl.java:323)
2009-02-11 17:23:14,915 ERROR [STDERR]     at atg.repository.RepositoryViewImpl.executeQuery(RepositoryViewImpl.java:246)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.process.ProcessManagerService.findAllMigrations(ProcessManagerService.java:5239)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.process.ProcessManagerService.completeAllPendingMigrations(ProcessManagerService.java:10290)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.process.ProcessManagerService.initializeProcesses(ProcessManagerService.java:7189)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.process.ProcessManagerService.startMessageSource(ProcessManagerService.java:14388)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.dms.patchbay.ElementManager.startMessageSource(ElementManager.java:882)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.dms.patchbay.PatchBayManager.startMessageSources(PatchBayManager.java:1941)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.dms.patchbay.PatchBayManager.doStartService(PatchBayManager.java:601)
2009-02-11 17:23:14,931 ERROR [STDERR]     at atg.nucleus.GenericService.startService(GenericService.java:496)

[Naga]

Do you know ATG9 is supported only on 420EAP and not 420GA ?

-Naga

> <?xml version="1.0" encoding="UTF-8"?>
> <datasources>
>
> <xa-datasource>
> <jndi-name>OracleDS</jndi-name>
> <track-connection-by-tx/>
> <isSameRM-override-value>false</isSameRM-override-value>
>
> <xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
> <xa-datasource-property
> name="URL">jdbc:oracle:thin:@HOST:PORT:nikedevx</xa-datasource-property>
> <xa-datasource-property name="User">test</xa-datasource-property>
> <xa-datasource-property name="Password">test</xa-datasource-property>
>
> <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</exception-sorter-class-name>
> <no-tx-separate-pools/>
> <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
> <metadata>
> <type-mapping>Oracle9i</type-mapping>
> </metadata>
> </xa-datasource>
>
> <xa-datasource>
> <jndi-name>ProfileOracleDS</jndi-name>
> <track-connection-by-tx/>
> <isSameRM-override-value>false</isSameRM-override-value>
>
> <xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
> <xa-datasource-property
> name="URL">jdbc:oracle:thin:@HOST:PORT:nikedevx</xa-datasource-property>
> <xa-datasource-property name="User">test</xa-datasource-property>
> <xa-datasource-property name="Password">test</xa-datasource-property>
>
> <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</exception-sorter-class-name>
> <no-tx-separate-pools/>
> <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
> <metadata>
> <type-mapping>Oracle9i</type-mapping>
> </metadata>
> </xa-datasource>
>
> <mbean
> code="org.jboss.resource.adapter.jdbc.vendor.OracleXAExceptionFormatter"
> name="jboss.jca:service=OracleXAExceptionFormatter">
> <depends
> optional-attribute-name="TransactionManagerService">jboss:service=TransactionManager</depends>
> </mbean>
>
> </datasources>
>
>



--
http://www.linkedin.com/in/nagarajanseshadri

[Raja Ramachandran]

Looks like the system is unable to enlist a resource into the
transaction. I have seen something similar happening when you have a
table that was modified outside of ATG without being committed. I
maybe way off the mark here.

On Wed, Feb 11, 2009 at 4:50 PM, Lambert Torres
<lambert...@gmail.com> wrote:

[Belmont, Adam]

Here's what  I use:

<?xml version="1.0" encoding="UTF-8"

?>
<datasources>
    <xa-datasource>
          <jndi-name>atgcore_ds</jndi-name>
          <track-connection-by-tx>false</track-connection-by-tx>
          <isSameRM-override-value>false</isSameRM-override-value>
          <min-pool-size>5</min-pool-size>
          <max-pool-size>100</max-pool-size>
          <blocking-timeout-millis>5000</blocking-timeout-millis>
          <idle-timeout-minutes>15</idle-timeout-minutes>
          <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
          <xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
          <xa-datasource-property name="URL">jdbc:oracle:thin:@db.company.com:1521:ora10r2</xa-datasource-property>
          <xa-datasource-property name="User">scott</xa-datasource-property>
          <xa-datasource-property name="Password">tiger</xa-datasource-property>
          <!-- Uncomment the following if you are using Oracle 9i
          <xa-datasource-property name="oracle.jdbc.V8Compatible">true</xa-datasource-property>
         -->
          <exception-sorter-class-name>
              org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter
          </exception-sorter-class-name>
    </xa-datasource>
</datasources>

The "track-connection-by-tx" value looks like it may be different.

Try getting a thread dump while it's hung to see if multiple java threads are involved. If it's just one thread then it could be some other process connecting to the db.

 -- Adam

---

From: atg_...@googlegroups.com [mailto:atg_...@googlegroups.com] On Behalf Of Lambert Torres
Sent: Wednesday, February 11, 2009 5:51 PM
To: atg_...@googlegroups.com
Subject: [atg_tech:2998] Problem with Oracle XA data source on JBoss/ATG9

[Lambert Torres]

Hi All,

Just wanted to follow up:

We have double and triple checked our ATG database schema, and the tables match up with what they should be.

 We've made sure that our Java version is the one supported, v1.5.0_17, and that our Oracle JDBC driver is the correct one for our database: v10.2.0.4. We've tried many permutations of settings in the oracle-xa-ds.xml, but every run produces this same behavior: The big stack trace in the log after a period of four-five minutes of complete unresponsiveness after JBoss startup.

Could this be due to some problem in our database server? Or, does this seem like an issue with some other JBoss or ATG9 configuration file?

 Oh, to comment on the use of JBoss AS 4.2.0GA, we understand that only EAS is supported, however another team has explained to us that, although we will be running in production and staging with EAS, AS should work just fine on developer desktops, and we don't have the budget or the need to purchase EAS licenses for every developer.

thanks again,
Lambert

[Belmont, Adam]

I agree. I don't think the issue is about EAP vs community versions. Did you get a stack trace while it was hung (before it times itself out) to see what java code was waiting at the time of the problem.

 

-- Adam

---

From: atg_...@googlegroups.com [mailto:atg_...@googlegroups.com] On Behalf Of Lambert Torres

Sent: Thursday, February 12, 2009 3:30 PM
To: atg_...@googlegroups.com
Subject: [atg_tech:3018] Re: Problem with Oracle XA data source on JBoss/ATG9

[Lambert Torres]

Thanks for the prompt reply.  I've enclosed it. 

Lambert

[Belmont, Adam]

Ok, so just one thread since this is happening at startup like you said. So there isn't any contention between processes going on.

Also this is happening when SQLJMS goes to ask for a connection.

It hasn't even attempted to build a statement or anything.

 

If that's the case I'd think that you could reproduce the problem with a non-ATG web application that used the same Datasource.

Just to rule out the basics try a simple non-ATG application that just tries to get a connection from your datasource.

I'd expect that you'd see the same hang on your call to "getConnection()".

 

Not that this helps you solve the problem but it may give you a simpler test case to deal with.

 

Also do you really have two datasources defined that connect to the exact database/username combination.

I assume not, but if so, that could be part of the issue.

Which one of the datasources is PatchBay (/atg/dynamo/messing/MessagingManager component) using for it's datasource?

Are there other development machines connecting to the exact same account elsewhere? Perhaps oracle is complaining because of that.

 

I saw some interesting info googling on "oracle XA_RETRY JBOSS" but nothing stood out. You might find something though.

 

 -- Adam

---

From: atg_...@googlegroups.com [mailto:atg_...@googlegroups.com] On Behalf Of Lambert Torres

Sent: Friday, February 13, 2009 4:33 PM
To: atg_...@googlegroups.com
Subject: [atg_tech:3023] Re: Problem with Oracle XA data source on JBoss/ATG9

[Gordon Cooke]

Or simply write a main method that makes a JDBC call.  This should be enough to identify if it is an issue with your DB or the drivers.  From there try running the same code in a web-app in jboss, outside of ATG.  Limit the things that you are actually testing.

gc