Strange new behaviour from hibernate multi-tenancy as of wildfly 20 upgrade

[Watchuta Awkin]

Hi there. I am new to multitenancy but since working from home, the project lead got retrenched and i got put onto her project where multitenancy is evidently being used a lot, to communicate with other WARs.. I just cannot understand this symptom: 

It appears, that source code, compiled a few years ago, deploys & runs as-is on wildfly 13,14,17,18(as long as all dsatasources are pre-configured on WF) .. but exact same code does not even deploy(or rather deploys but can not be enabled) on WF20. All I see in WF error log is the generic hibernate errors thrown, citing the persistent units from the persistence.xml

ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([("deployment" => "repository-1.0.1-SNAPSHOT.jar
")]) - failure description: {"WFLYCTL0080: Failed services" => {
"jboss.persistenceunit.\"repository-1.0.1-SNAPSHOT.jar#dsx_PU\"" => "org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.Jdb
cEnvironment]
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
Caused by: org.hibernate.HibernateException: Improper set up of DataSourceBasedMultiTenantConnectionProviderImpl",
"jboss.persistenceunit.\"repository-1.0.1-SNAPSHOT.jar#ds_PU\"" => "org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEn
vironment]
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
Caused by: org.hibernate.HibernateException: Improper set up of DataSourceBasedMultiTenantConnectionProviderImpl"
}}

we use/set the following properties in persistence.xml (again, this works in all previous versions of wildfly, but not on 20) 

hibernate.multi_tenant_connection_provider="org.hibernate.engine.jdbc.connections.spi.DataSourceBasedMultiTenantConnectionProviderImpl"

hibernate.multiTenancy="DATABASE"

hibernate.connection.datasource="java:jboss/datasources"

hibernate.multi_tenant.datasource.identifier_for_any="dsx_DS"

hibernate.dialect="org.hibernate.dialect.Oracle9iDialect"

hibernate.tenant_identifier_resolver="xyz.CustomResolver"

Is there perhaps some sort of logging category that can be added to get more specific error info. Was there some glaring change to hibernate-core that I have missed, please provide URL for me to go research in the right places.. 

Has anyone managed to deploy their old multi tenancy code on WF20? 

I feel like am hitting brick wall with no progress by just doing google searches.

Please help point me in the right direction!
 

[Scott Marlow]

On Tuesday, February 2, 2021 at 2:54:30 PM UTC-5 watchu...@gmail.com wrote:

Hi there. I am new to multitenancy but since working from home, the project lead got retrenched and i got put onto her project where multitenancy is evidently being used a lot, to communicate with other WARs..

Hi, I have read about the feature but haven't used multitenancy.

 

I just cannot understand this symptom: 

It appears, that source code, compiled a few years ago, deploys & runs as-is on wildfly 13,14,17,18(as long as all dsatasources are pre-configured on WF) .. but exact same code does not even deploy(or rather deploys but can not be enabled) on WF20. All I see in WF error log is the generic hibernate errors thrown, citing the persistent units from the persistence.xml

ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([("deployment" => "repository-1.0.1-SNAPSHOT.jar
")]) - failure description: {"WFLYCTL0080: Failed services" => {
"jboss.persistenceunit.\"repository-1.0.1-SNAPSHOT.jar#dsx_PU\"" => "org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.Jdb
cEnvironment]
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
Caused by: org.hibernate.HibernateException: Improper set up of DataSourceBasedMultiTenantConnectionProviderImpl",
"jboss.persistenceunit.\"repository-1.0.1-SNAPSHOT.jar#ds_PU\"" => "org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEn
vironment]
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment]
Caused by: org.hibernate.HibernateException: Improper set up of DataSourceBasedMultiTenantConnectionProviderImpl"
}}

we use/set the following properties in persistence.xml (again, this works in all previous versions of wildfly, but not on 20) 

hibernate.multi_tenant_connection_provider="org.hibernate.engine.jdbc.connections.spi.DataSourceBasedMultiTenantConnectionProviderImpl"

hibernate.multiTenancy="DATABASE"

hibernate.connection.datasource="java:jboss/datasources"

hibernate.multi_tenant.datasource.identifier_for_any="dsx_DS"

hibernate.dialect="org.hibernate.dialect.Oracle9iDialect"

hibernate.tenant_identifier_resolver="xyz.CustomResolver"

Is there perhaps some sort of logging category that can be added to get more specific error info. Was there some glaring change to hibernate-core that I have missed, please provide URL for me to go research in the right places.. 

The Hibernate source is at https://github.com/hibernate/hibernate-orm (git clone https://github.com/hibernate/hibernate-orm) will get you the latest "master" branch, you will need to checkout the 5.3.x branch which should reflect what is in WildFly 22 (and still includes changes that went into WildFly 20).

I don't need to see more of the exception call stack but IMO, you should see where the exception is thrown from in Hibernate ORM and look for new code changes that might be involved.  My guess is that some code change is causing the "Improper set up of DataSourceBasedMultiTenantConnectionProviderImpl" failure.  It might help to review the code that throws the exception to see why the relevant Hibernate ORM code is complaining.

Scott

[Scott Marlow]

FYI, this looks the same as https://issues.redhat.com/browse/WFLY-13617 which I think should be looked at in Hibernate ORM.  Did you create a Hibernate ORM (HHH) jira for this failure?  I'm going to suggest that be done for https://issues.redhat.com/browse/WFLY-13617.

Scott