HHH000376: Could not updateQuery hi value in: hibernate_sequence

[Michael Dempfle]

Hi, we have a really strange problem on our QA system . It started without any deployment going on in our Saga:

java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '327' for key 'PRIMARY' at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:117) at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:955) at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1094) at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1042) at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1345) at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:1027) at jdk.internal.reflect.GeneratedMethodAccessor85.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.mysql.cj.jdbc.ha.MultiHostConnectionProxy$JdbcInterfaceProxy.invoke(MultiHostConnectionProxy.java:105) at com.mysql.cj.jdbc.ha.FailoverConnectionProxy$FailoverJdbcInterfaceProxy.invoke(FailoverConnectionProxy.java:98) at com.sun.proxy.$Proxy140.executeUpdate(Unknown Source) at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) at org.hibernate.id.enhanced.TableStructure.executeUpdate(TableStructure.java:205) at org.hibernate.id.enhanced.TableStructure.access$900(TableStructure.java:46) at org.hibernate.id.enhanced.TableStructure$1$1.execute(TableStructure.java:163) at org.hibernate.id.enhanced.TableStructure$1$1.execute(TableStructure.java:126) at org.hibernate.jdbc.WorkExecutor.executeReturningWork(WorkExecutor.java:55) at org.hibernate.jdbc.AbstractReturningWork.accept(AbstractReturningWork.java:34) at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcIsolationDelegate.delegateWork(JdbcIsolationDelegate.java:57) at org.hibernate.id.enhanced.TableStructure$1.getNextValue(TableStructure.java:125) at org.hibernate.id.enhanced.NoopOptimizer.generate(NoopOptimizer.java:40) at org.hibernate.id.enhanced.SequenceStyleGenerator.generate(SequenceStyleGenerator.java:482) at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:119) at org.hibernate.event.internal.DefaultPersistEventListener.entityIsTransient(DefaultPersistEventListener.java:192) at org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:135) at org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:62) at org.hibernate.internal.SessionImpl.firePersist(SessionImpl.java:800) at org.hibernate.internal.SessionImpl.persist(SessionImpl.java:785) at jdk.internal.reflect.GeneratedMethodAccessor106.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:308) at com.sun.proxy.$Proxy133.persist(Unknown Source) at org.axonframework.modelling.saga.repository.jpa.JpaSagaStore.storeAssociationValue(JpaSagaStore.java:223) at org.axonframework.modelling.saga.repository.jpa.JpaSagaStore.insertSaga(JpaSagaStore.java:302) at org.axonframework.modelling.saga.repository.AnnotatedSagaRepository.storeSaga(AnnotatedSagaRepository.java:217) at org.axonframework.modelling.saga.repository.AnnotatedSagaRepository.lambda$doCreateInstance$3(AnnotatedSagaRepository.java:139) at org.axonframework.messaging.unitofwork.MessageProcessingContext.notifyHandlers(MessageProcessingContext.java:71) at org.axonframework.messaging.unitofwork.BatchingUnitOfWork.lambda$notifyHandlers$2(BatchingUnitOfWork.java:155) at java.base/java.util.ArrayList$Itr.forEachRemaining(ArrayList.java:1032) at org.axonframework.messaging.unitofwork.BatchingUnitOfWork.notifyHandlers(BatchingUnitOfWork.java:155) at org.axonframework.messaging.unitofwork.AbstractUnitOfWork.changePhase(AbstractUnitOfWork.java:222) at org.axonframework.messaging.unitofwork.AbstractUnitOfWork.commitAsRoot(AbstractUnitOfWork.java:83) at org.axonframework.messaging.unitofwork.AbstractUnitOfWork.commit(AbstractUnitOfWork.java:71) at org.axonframework.messaging.unitofwork.BatchingUnitOfWork.executeWithResult(BatchingUnitOfWork.java:111) at org.axonframework.eventhandling.AbstractEventProcessor.processInUnitOfWork(AbstractEventProcessor.java:151) at org.axonframework.eventhandling.TrackingEventProcessor.processBatch(TrackingEventProcessor.java:395) at org.axonframework.eventhandling.TrackingEventProcessor.processingLoop(TrackingEventProcessor.java:269) at org.axonframework.eventhandling.TrackingEventProcessor$TrackingSegmentWorker.run(TrackingEventProcessor.java:990) at java.base/java.lang.Thread.run(Thread.java:834)

We see 

Duplicate segment claim detected and we are not able to solve this in any way in the axon dashboard. Seems the whole claiming is going wild. The sized do not match and anything we do seems to make it worse. 

Any idea what can cause this? 

Best, Michael

[Michael Dempfle]

Hi ,

The problem was, that in the hybernate_sequence table where 2 entries: 

326 and 327

Axon was reading 326 and tried to write 327 to the table which already existed. 

How can this happen? And how can this be prevented. I think this is a Axon problem which should be fixed asap. It crashed the whole system. 

Best, 

Michael

[Allard Buijze]

Hi Michael,

the sequence table is used internally by Hibernate. Since Hibernate 5 they changed the default sequence to one using a table instead of MySQL's native auto-increment. Axon has absolutely no influence on how this sequence is used, although there are ways to force Hibernate to use the auto-increment instead. 

The double entry in the table is more likely to be an issue in Hibernate.

Kind regards,

Allard

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/axonframework/93e341c4-d487-42b5-931d-47401f10c7f3%40googlegroups.com.

[Michael Dempfle]

Hi Allard,

But could the framework not do an additional check to solve this when it happens? 

Best, Michael

Am Mittwoch, 26. Februar 2020 16:25:49 UTC+1 schrieb Allard Buijze:

Hi Michael,

the sequence table is used internally by Hibernate. Since Hibernate 5 they changed the default sequence to one using a table instead of MySQL's native auto-increment. Axon has absolutely no influence on how this sequence is used, although there are ways to force Hibernate to use the auto-increment instead. 

The double entry in the table is more likely to be an issue in Hibernate.

Kind regards,

Allard

On Fri, 14 Feb 2020 at 17:58, Michael Dempfle <mi...@mdempfle.de> wrote:

Hi ,

The problem was, that in the hybernate_sequence table where 2 entries: 

326 and 327

Axon was reading 326 and tried to write 327 to the table which already existed. 

How can this happen? And how can this be prevented. I think this is a Axon problem which should be fixed asap. It crashed the whole system. 

Best, 

Michael

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to axonfr...@googlegroups.com.

[Allard Buijze]

I don't think it is Axon's responsibility, in the first place, but Hibernate's. Hibernate abstracts these sequences away to the extent that they are hard to access. Furthermore, Hibernate creates them, and has certain expectations of only a single element to be available. So it should also check on those assumptions.

Cheers,

Allard Buijze

CTO

To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/axonframework/07d2b348-7ea3-4ea4-9056-c17dd2e0f87c%40googlegroups.com.