Parallel Java Delegates causing deadlock when transitioning to end activity

[Gareth]

Hi All,

I've done some searching on the forum before posting, but couldn't get to the bottom of this issue.  

Im using Camunda v7.2 running on Postgres 9.2.4 and have a simple process that executes two Java delegates in a tight loop ( decrementing a counter ) in parallel, which modify different process variables ( counters ), the delegates complete successfully, but when the process tries to transition to the end event I receive the following error, indicating a Deadlock has occurred.  Once the lock timeout expires, the lock is released and the process succeeds ( complete with the correct process variables )

As this error happens outside of the Java Delegates, is there anything I can do, or a setting I can enable to allow parallel flows to modify process variables ? ( This seems very basic so i must be missing something ? )

( It looks similar to https://groups.google.com/forum/#!searchin/camunda-bpm-users/deadlock/camunda-bpm-users/8fUx40_PhAs/8GD7_xzVsDYJ - but I didn't understand the solution offered )  I can provide the bpmn2, but it references custom delegates code so im not sure how much use it would be ) 

Thanks in advance,

Gareth

Oct 14, 2015 1:29:45 PM org.camunda.bpm.engine.impl.interceptor.CommandContext close

SEVERE: Error while closing command context

org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:45 PM org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable run

WARNING: Exception while executing job with id 4048897e-726f-11e5-8060-3e7a6115a7d6

org.camunda.bpm.engine.ProcessEngineException: Process engine persistence exception

at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.rethrow(CommandInvocationContext.java:60)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:229)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

... 7 more

Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:45 PM org.camunda.bpm.engine.impl.interceptor.CommandContext close

SEVERE: Error while closing command context

org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:45 PM org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable run

WARNING: Exception while executing job with id 4048897e-726f-11e5-8060-3e7a6115a7d6

org.camunda.bpm.engine.ProcessEngineException: Process engine persistence exception

at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.rethrow(CommandInvocationContext.java:60)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:229)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

... 7 more

Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:45 PM org.camunda.bpm.engine.impl.interceptor.CommandContext close

SEVERE: Error while closing command context

org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:45 PM org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable run

WARNING: Exception while executing job with id 4048897e-726f-11e5-8060-3e7a6115a7d6

org.camunda.bpm.engine.ProcessEngineException: Process engine persistence exception

at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.rethrow(CommandInvocationContext.java:60)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:229)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.deleteExecution-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_EXECUTION where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

... 7 more

Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "act_ru_execution" violates foreign key constraint "act_fk_exe_parent" on table "act_ru_execution"

  Detail: Key (id_)=(3feb00c6-726f-11e5-8060-3e7a6115a7d6) is still referenced from table "act_ru_execution".

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:46 PM org.camunda.bpm.engine.impl.interceptor.CommandContext close

SEVERE: Error while closing command context

org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.JobEntity.deleteMessage-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_JOB where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:46 PM org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext trySetThrowable

SEVERE: masked exception in command context. for root cause, see below as it will be rethrown later.

org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.JobEntity.deleteMessage-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_JOB where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

Oct 14, 2015 1:29:46 PM org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable run

WARNING: Exception while executing job with id 4048897d-726f-11e5-8060-3e7a6115a7d6

org.camunda.bpm.engine.ProcessEngineException: Process engine persistence exception

at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.rethrow(CommandInvocationContext.java:60)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:229)

at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:106)

at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:79)

at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:67)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.ibatis.exceptions.PersistenceException:

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

### The error may involve org.camunda.bpm.engine.impl.persistence.entity.JobEntity.deleteMessage-Inline

### The error occurred while setting parameters

### SQL: delete from ACT_RU_JOB where ID_ = ? and REV_ = ?

### Cause: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.delete(DefaultSqlSession.java:165)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.executeDelete(DbSqlSession.java:179)

at org.camunda.bpm.engine.impl.db.sql.DbSqlSession.deleteEntity(DbSqlSession.java:164)

at org.camunda.bpm.engine.impl.db.AbstractPersistenceSession.executeDbOperation(AbstractPersistenceSession.java:39)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:265)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.flushSessions(CommandContext.java:258)

at org.camunda.bpm.engine.impl.interceptor.CommandContext.close(CommandContext.java:187)

... 7 more

Caused by: org.postgresql.util.PSQLException: ERROR: deadlock detected

  Detail: Process 57120 waits for ShareLock on transaction 3100; blocked by process 57118.

Process 57118 waits for ShareLock on transaction 3099; blocked by process 57120.

  Hint: See server log for query details.

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)

at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:419)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:412)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:48)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 14 more

[Gareth]

To follow up on this.  Camunda also seems to have transaction issues with parallel flows and sub processes, whereby a parent process cannot be marked as completed due to the below optimistic locking error.  The use case here is a simple looping process that calls X subprocesses in a loop and then ends.  However the parent process never gets updated.

The only way i've found around this is to prevent parallel flows running ... in parallel.  Which kind of defeats the object, but the only way I can get the engine to run reliably.

parallelGateway.setCamundaExclusive( true );

If anyone has any other solutions id be grateful to hear them.

Kind Regards

Gareth

WARNING: Exception while executing job with id 804b2b78-7317-11e5-8060-3e7a6115a7d6

org.camunda.bpm.engine.OptimisticLockingException: Could not execute UPDATE MessageEntity[804cb219-7317-11e5-8060-3e7a6115a7d6]. Entity was updated by another transaction concurrently

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.handleOptimisticLockingException(DbEntityManager.java:287)

at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.flush(DbEntityManager.java:267)

[thorben....@camunda.com]

Hi Gareth,

Optimistic locking is the process engines mechanism to cope with parallel updates on the same entities. This is for example needed when process flow needs to be synchronized, for example if you call X subprocesses from a process instance, the completion of a subprocess must synchronize on the parent process instance to ensure that the last subprocess finishing also finishes the parent process instance. Similar synchronization issues exist with parallel gateways, etc. So OptimisticLockingExceptions are generally expected behavior in these scenarios. There are various ways to deal with it:

1. Use exclusive jobs as you mentioned. For many users, it is okay to execute jobs not truely in parallel. They rather achieve parallelism on the inter-process-instance level, not intra-process-instance. Even with exclusive jobs, the purpose of a parallel gateway is not defeated by exclusive jobs. For example if you have a parallel gateway followed by two user tasks, the tasks may nevertheless be performed in parallel.
2. Set transaction boundaries [1] using async flags in such a way that a job failure and retry is not crucial. For example, if you perform an expensive computation and then there is a synchronization point in the process, set an asynchronous continuation flag between these two steps. This way, optimistic locking due to synchronization will not result in repeating the expensive computation. You have to be careful with setting the transaction boundaries, though, as not all synchronization issues are that obvious.

As for the deadlock issue: Are you able to identify which statements cause the deadlock? You can also set the log level for the class org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager to fine to have the types of statements logged when the process engine flushes its cache. That may be useful in identifying the problem.

Cheers,
Thorben

[1] https://docs.camunda.org/manual/7.3/guides/user-guide/#process-engine-transactions-in-processes-transaction-boundaries

[Gareth]

Thanks for your reply Thorben.  Its good to know exclusive jobs are one of the normal options.  The exclusive option also removes the deadlock issue, but ill make a note of the logging entry for future debugging.