Clear all old/terminated process instances

[Frank Bitzer]

Hi there,

what is the best way to clear all old process instances for a particular process? (despite of dropping the whole database :)

I now it is possible to suspend running process instances via Cockpit. However I also want to clear historic (=already terminated) process instances.

Background: we want to go into production and I want to get rid of all test data and all "old" process instances.

If I have to delete the data in the database directly, what are the tables I have to touch?

Cheers

Frank

[Daniel Meyer]

Hi Frank,

the history service provides delete methods such as

historyService.deleteHistoricProcessInstance(...)

Regards,

Daniel

[j...@atlas.cz]

We would like to delete all process instances finished more then half year ago.
Is only officially supported way to query for them and then to call historyService.deleteHistoricProcessInstance one by one?

thank you
best regards
jano

[Ingo Richtsmeier]

Hi jano, 

depending on your database, maybe its worth to have a look at this blog post: http://blog.camunda.org/2015/07/how-to-clean-uppurge-camunda-history.html

Kind Regards, Ingo

[webcyberrob]

This raises an interesting opportunity. Should there be a set of housekeeping processes available. Examples include purge history, purge deployments etc... Hence an admin could run them when necessary. Ideally I would configure them to run weekly during a quiet time and purge data more than 18 months old etc. hence these processes nibble away at the data rather than big extract like processes. Perhaps this could become a community project - Camunda Housekeeping...

[Bernd Rücker (camunda)]

You volunteer? ;-)

By the way - our best practice is: Write Java code to query process
instances finished X days ago (typical values 3-6 months - depending on
business requirements), iterate over them and delete all history entries
accordingly.
@Rob: Could be done in an own process - but could also be plain Java.
Depends if you already have a timing mechanism in place (then you might just
use this) - otherwise leverage the camunda start timer event.

If you have huge numbers of entries this might be too slow, then use SQL /
Stored procedures to do the job directly on the database. See
http://blog.camunda.org/2015/07/how-to-clean-uppurge-camunda-history.html.

-----Ursprüngliche Nachricht-----
Von: camunda-...@googlegroups.com
[mailto:camunda-...@googlegroups.com] Im Auftrag von webcyberrob
Gesendet: Donnerstag, 16. Juli 2015 01:12
An: camunda-...@googlegroups.com
Betreff: [camunda-bpm-users] Re: Clear all old/terminated process instances

--
You received this message because you are subscribed to the Google Groups
"camunda BPM users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to camunda-bpm-us...@googlegroups.com.
To post to this group, send email to camunda-...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/camunda-bpm-users/d8878275-35c9-47bd-b2c5-cb7f83f4983a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[webcyberrob]

Thanks Bernd,

Just curious. If best practice is to iterate via java code, why would you not 'iterate' using a multi-instance process? I guess this is where Im coming from with a generic housekeeping process. Perhaps it could be iterate over each deployment. For each deployment, iterate over completed instances. For each instance, if older than threshold, then delete. If the process model included an attribute that captured the 'dwell' time in history, then the one housekeeping process would work well for all deployed processes.

My only concern with this approach is it could put load on the database tier. If that was the case, partitioning the tables and a bulk purge via dropping a partition  would be more effective.

regards

Rob

[Bernd Rücker (camunda)]

Basically the reason is to keep it as simple as possible – and to use only components helping you to solve the problem at hand easier. For daily clean-up jobs I think a Cron/Quarz/EJB-Timer solution is perfectly fine  - as you do not use any special features of the process engine (the only features I see is the scheduler and maybe the logging/history). But only if you have scheduler-infrastructure already in place (then you normally already have control over which job run at which time, suspending them, seeing logs, ..). If you do NOT have this in place yet it is pretty nice to use the start timer mechanism for it.

 

Make sense?

To view this discussion on the web visit https://groups.google.com/d/msgid/camunda-bpm-users/c6a27081-ff2f-4144-9305-6036708e006c%40googlegroups.com.

[webcyberrob]

Hi Bernd,

I think an ideal solution would be a suite of housekeeping processes maintained as part of the engine. Rationale: you guys know the schema as well as anyone and thus know how to purge.

To your point, if I had an operations job scheduler, I may choose to initiate the housekeeping processes from my job scheduler as I centralize operations. If I don,t have a job scheduler, I use a timer start process in the engine to trigger the maintenance processes.

Does that sound like a good compromise?

regards

Rob

[Bernd Rücker (camunda)]

Hi Rob.

 

To your point, if I had an operations job scheduler, I may choose to initiate the housekeeping processes from my job scheduler as I centralize operations. If I don,t have a job scheduler, I use a timer start process in the engine to trigger the maintenance processes.

Does that sound like a good compromise?

 

Yes – it does. Exactly what I had in mind.

 

I think an ideal solution would be a suite of housekeeping processes maintained as part of the engine. Rationale: you guys know the schema as well as anyone and thus know how to purge.

 

The reason I am a bit reluctant here is that there are a lot of configuration possibilities in customer scenarios (like configuring the history level, customize history events, add own indices, …). I think such housekeeping processes could only make a best guess maybe not working optimal in high load scenarios where at some point you are more efficient if you do deletion of data directly on the database (tuned for the specific database).

 

But I agree that an community extension without the requirement of run in each and every possible situation would be already a great step in the right direction :-)

 

Cheers

Bernd

To view this discussion on the web visit https://groups.google.com/d/msgid/camunda-bpm-users/2f648550-5597-4f3c-9fe8-42c13b792744%40googlegroups.com.