how to add branches to a branch point while tasks from existing branches are running as scheduler jobs
Nathan Schneider
Jon the Wise: Do not quit the current ducttape instance. Add the branches in the workflow, then start a new ducttape instance, and see whether it recognizes that it should leave the currently running task alone (should show WAIT FOR LOCK). This is an experimental feature, so be careful.
Me: I get the following error, and setting the global ducttape_enable_multiproc=true as advised does not help:
Exception in thread "main" java.lang.RuntimeException: It appears another ducttape process currently holds locks for this workflow and multi-process mode hasn't been explicitly enabled with 'ducttape_enable_multiproc=true'. If you think no other ducttape processes are running on this workflow, try using the 'ducttape workflow.tape unlock' command.
at ducttape.cli.ExecuteMode$.run(ExecuteMode.scala:59)
at Ducttape$$anonfun$main$6.apply(ducttape.scala:653)
at ducttape.cli.ErrorUtils$.ex2err(ErrorUtils.scala:59)
at Ducttape$.main(ducttape.scala:478)
at Ducttape.main(ducttape.scala)Jon: The error message is buggy, it should say ducttape_experimental_multiproc=true
Me: OK, now when I start the second ducttape instance it says it wants to delete the job that is currently running. It shows WAIT FOR LOCK for the tasks that depend on it that have not started yet.
Jon: *looks at code* The problem is this logic in exec/CompletionChecker.scala:
if (CompletionChecker.isBroken(taskEnv)) {
debug("Broken: " + task)
_broken += ((task.name, task.realization))
} else if (CompletionChecker.isLocked(taskEnv)) {
debug("Locked: " + task)
_locked += ((task.name, task.realization))
}It seems to be deciding that the task is broken because not all the requisite outputs are there, which is why it plans to delete the task. It should first check whether it is locked. However, the lock should prevent it from actually going through with the deletion.
Me: *verifies that this works* w00t!
