Multiple dependent jobs - restarting in the middle

[Bart Kedryna]

Hello,

I'm trying to migrate some jobs from automate (formerly known as skybot) to rundeck. One of the nice features in automate is job dependency.- run job1, and if it succeeds, run job2, and then job3, etc. But if job2 fails, we can re-run it and then job3 will follow - or we can run job2 manually and then mark it as 'completed' to make it execute job3.

I created a 'master' job in rundeck which uses the 'jobref' option to execute other jobs:

sequence:
  commands:
    - jobref:
        name: job1

        useName: true

    - jobref

        name: job2

        useName: true

And so on. This works fine, but if job2 fails, the only way to continue the flow is to execute the failed job manually, wait for it to complete, then the next one, etc.

Is there some way to make rundeck do all this work? I don't want to re-execute job1, I just want to re-execute job2 and the other jobs that follow it.

Thanks

[rac...@rundeck.com]

Hi!

Process Automation (formerly “Rundeck Enterprise”) includes the Ruleset Strategy, it’s perfect for that use case.

Another approach is to use The Job Reference Step aimed at your target jobs. i.e:

We need to run three jobs: JobA, JobB, and JobC.

JobA:

- defaultTab: nodes
  description: ''
  executionEnabled: true
  id: 23a8fc6c-de30-42d1-b10a-9d46f017b3dc
  loglevel: INFO
  name: JobA
  nodeFilterEditable: false
  plugins:
    ExecutionLifecycle: null
  scheduleEnabled: true
  sequence:
    commands:
    - exec: echo "I am the Job A"
    keepgoing: false
    strategy: node-first
  uuid: 23a8fc6c-de30-42d1-b10a-9d46f017b3dc

JobB (containing an intentional failure for testing proposes):

- defaultTab: nodes
  description: ''
  executionEnabled: true
  id: 55020128-f630-45e9-a956-dfe4258a15ec
  loglevel: INFO
  name: JobB
  nodeFilterEditable: false
  plugins:
    ExecutionLifecycle: null
  scheduleEnabled: true
  sequence:
    commands:
    - exec: echo "I am the Job B"
    keepgoing: false
    strategy: node-first
  uuid: 55020128-f630-45e9-a956-dfe4258a15ec

JobC:

- defaultTab: nodes
  description: ''
  executionEnabled: true
  id: 3d5794ef-be09-4c2a-a8fd-106e4712593c
  loglevel: INFO
  name: JobC
  nodeFilterEditable: false
  plugins:
    ExecutionLifecycle: null
  scheduleEnabled: true
  sequence:
    commands:
    - exec: echo "I am the Job C"
    keepgoing: false
    strategy: node-first
  uuid: 3d5794ef-be09-4c2a-a8fd-106e4712593c

And the Parent Job, to run your target jobs (set the strategy: sequential and set “If a step fails” > “Run remaining steps before failing.” ):

- defaultTab: nodes
  description: ''
  executionEnabled: true
  id: cad1e7bf-84f8-4a24-8ea8-ff904fbf97af
  loglevel: INFO
  name: ParentJob
  nodeFilterEditable: false
  plugins:
    ExecutionLifecycle: null
  scheduleEnabled: true
  sequence:
    commands:
    - jobref:
        group: ''
        name: JobA
        nodeStep: 'true'
        uuid: 23a8fc6c-de30-42d1-b10a-9d46f017b3dc
    - jobref:
        group: ''
        name: JobB
        nodeStep: 'true'
        uuid: 55020128-f630-45e9-a956-dfe4258a15ec
    - jobref:
        group: ''
        name: JobC
        nodeStep: 'true'
        uuid: 3d5794ef-be09-4c2a-a8fd-106e4712593c
    keepgoing: true
    strategy: sequential
  uuid: cad1e7bf-84f8-4a24-8ea8-ff904fbf97af

Check the result here.

You can import these jobs to test in your environment, take a look at this.

Hope it helps!

​

[Bart Kedryna]

Thank you for the suggestion. However, that's not exactly what I was asking. I don't want job C to run when job B fails. I want job C to only run when job B succeeds, because it uses data that job B generates, so I have "keepgoing" set to false. All the jobs have to complete successfully every day, so when job B fails, I will investigate the problem and fix it. Once I fix the problem, I want to re-execute job B, and when it completes successfully, I want rundeck to automatically run job C, and then job D, then job E, and so on.

If the first one fails, then the solution is easy - run the parent job again.

But if job B fails, how can I complete the whole chain, assuming that I don't want to run job A again? Is there a way to do it automatically, or do I have to manually run all the jobs one after the other (which could take hours)?

[rac...@rundeck.com]

I see, my bad. Yeah, the easiest way is to use the Ruleset Strategy (only for Process Automation).

On Rundeck OSS, the "tricky way" to do that doing some script-fu, a job with an inline script that runs the jobs using rd-cli (or rundeck API via curl calls), checks the execution status and then decides if the JobC runs or not.  Basically creating a specific execution behavior via API/RD-CLI.

Regards!