How to change Azkaban Failure Options at schedule execution

369 views
Skip to first unread message

wyu...@gmail.com

unread,
Oct 17, 2014, 11:56:59 PM10/17/14
to azkab...@googlegroups.com
Hi

I want to change Azkaban Failure Options from "Finish Current Running" to "Finish All Possible" at schedule execution.

I want to keep executing jobs as long as its dependencies are met.

I can change  Azkaban Failure Options at manual execution, but it seems that I can't change at schedule execution.

Could you teach me how to change Azkaban Failure Options at schedule execution?

mathew...@gmail.com

unread,
Sep 4, 2015, 7:53:25 AM9/4/15
to azkaban, wyu...@gmail.com
Hi,

I'm also very interested in being able to configure Failure Options for Scheduled workflows. We are trialing Azkaban for some of our Data Science workflows, but these need to run on a scheduled basis and we need to be able to source control all of the configuration.

At the moment, it seems like schedules need to be manually setup, and worse, they have several settings (including failureAction) for which the defaults are completely wrong for our needs (meaning we have to document a series of manual steps to setup).

Is it not possible to pass additional "Execute Flow" parameters (http://azkaban.github.io/azkaban/docs/latest/#api-execute-a-flow) to the "Schedule Flow" api call (http://azkaban.github.io/azkaban/docs/latest/#api-schedule-a-flow)? At least then we can script the schedule setup with the correct defaults.

Thanks,

Matthew Gwynne

wyu...@gmail.com

unread,
Sep 4, 2015, 8:37:38 AM9/4/15
to azkaban, mathew...@gmail.com
Hi

In order to resolve this problem, I implemented the Azkaban CLI tool(https://github.com/wyukawa/eboshi). The usage is the following.

eboshi addSchedule --url http://localhost:8081 --username azkaban --password azkaban --project azkaban_project --flow azkaban_flow --date '08/07/2014' --time '10,30,AM,JST' --period 1d --option '{"failureAction":"finishPossible"}'

In this case, I set failureAction to finishPossible.

So,
>Is it not possible to pass additional "Execute Flow" parameters (http://azkaban.github.io/azkaban/docs/latest/#api-execute-a-flow) to the >"Schedule Flow" api call (http://azkaban.github.io/azkaban/docs/latest/#api-schedule-a-flow)?

mathew...@gmail.com

unread,
Sep 4, 2015, 11:51:20 AM9/4/15
to azkaban, mathew...@gmail.com, wyu...@gmail.com
Perfect - thanks! Will take a look!
Reply all
Reply to author
Forward
0 new messages