Where do you run your main process?
15 views
Skip to first unread message
John Lavoie
Jul 10, 2017, 11:41:18 AM7/10/17
to cascading-user
Hi everyone,
We've been growing unhappy with the design of how we spin up our jobs and are considering making some changes. I'm curious what everyone else does. The job process is the only single-point-of-failure in our pipeline and we're looking for more robustness as well as more decoupling and dynamic deployment capabilities.
Our app exposes a web service where users can submit requests for processing with various options. We take those incoming requests and spawn new processes to execute the cascading workflow on the same app server that handles the request. If there is any hiccup with the job process the entire workflow will fail. And with some jobs running for multiple hours, restarting, even with checkpoints, is expensive and time consuming.
We are considering moving the the main process that starts our cascading workflow off of the app servers to somewhere else. Where do you run yours?
John Lavoie
We've been growing unhappy with the design of how we spin up our jobs and are considering making some changes. I'm curious what everyone else does. The job process is the only single-point-of-failure in our pipeline and we're looking for more robustness as well as more decoupling and dynamic deployment capabilities.
Our app exposes a web service where users can submit requests for processing with various options. We take those incoming requests and spawn new processes to execute the cascading workflow on the same app server that handles the request. If there is any hiccup with the job process the entire workflow will fail. And with some jobs running for multiple hours, restarting, even with checkpoints, is expensive and time consuming.
We are considering moving the the main process that starts our cascading workflow off of the app servers to somewhere else. Where do you run yours?
John Lavoie
Ken Krugler
Jul 10, 2017, 6:33:49 PM7/10/17
to cascadi...@googlegroups.com
Hi John,
We run our workflows on a smaller “controller” in AWS.
We typically wrap jobs with a FlowRunner (class in cascading.utils project) that gives us a way to monitor the workflow execution running in a separate thread.
— Ken
--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at https://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/a0c6e2c0-5cdf-4156-b06d-2c92a2ed1d97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--------------------------
Ken Krugler
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
Reply all
Reply to author
Forward
0 new messages