Critique on job call iteration....

[Charles Scott]

I have a rundeck job that iterates on a variable list.   Each iteration executes a parameterized rundeck job (different params each time).  I decided to use the run command (I suppose it would be better to use the web api).

The run command is apparently asynchronous as I was getting an error showing that the forked job is already running (the previous apparently hadn't completed yet)

executing promote job for rundeckJobId: zap/promoteZapBuild
	12:42:23	ERROR RundeckCentralDispatcher: Job "promoteZapBuild" [21] is currently being executed (execution [293])
	12:42:23	ERROR RundeckCentralDispatcher: Server reported an error: Job "promoteZapBuild" [21] is currently being executed (execution [293])
	12:42:23	Error: Failed request to run a job: Job "promoteZapBuild" [21] is currently being executed (execution [293])
	12:42:23	error runnbuild zap/promoteZapBuild for build: zap-httpd-config

To verify that it is asynchronous, I injected a sleep to verify that it is indeed asynchronous.   

def runRundeckJob(rundeckJobId, build):

   print "executing promote job for build: " + build

   #return subprocess.call(["run", "-j", rundeckJobId, "--", "-archiveFile", "archive.zip", "-build", build, "-ciHost", "dev-ctier1.acme.com:8180", "-yumDir", "/data/yum/zap-repo", "createRepo", "false"])

   status=subprocess.call(["run", "-j", rundeckJobId, "--", "-archiveFile", "archive.zip", "-build", build, "-ciHost", "dev-ctier1.acme.com:8180", "-yumDir", "/data/yum/zap-repo", "createRepo", "false"])

   if status == 0:

      time.sleep(10)

      return 0

   else:

      return status

Is it possible to execute a job synchronously w/o being overly clever?

[Greg Schueler]

Hi Chuck,

yes, the "run" tool simply tells the server to execute the job, but does not wait for it to finish.

it has been requested before to add a flag like you suggest to make it not return until the job is done. (also to receive the output of the job as it happens).

You would have to write your own logic to wait until it is done.  you can test if the job is done by using the rd-queue to see if the Execution ID is among the running jobs...

it might be a useful standalone tool to query an execution's status and/or wait until it is done, (e.g. "rd-status -i ID --wait")

[Moses Lei]

Another topic that might come out of this error... a while ago we were talking about letting jobs be executed more than once simultaneously-- perhaps on a job-by-job basis as an option. What are people's thoughts on that at this point?

Moses

--
Moses Lei
[ Professional Services | DTO Solutions, Inc. ]
[ mobile: 703.901.5969 | e-mail: ml...@dtosolutions.com | aim/gtalk: ml...@controltier.com | yahoo: moseslei | windows live (msn): ml...@dtosolutions.com ]

On Thu, Jun 9, 2011 at 12:48 PM, Charles Scott <connar...@gmail.com> wrote:

[Chuck Scott]

I thought that as well. In my particular case, this job fits the category of a macro when called indirectly and a job when used directly.  

Sent from my iPhone

[Kaizer Sogiawala]

I think allowing jobs to be run more than once simultaneously should be made a non-default option. The jobs should be setup explicitly to enable it to allow running multiple times concurrently.

There is another axis for this - Should multiple users be allowed to run the same job simultaneously? (Consider two ops responding to same issue)

Would the behavior of job change it is is a dispatch or local?

[flu-]

Hi all,
I'd like to voice my opinion on parallel job execution as well. I
use rundeck to provision hosts when deploying clusters of machines.
Often times, I will need to create a staging cluster, a production
cluster, etc. Being able to execute the same job multiple times in
parallel would save me a _lot_ of time. I understand that given the
nebulous nature of rundeck jobs, this could expose the user to some
risk (say with shared resources). I'm of the opinion that a simple
warning along the lines of, "Use at your own risk" would suffice as
protection.
If we really wanted to get fancy with it, allowing a workflow to
explicitly contain parallel steps would also be great.

As for different operators running the same job simultaneously, I
believe that should also be supported, again with a warning that it is
already being run by another.

flu

[Kaizer Sogiawala]

I agree. A warning is a good idea. Something like, "An instance of this job has been started by "XYZ". 

Also, the command line can handle this by providing a flag. I feel the default should be to not run multiple instances of same job, regardless of the user.

[Chuck Scott]

I agree with this. In my use case it's about reducing redundant code. Essentially I lose the ability to modularize jobs where it may be called many times by another in a tight loop.

Since I could not run a job sequentially with synchronous run or allow job to be run concurrently, I had to reimplement a job that executes same code in the non reusable one.

Sent from my iPhone