running sas jobs on wrds cloud

[joost impink]

Hi, I'm forwarding this post:http://www.wrds.us/index.php/forum_wrds/viewthread/623/

Before the move to the WRDS cloud, I was able to submit sequential programs like this:
nohup sas program1.sas; sas program2.sas; sas program3.sas &

Where each program would start as soon as the earlier one finished.

Is there a way to do the same thing now using the WRDS cloud and qsas?  I don’t want to run these programs in parallel - they need to run sequentially.

Thanks

[christoph...@gmail.com]

According to the WRDS Cloud Manual (https://wrds-web.wharton.upenn.edu/wrds/support/Accessing%20and%20Manipulating%20the%20Data/_009WRDS%20Cloud%20Manual.cfm) it seems like you should put your commands into a script, then run the script using the 'qsub' command.

nohup is no longer necessary with the qsub command, according to this web page (https://wrds-web.wharton.upenn.edu/wrds/support/Accessing%20and%20Manipulating%20the%20Data/_011Transitioning%20to%20the%20WRDS%20Cloud.cfm)

Chris Green

[john.mo...@gmail.com]

I understand that the nohup option isn’t necessary any longer, nor is the & at the end.

I’ve read the manual, but still don’t understand what it means to put the commands into a script and run qsub.  Suppose I have 2 programs:
p1.sas
p2.sas

and I want p2.sas to run after p1.sas has finished. 

What do I do? 

I tried the following:
Create a text file called batch.sh

Edit that file to include only the following lines:

#!/bin/bash
#$ -cwd
sas p1.sas
sas p2.sas

I then submitted this with “qsub batch.sh”

p1.sas and p2.sas are programs that I’ve run before individually, and I know they work.  But this batch file doesn’t seem to be working - the expected datasets haven’t been created.  When I type in qstat to check the progress, the program seems to be still running, even though the underlying p1 and p2 sas programs would have been finished in a fraction of the time that it’s taken so far. (I ended up killing the batch job without it ever completing.)

[christoph...@gmail.com]

Well, you're submitting your job to queue, so sometimes it can take a few minutes to run depending on cluster load. Hard to say if that's your problem.

Some debugging ideas:

1) Try replacing p1.sas with a very simple sas script, and commenting out 'sas p2.sas' for now, so that you are just trying to run the simple sas script in batch. Does this batch job run to completion?

2) I think something in the qstat output will tell you the status of the job, i.e., if it's queued or running. Don't know that off the top of my head, but it's probably in the cloud manual or the Grid manual.

3) After submitting a job, see if anything is being written to the corresponding log file. You can use 'tail -f p1.log' to view the log file for p1.sas as it is written.

Chris