[Condor-users] R and condor

[Paul Hewson]

Hello,
(I've already posted this on the R-help list, apologies for cross-posting)

I would be extremely grateful if anyone is able to provide any (rather obscure) advice on using R with Condor. Essentially, it seems possible here to use condor with stand alone executables, but I'm getting stuck trying to submit batch jobs which software on the target machine.

think I'm following instructions (Xianhong Xie,R News 5(2) 13-15 http://cran.r-project.org/doc/Rnews/Rnews_2005-2.pdf) correctly, but my job just stays held in the queue (for days / months). I've checked condor_status to make sure there are plenty of machines available, but can't see any way to attack the problem of the non-running jobs.

I did wonder whether I should compile some small stand-alone executable to make sure that runs OK, but even then I'm not sure where I would go next. I have exhausted local advice on the matter.

For what it's worth, the condor script (and the even simpler R script) I'm working with at the moment are here:

Condor script file (submitted using condor_submit):

## condor script starts here:
Universe = vanilla
Executable = C:\Program Files\R\R-2.5.1\bin\Rterm.exe
Getenv = true
Arguments = vanilla
Input = test0.R
Error = test0.err
Log = test0.log

Output = test.out
Queue

## R script file (test0.R)
pdf("test1.pdf", width = 6, height = 6)
plot(c(1:10), c(1:10))
dev.off()

I really would be grateful for any suggestions, hints, comments, sample working scripts etc.

Best

Paul

-=-=-=-=-=-=-=-=-=-=-=-=
Paul Hewson
Lecturer in Statistics
School of Mathematics and Statistics
University of Plymouth
Drake Circus
Plymouth PL4 8AA

tel (01752) 232778 (Campus)
tel (01752) 764437 (Tamar Science Park)
fax (01752) 232780

email: paul....@plymouth.ac.uk
web: http://www.plymouth.ac.uk/staff/phewson
-=-=-=-=-=-=-=-=-=-=-=-=

______________________________________________
R-h...@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-use...@cs.wisc.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/

[Torrin Jones]

What does condor_q -analyze say?

Have you tried running jobs that don't use R?

 

[Jaime Frey]

You say that your job stays held in the queue. If it's in the Held
state, Condor won't try to run it. 'condor_q -hold' will tell you why
it's held and condor_release will tell Condor to try running it again.

If your job is not in the Held state, check the user log (test0.log)
to see if Condor has tried executing it. It's possible that Condor is
repeatedly failing to run it.

Thanks and regards,
Jaime Frey
UW-Madison Condor Team

[DeVoil, Peter]

Paul,
The command line args for R are a bit obscure. To get started, write
your job as a small dos batch file and get it running on your machine.
Then worry about running it on condor. Your batch file will be something
like (use % substitutions later)
---
"C:\Program Files\R\R-2.5.1\bin\Rterm.exe" --slave --vanilla < test0.R
--
This should run from the command line.

Next tell condor about it:
---
Universe = vanilla
Executable = test0.bat
transfer_input_files = test0.R
Error = test0.err
Output = test0.out
Log = test0.log
Queue
--

You should see some progress now..
P

Output = test.out
Queue

Best

Paul

********************************DISCLAIMER****************************
The information contained in the above e-mail message or messages
(which includes any attachments) is confidential and may be legally
privileged. It is intended only for the use of the person or entity
to which it is addressed. If you are not the addressee any form of
disclosure, copying, modification, distribution or any action taken
or omitted in reliance on the information is unauthorised. Opinions
contained in the message(s) do not necessarily reflect the opinions
of the Queensland Government and its authorities. If you received
this communication in error, please notify the sender immediately and
delete it from your computer system network.

[Paul Hewson]

Hello,
Thanks for this. Sorry, I was a bit sloppy with my nomenclature. Essentially, I can't get beyond the idling stage - jobs being rejected for "unknown" reasons.

I now have sample scripts that run on other condor setups, and assuming that I've correctly changed the path/to/R they are sufficiently simple I can't see why they won't run here.

Best

Paul

[Scillieri, John]

(ALSO POSTED ON THE R MAILING LIST)

Paul,

This is what some of our condor files look like. The example script
shown
goes through and cleans out our log directory (hence the name). The key
aspects are the 'should_transfer_files=No' and
'transfer_executable=False'. The Rscript and Rterm exe's are just small
wrappers that call into the R DLL. It's somewhat of a pain transferring
them all around (unless you don't have a shared file system).

--- CleanCondorLogs.condor -----------------------

JobName = CleanCondorLogs

LogHeader = $(JobName).$(Cluster).$(Process).$$(Machine)

input = <path to r script>/CleanCondorLogs.R
output = <log directory here>$(LogHeader).out
Error = <log directory here>$(LogHeader).err
log = <log directory here>/CleanCondorLogs.log

executable = $ENV(SystemRoot)\system32\cmd.exe
arguments = /Q /C RunR.bat

transfer_executable = False
should_transfer_files = No
getenv = True
environment = "TMP=C:\temp TEMP=C:\temp"
run_as_owner = true
Universe = vanilla

--------------------------------

The RunR.bat file looks like this:

------- Run.R.bat --------------

"C:\R\R-2.6.1\bin\Rscript.exe" --no-save --no-restore --no-init-file
--no-environ -

--------------------------------

We use the Run.R.bat file because it lets us not only mount required
network drives in Condor, but also upgrade our R as we see fit without
having to change 100+ condor files to point to the new path to R.

Hope that helps, let me know if you still run into trouble.

John

Best

Paul

>>> This e-mail and any attachments are confidential, may contain legal, professional or other privileged information, and are intended solely for the addressee. If you are not the intended recipient, do not use the information in this e-mail in any way, delete this e-mail and notify the sender. CEG-IP1