On Mar 9, 10:17 am, Benjamin Sedoc <bse...@gmail.com
> I have written a python mig interface module that extends the
> module to simplify and speedup grid application development. It hides
> mRSL configuration and introduces a more direct user-to-resource job
> flow. Along with error detection to ensure a quick, more precise
> response, it also provides local mode execution, where a local backend
> of miglib emulates mig server operations. This makes debugging
> considerably faster.
> The goal is to have a standard grid developer API for broad use, so I
> interested in comments, requirements, ideas you may have...
> The code is available via:
> svn checkouthttp://migrid.googlecode.com/svn/trunk/user-projects/miginterface
It looks good so far, but I do have some questions and comments.
Why not a list or dictionary for output_files and such list arguments
They should allow multiple values with optional (src, dst) format.
You should always explicitly use $PYTHON in execute when using PYTHON-
X runtime envs. The ./my_script.py implicitly uses the python
interpreter path from the '#!/path/to/python' header in the script.
That is likely to cause problems with custom/multiple python
Did you consider merging e.g. job_info() and jobs_info() and just
always require a list as input? It looks like less redundancy and
maintenance without loss of functionality.
Similarly job_finished() may be a bit to specific. You could consider
a more general job_wait(job_id, state, timeout) that waits for the job
to reach the provided state but with a time out. Then users could
still use something like:
finished = job_wait(job_id, FINISHED, 0)
finished = False
to check a state without delay and use the function to wait for jobs
to finish in general.
Negative (or None) as timeout could disable timeout and thus would
also simplify your first job example.
The local mode idea is very cool!
Maybe consider using an OS independent
os.path.join(os.path.expanduser('~'), '.mig', 'miginterface')
path for log and other files currently in what sounds like hard coded /
tmp/ . If you consistently use the tempfile module for getting the tmp
path, this is less important.
typo on the wiki and in example3.py: parameter_sweet_script ->
...and on a 'personal taste' note I would prefer more telling names
instead of exampleX.py :-)
Technical note to the README file: Is it generated from a t2t file
(the table of contents looks and should be auto-generated)?
Feel free to include it somewhere for easy future updates if so.
README change: ensure anonymity -> allow anonymity
local mode: consider wrapping the job commands in a ulimit'ed job
script like we do for real jobs to prevent job problems with to small
resource request entries when later submitting to real grid.
You can search for ulimit in http://code.google.com/p/migrid/source/browse/trunk/mig/server/genjobscriptsh.py
Similarly such a job wrapper script would allow setting the MIG_X env
variables that some jobs may rely on.
local mode: you use subprocess and a unix specific 'ps' command to
check process status. Wouldn't it be possible to use only the
multiprocess functions like is_alive() or similar to make it simpler
and less platform dependent? The same applies for explicit kill
command vs terminate().
I have tested the basic examples and they ran without problems both
with local and grid execution - good job!