New gridtk features

[Manuel Günther]

I recently implemented two new features into gridtk:

1. The first feature (which I actually implemented second) allows to add time stamps into the log files, when the job is started and when it finished.

   The time stamp including command line is written into the error log file (the standard way of logging with python).

   It relies on the -vv option during submission of jobs (in grid mode) or during running the local scheduler (in local mode).

  

2. I implemented a feature that allows to run the script in a given directory, instead of in the current directory.

   You can use the --exec-dir option during submission of the jobs.

   Theoretically, this feature should be applicable to both modes (grid or local).

Notes:

A. Unfortunately, I was not able to test the new functionality in the grid, as we don't have an SGE here.

B. As I had to modify the API of the Job model inside gridtk, the new gridtk won't work with the old sql databases, and vice versa.

I haven't yet pushed the new version of gridtk, as I wanted first to have your opinion about the changes (particularly the API change) and second I need someone to test the new functionality in the grid. Any volunteers?

[Tiago Freitas Pereira]

Dear Manuel,


Thanks for the updates.

I can test here in our side.

Cheers

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Tiago Freitas Pereira]

Dear Manuel,

I have been testing the new gridtk in our grid,
Well the new features are working fine and nothing is broken :-)

Thanks for the updates

Cheers

--

Tiago

[Manuel Günther]

Since I am breaking the API anyways, I added a new and long-waited feature: time stamps.

In the latest GitHub version, you should now be able to get timing information for three types of time stamps: submission time, start time (including wait time) and end time (including execution time), via using the --print-times (or shortly -t) option of jman ls

Enjoy!

[Manuel Günther]

Alright, version 1.3.0 of gridtk is now available. Note that it is not compatible with the old databases, as we now have new columns in the SQL file. 

Unfortunately I don't know how to tell SqlAlchemy that it should just ignore columns that it cannot find in the submitted.sql3... There seem to be some database migration methods, but I am not sure if it it worth investigating in that. If everything fails, you can get a local installation of an old GridTK version to read the old databases...

Let me know how you like the new features...

Enjoy!

Manuel