Runmanager remote API

[Rohit Prasad Bhatt]

Hi all,

I have two questions regarding runmanager remote API:

1. Can I create new globals using the API? I know that I can set the value of existing globals but could not find how I can create new globals.

2. How do I make the engage function of the runmanager API blocking? What I mean is the following. Consider a list of experiments where for every experiment a new "Experiment.py" file has to be generated, the values of globals are set and then we start the compilation by executing engage. Here is a dummy code.

################_start_################

exp_list=[exp_1, exp_2, exp_3]

for exp in exp_list:

    generate_exp_script()

    set_values_of_globals()

    remoteClient.engage() #remoteClient is runmanager object as defined in the API usage

################_end_################

I wanted to check if this code would wait for "remoteClient.engage()" to finish generating HDF files for all shots of exp_1 before proceeding to exp_2 in the loop? My experience tells me its does not. This is clear if I look into the script inside the HDFs for different shots. They are inconsistent with what I expect. However if I put a long enought wait time after the engage statement in the loop (by writing "time.sleep(10 or 20 etc...)") then all HDFs have correct script in them.

My explanation for this is that the "remoteClient.engage()" is not blocking. So after it executed in the loop for the first time a thread got launched which does the compilation. But the program has already moved to exp_2 in the loop and so has changed the "Experiment.py" file. Since the thread launched in the first iteration might still be running, it will now pack this wrong "Experiment.py" file into the shots of exp_1. And since exp_1 and exp_2 can have very different contents I need to make sure that the loop actually waits for the compilation of exp_1 to finish before moving to exp_2 and so on..

For now I just put in a wait by hand but I was wondering if there is a better alternative??

Regards,
Rohit Prasad Bhatt

[Zak V]

Hi Rohit,

1. At the moment the runmanager remote API doesn't support adding new globals. Although I haven't looked at it too closely, I suspect that it wouldn't be too hard to add support for that though if you're interested in trying it out.

Alternatively, a somewhat hacky workaround would be to edit the globals files directly using the functions in runmanager's `__init__.py`. In particular take a look at the functions `new_global()`, `get_value()`, `set_value()`, `get_units()`, and `set_units()` which can be imported as e.g. `from runmanager import new_global`. Compared to the functions in `runmanager.remote`, these functions have the disadvantage that they edit the globals files directly instead of through the runmanager GUI. In practice that means that the runmanager GUI won't update to show that you've added a new global or changed its value when using those functions. You'll have to close/open the globals file or close/open runmanager for the GUI to update.

2. You are correct that `runmanager.remote.engage()` isn't blocking, and there's currently no way to see if runmanager is currently compiling using `runmanager.remote` (https://github.com/labscript-suite/runmanager/issues/97). You are also correct that currently runmanager doesn't cache the labscript file (https://github.com/labscript-suite/runmanager/issues/54) so editing the labscript file will change the shots in the compilation queue that have yet to be compiled. Notably runmanager does cache the values of globals, so changing the values of globals after runmanager starts compiling shots won't have any effect on the shots that are in the compilation queue, even though they haven't been compiled yet.

A possible workaround would be to use a different labscript file for each of your experiments and use `runmanager.remote.set_labscript_file()` to point to the experiment's labscript file before each call to `engage()`. That will send the shot files to different directories by default, but you can explicitly set the desired shot output folder with `runmanager.remote.set_shot_output_folder()` if necessary. Runmanager can handle engage being called multiple times before it finishes compiling all of the shots from the first engage (though the abort button will have some issues, see https://github.com/labscript-suite/runmanager/issues/33) so you don't need to wait for one experiment's shots to finish compiling before requesting compilation of another experiment's shots.

Hopefully those workarounds work, let us know how it goes!

Cheers,

Zak

[Fred Jendrzejewski]

Hi everyone,

I restared the work on the topic here. In the meantime I have become quite a big fan of python tests and continous integration. So I tried to set up simple tests which are run on every PR here. They run locally, but it would seem that the intallation on the runner somehow fails:

- However, it would seem that I cannot find a way to "automatically" set open groups of global variables. 

- It would also seem as if I am not able to open the zprocess automatically.

Would anyone have an idea how to fix this ? Obviously, I would be happy to send a working solution of this upstream for blacs / runmanager if there is interest...

Greetings,

Fred