Migration to GitHub and other important changes
the labscript suite
Dear labscript suite users—current, and future.
We have completed migrating the labscript suite codebase from BitBucket to GitHub! The repositories on BitBucket will be deleted at the end of June. All issue tracking and development will now be done on GitHub. The repositories are now at github.com/labscript-suite.
We have also made it much easier to install and upgrade the labscript suite by releasing source and built distributions on the Python Package Index (PyPI) and Anaconda Cloud, which we outline below.
Installing the labscript suite
The most exciting change for current and new labscript suite users accompanying the migration to GitHub is the distribution of labscript suite components as Python packages on PyPI and Anaconda Cloud.
This makes it far easier to get started using the labscript suite, as you no longer require a Mercurial or Git installation (or any knowledge of version control software); components can be installed and upgraded using:
pip: the standard package manager common to all Python distributions; or
conda: a binary package and environment manager, part of the Anaconda Python distribution.
For further information, please see our documentation, which includes information about both regular and developer (editable) installations of the labscript suite.
Should you wish to keep using an editable installation (most similar to our previous installation method, and ideal for those with custom modifications to the labscript suite), we recommend undertaking the developer installation procedure, whereby pip sources the packages directly from GitHub.
Recent changes to the labscript suite
Upon migrating the code base to GitHub and publishing distributions on PyPI and Anaconda Cloud in April–May 2020, existing users should be aware of the following recent changes.
Profile directories
The labscript suite profile directory, containing application configurations, logs, and user-side code, is now located by default in the current user’s home directory, e.g. for a local user named wkheisenberg, this is:
C:\Users\wkheisenberg\labscript-suite on Windows.
~/labscript-suite or /home/wkheisenberg/labscript-suite on Linux and Mac OS X.
A typical structure of the profile directory is:
~/labscript-suite/
├── app_saved_configs/
│ ├── default_experiment/
├── labconfig/
├── logs/
└── userlib/
├── analysislib/
├── labscriptlib/
├── pythonlib/
└── user_devices/
This structure is created by calling the command labscript-profile-create in a terminal after installing labscript-utils (per the installation guide).
Note: As of labscript-suite/labscript-utils#37 an editable installation can be located within the labscript-suite profile directory.
Secure communication
Interprocess communication between components of the labscript suite is based on the ZeroMQ (ZMQ) messaging protocol. We have supported secure interprocess communication via encrypted ZMQ messaging since February 2019 (labscript-utils 2.11.0).
As of labscript-utils 2.16.0, encrypted interprocess communication will be the default. If you haven’t already, this means you’ll need to create a new shared secret (or pre-shared key) as follows:
Run python -m zprocess.makesecret from the labconfig directory.
Specify the path of the resulting shared_secret in your labconfig. For example:
[security]
shared_secret = %(labscript_suite)s/labconfig/zpsecret-09f6dfa0.key
Copy the same pre-shared key to all computers running the labscript suite that need to communicate with each other, repeating step 2 for each of them.
Treat this file like a password; it allows anyone on the same network access to labscript suite programs.
If you are on a trusted network and don’t want to use encrypted communication, you may instead set:
[security]
allow_insecure = True
Notes:
Steps 1 and 2 are executed automatically as part of the labscript-profile-create command. However, for multiple hosts, step 3 above must still be followed to ensure the same public-key is used by all hosts running labscript suite programs.
There is an outstanding issue with the ZMQ Python bindings on Windows (zeromq/pyzmq#1148), whereby encryption is significantly slower for Python distributions other than Anaconda. Until this issue is resolved, we recommend that Windows users on an untrusted network use the Anaconda Python distribution (and install pyzmq using conda install pyzmq).
Application shortcuts
Operating-system menu shortcuts, correct taskbar behaviour, and environment activation for the Python GUI applications (blacs, lyse, runmanager, and runviewer) is now handled by a standalone Python package desktop-app (per the installation guide). This currently supports Windows and Linux (Mac OS X support is forthcoming).
Source code structure (developer installation)
Existing users who move to a developer (editable) installation, please note the following structural changes to the labscript suite source code:
Each package has a top-level folder containing setup.py and setup.cfg used to build a distribution from source. The functional code base now resides in a subfolder corresponding to the name of the Python module, e.g. an editable installation might contain folders:
<path-to-your-labscript-installation>/
├── blacs/
│ ├── blacs/
├── labscript/
│ ├── labscript/
├── labscript-devices/
│ ├── labscript_devices/
├── labscript-utils/
│ ├── labscript_utils/
├── lyse/
│ ├── lyse/
├── runmanager/
│ ├── runmanager/
└── runviewer/
└── runviewer/
Package names (shared by repositories and top-level folders) are now hyphenated, e.g. labscript-devices and labscript-utils.
Module names remain underscored, e.g. labscript_devices and labscript_utils.
The mixing of hyphen and underscores is inelegant but conventional.
References to blacs are now lowercase.
As installation no longer requires a separate package, the repository formerly named ‘installer’ has been renamed to ‘labscript-suite’, and is a metapackage for the labscript suite (installing it via pip/conda installs the suite).
Versioning (developer installation)
Aside from the maintenance branches described in our workflow documentation, versions of the labscript suite packages are introspected at run-time using either the importlib.metadata library (regular installations) or setuptools_scm (developer installations). Thus any changes to an editable install will be traceable by local version numbers, e.g. editing the released version of a package with version 2.4.0 will result in 2.4.0dev1+gc28fe94, for example. This will help us diagnose issues users have with their editable installations.
BitBucket archive
All commit history and issues were preserved in the migration to GitHub, however some repository metadata (such as pull request discussions) could not be migrated directly. As such, we have created an archived copy of everything that was on BitBucket. This includes:
Issues (as they appear on BitBucket);
Pull requests discussions;
Commit comments for every labscript suite repository; and
Every public fork (as of 1st February, 2020).
This archive can be found at bitbucket-archive.labscriptsuite.org (this page can take some time to load for the first time). Copies of every public fork of our BitBucket repositories are at github.com/labscript-suite-bitbucket-archive. As these copies are for our own archival purposes, we will not be transferring ownership of these repositories back to their original owners.
What to do if you had custom code in a fork on BitBucket
labscript experiment scripts and lyse analysis scripts can be copied or moved to the new labscriptlib/analysislib folders. We deem these user-side code as they are not within the codebase of the labcript suite programs, and thus do not require a developer (editable) installation (see our installation documentation).
Customisations of the labscript suite will need to be reintegrated into the new package structure, using a developer (editable) installation. For example, to include your own custom labscript devices, you should undertake the developer installation procedure for the labscript-devices repository, and copy your custom or modified device files into the labscript_devices folder alongside the existing device files. Please also consider contributing these back to the main project by pushing them to your fork and issuing a pull request!
The procedure for migrating customisations of other components will depend on how up-to-date your fork is. Please open a thread on the mailing list to discuss with us how to migrate your custom features and/or how to contribute them back to the base labscript suite repositories.
Migrating other repositories to GitHub
Should you have other repositories on BitBucket such as labscriptlib, analysislib, userlib, or labconfig (or any project unrelated to the labscript suite) we strongly suggest using the tools we developed to migrate the labscript suite to GitHub. These are philipstarkey/bitbucket-hg-exporter and chrisjbillington/hg-export-tool which can be used together. See the documentation of those projects for further details.
Contributing to the labscript suite
We are very grateful for all the contributions users have made in the past decade to make the labscript suite the most widely used open-source experiment control and automation system in quantum science. These include development, suggestions, and feedback, and we look forward to this continuing on GitHub.
Issue tracking
The issue tracking on GitHub is very similar to BitBucket, with the added advantage that you can add inter-repository issue references, e.g. referring to labscript-suite/runmanager#68 in any issue or pull request will link to the corresponding issue. We have imported all issues from the BitBucket repositories into the GitHub repositories. This import is not perfect (as each comment is now posted by Phil Starkey) but the comments have been modified to contain the original author attribution. We have also updated all links to files, pull requests, issues, and commits so that they point to the equivalent GitHub location and/or the archived copy of the data (as discussed below).
Please use the GitHub issue tracker (rather than the mailing list) for:
Reporting bugs (when something doesn’t work or works in a way you didn’t expect);
Suggesting enhancements: new features or requests;
Issues relating to installation, performance, or documentation.
For advice on how to use the existing functionality of the labscript suite, please use our mailing list.
Request for developers
We would like to reaffirm our invitation for users to directly contribute toward developing the labscript suite. We have established a separate discussion forum on Zulip for discussing development direction and design. If you are interested in being a part of these discussions, and/or testing and merging pull requests, please reach out to us.
Further guidance on contributing—including the branching model we use, and the procedure for issuing pull requests—can be found in our documentation.
Warm regards,
The labscript suite core development team