Labscript-Suite installation after testing Opal Kelly

[Shuwei Jin]

Dear community,

we are a new group at the University of Heidelberg and we planned to use Labscript as the experimental control system. More particularly, we have installed, in the beginning, labscript in Python 3.7 virtual environment and would like to use an opal Kelly FPGA as a pseudoclock. 

I have encountered several issues during testing/installation and would appreciate very much if anyone can provide me with any guidance.

- About the Opal Kelly:

1) Differently from the Model XEM3001, we have bought a new model XEM6001 (Opal Kelly now sells the old model XEM3001 only in packages of 100 now). Would this new model also work as the labscript code now for generating the clock signal? I would guess yes but is there any updates that needs to be also implemented?

2) During the testing of the Opal Kelly I have followed very closely the discussions in this previous thread. In particular, I have downloaded the bit files from Github and put it in the correct path (For me it was: 'C:\Users\Jin\anaconda3\envs\py37\Lib\site-packages'). Furthermore, I also copied the API files provided by the FrontPanel software to the correct location as indicated here. 

However, when I run the connection table from runmanager, it still gives the error as indicated in this message ('module _ok cannot be found').

We thought it might be a problem related to the version of python as indicated previously (although SDK now seems to support every version of Python from 3.5 up... but you never know....). 

- About installation

We reinstalled labscript in python 3.6 virtual environment, but encountered the following errors during (re-)installation.

While creating the profile while executing on conda: labscript-profile-create we get the following error message:

Error processing line 1 of C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript-suite.pth:

  Traceback (most recent call last):

    File "C:\Users\Jin\anaconda3\envs\py36\lib\site.py", line 168, in addpackage

      exec(line)

    File "<string>", line 1, in <module>

    File "<string>", line 1, in <module>

    File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_profile\__init__.py", line 56, in add_userlib_and_pythonlib

      paths = config.get('DEFAULT', option).split(',')

    File "C:\Users\Jin\anaconda3\envs\py36\lib\configparser.py", line 800, in get

      d)

    File "C:\Users\Jin\anaconda3\envs\py36\lib\configparser.py", line 394, in before_get

      self._interpolate_some(parser, option, L, value, section, defaults, 1)

    File "C:\Users\Jin\anaconda3\envs\py36\lib\configparser.py", line 435, in _interpolate_some

      if "%" in v:

  TypeError: argument of type 'WindowsPath' is not iterable

Remainder of file ignored

zprocess warning: pyzmq is using bundled libzmq, which on Windows

is not built with the cryptography library libsodium. Encryption/decryption will be

slow. Use the conda pyzmq package for fast cryptography.

It seems that labscript cannot read the correct file path or so (Important to mention that we have indeed removed the labscript-suite folder during the previous installation). And more intriguingly, the program does create the folder the labscript-suite with many (but not all) folders inside it (For me the path is 'C:\Users\Jin\labscript-suite').

We tried to figure out where the problem comes but unfortunately without success.....

When we continued with the creation of the desktop app command desktop-app install blacs lyse runmanager runviewer it gaves the same error (argument of type 'WindowsPath' is not iterable). The short cuts are indeed installed though, but cannot be correctly launched.....

When I try to launch runmanager for example, I get the following error:

Traceback (most recent call last):

  File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\runmanager\__main__.py", line 56, in <module>

    import runmanager

  File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\runmanager\__init__.py", line 29, in <module>

    import labscript_utils.h5_lock

  File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_utils\h5_lock.py", line 93, in <module>

    connect_to_zlock_server()

  File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_utils\ls_zprocess.py", line 317, in connect_to_zlock_server

    client = ProcessTree.instance().zlock_client

  File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_utils\ls_zprocess.py", line 143, in instance

    config = get_config()

  File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_utils\ls_zprocess.py", line 98, in get_config

    shared_secret_file = labconfig.get('security', 'shared_secret')

  File "C:\Users\Jin\anaconda3\envs\py36\lib\configparser.py", line 800, in get

    d)

  File "C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_utils\labconfig.py", line 31, in before_get

    value = super(EnvInterpolation, self).before_get(*args)

  File "C:\Users\Jin\anaconda3\envs\py36\lib\configparser.py", line 394, in before_get

    self._interpolate_some(parser, option, L, value, section, defaults, 1)

  File "C:\Users\Jin\anaconda3\envs\py36\lib\configparser.py", line 435, in _interpolate_some

    if "%" in v:

TypeError: argument of type 'WindowsPath' is not iterable

We would be very happy, at this stage, if anyone can help us get pass this installation error so that we can proceed with the test for the Opal Kelly as the pseudoclock.....

Many thanks in advance!

Sonny greetings,

Shuwei Jin

[Chris Billington]

Hi Shuwei Jin,

The "TypeError: argument of type 'WindowsPath' is not iterable" is due to a bug in labscript specifically when running with Python 3.6, apologies.

If you make the changes in this pull request here:

https://github.com/labscript-suite/labscript-utils/pull/81/files

to the files

 C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_utils\labconfig.py

and

C:\Users\Jin\anaconda3\envs\py36\lib\site-packages\labscript_profile\__init__.py

That should resolve it.

I'm less familiar with the situation with the Opal Kelly, hopefully the version you've got works with Python 3.6. Though, Python 3.6 support ends at the end of this year, so it will start to become difficult to install and maintain Python 3.6 environments after that.

Regards,

Chris

--
You received this message because you are subscribed to the Google Groups "the labscript suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/e8f8aef1-528b-4edb-8acc-c28556089a81n%40googlegroups.com.

[Shuwei Jin]

Dear Chris,

Many thanks for your quick reply!

I was maybe a bit too quick about the procedure on testing the Opal Kelly and please allow me to go through my procedure when labscript suite is installed on python 3.8 virtual environment, as I have done in the very beginning. 

Following the guide instructions on the installation instruction, everything works well. When I create a profile with py38, I get a warning:

 zprocess warning: pyzmq is using bundled libzmq, which on Windows

is not built with the cryptography library libsodium. Encryption/decryption will be

slow. Use the conda pyzmq package for fast cryptography.

Nevertheless, everything still works smoothly when I created the desktop app and so on. Following the instructions described in this previous thread, I copied the connection table to the right location and can launch runmanager and BLACS normally without error. 

For testing the Opal Kelly, I have 

- downloaded the two bit files from Github and put it in the labscript_devices folder (For me it was 'C:\Users\Jin\anaconda3\envs\py38\Lib\site-packages\labscript_devices') together with the file CiceroOpalKellyXEM3001.py

and 

- copied the API python files from the FrontPanel software to the appropriate destination (For me this is 'C:\Users\Jin\anaconda3\envs\py38\Lib\site-packages\ok' The folder 'ok' I created myself with the two API files - '_ok.pyd' and 'ok.py'  - copied in it)

When I run my connection table in runmanager I do not get any error. It just says 'ready'.

While in BLACS I get the following error:

Exception in worker - Tue Sep 21, 16:18:26 :

Traceback (most recent call last):

  File "C:\Users\Jin\anaconda3\envs\py38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 556, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1166, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: DLL load failed while importing _ok: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\blacs\tab_base_classes.py", line 923, in mainloop
    results = func(*args,**kwargs)
  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\labscript_devices\CiceroOpalKellyXEM3001.py", line 443, in init
    from ok import ok
  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\ok\ok.py", line 17, in <module>
    _ok = swig_import_helper()
  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\ok\ok.py", line 16, in swig_import_helper
    return importlib.import_module('_ok')
  File "C:\Users\Jin\anaconda3\envs\py38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_ok'

Fatal exception in main process - Tue Sep 21, 16:18:26 :

Traceback (most recent call last):

  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\blacs\tab_base_classes.py", line 837, in mainloop
    next_yield = inmain(generator.send,results)
  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 88, in inmain
    return get_inmain_result(_in_main_later(fn, False, *args, **kwargs))
  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 150, in get_inmain_result
    raise value.with_traceback(traceback)
  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 46, in event
    result = event.fn(*event.args, **event.kwargs)
  File "C:\Users\Jin\anaconda3\envs\py38\lib\site-packages\blacs\tab_base_classes.py", line 536, in _initialise_worker
    raise Exception('Device failed to initialise')
Exception: Device failed to initialise

This looks very similar to the error message already mentioned previously in this thread. It seems that the module ok cannot be found. 
As previous also discussed, this could be due to an issue related to the python version. This was the reason why I installed then labscript in python 3.7 and later also 3.6 for testing but encountered other errors during installation..... (And @ Chris: Thanks for the info about the bug!)

At this point I wish to know if it is possible to have everything working in the python virtual env 3.8. 

I am not sure whether the previous error is due to the python version on which labscript is run or because I was using a newer FPGA board (Opal Kelly XEM6001 instead of Opal Kelly XEM3001).

I would be very happy if anyone can share any thoughts about solving the previous error!

Thanks in advance!

Best regards,

Shuwei Jin

[Philip Starkey]

Hi Shuwei Jin,

The import error for the OpalKelly library might be as simple as a missing empty __init__.py file in the folder with the other opal kelly library files. Without that the ok folder won't be treated as a Python module and so won't be importable. If that doesn't work, you should focus on trying to get the ok library importable from a standard Python terminal without worrying about labscript suite at all. If you have purchased an Opal Kelly development board, their support may be able to assist (since at that point the failure is not due to the labscript suite). The error message does hint at it being related to the version of Python you are using though. In that case, you might need to check with Opal Kelly support as to whether the version of Opal Kelly Front Panel you have installed is compatible with the version of Python you are using.

Library import issues aside - I have no idea if the firmware will be compatible with the XEM6001 board. I suspect at the very least the .bit files will need to be recompiled from the verilog in the repository. This firmware is not something we are particularly familiar with. I made a minor tweak to it many years ago, but it originates from the Cicero control system. Minimal support is there for people who might want to migrate from Cicero to the labscript suite (and thus already have the XEM3001 board), but we're not really pursuing further FPGA development ourselves. We'd be happy to include support for the XEM6001 though if you do manage to get it working!

If you find that getting the XEM6001 working is too hard or too time consuming, you might be interested in trying the PrawnBlaster, which only requires a $4USD investment.

I hope this helps! Let us know how you go.

Cheers,

Phil

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/f9c23ff7-29f1-490c-9f4c-2f828198bf9cn%40googlegroups.com.

--

Dr Philip Starkey

Senior Technician (Lightboard developer)

School of Physics & Astronomy

Monash University

10 College Walk, Clayton Campus

Victoria 3800, Australia

[Shuwei Jin]

Dear Philip,

Actually we also bought the Raspberry Pi board at the same time for testing and it is indeed much easier to get it working than the OpalKelly.....So probably we will stick with PrawnBlaster for now until further problems arise.....

If I have time, I will also take a closer look at the OpalKelly....

Thanks for your explanation and suggestion! It really helps us a lot!

Best regards,

Shuwei Jin