MLOOP compatibility issues

[Alexander Chuang]

Hello labscripters,

I've been trying to set up mloop with lyse, but have been getting some errors when following the readme from https://github.com/rpanderson/analysislib-mloop. I have a standard labscript install using conda as from the docs (https://docs.labscriptsuite.org/en/latest/installation/regular-anaconda/), but there are is a labscript-utils version error that only leads to more errors when I suppress it. If anybody has any conda, pip or mloop experience they could share, I would really appreciate it!

When I run mloop_multishot.py , I get a VersionException below.

File "C:\Users\alexc\anaconda3\envs\py38\Lib\site-packages\lyse\analysis_subprocess.py", line 323, in do_analysis

exec(code, self.routine_module.__dict__)

File "C:\Users\alexc\labscript-suite\userlib\analysislib\example_apparatus\mloop_multishot.py", line 17, in <module>

check_version('labscript_utils', '2.12.5', '4.0')

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\labscript_utils\versions.py", line 264, in check_version

raise VersionException(msg.format(**locals()))

labscript_utils.versions.VersionException: labscript_utils 0.0.0 found. 2.12.5 <= labscript_utils < 4.0 required.

What's odd is if I run conda list  on my labscript env, I find:

labscript-utils 3.2.1 py_0 labscript-suite

and furthermore, if I see from the console that the labscript_utils module is imported from what seems to be the correct file, but again with conflicting __version__ .

(py38) C:\Users\alexc>python Python 3.8.13 (default, Mar 28 2022, 06:59:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32

Type "help", "copyright", "credits" or "license" for more information.

>>> import labscript_utils

>>> labscript_utils.__version__

'0.0.0'

>>> labscript_utils.__file__

'C:\\Users\\alexc\\anaconda3\\envs\\py38\\lib\\site-packages\\labscript_utils\\__init__.py'\

I've checked the labscript_utils folder and it appears to be having the bugfix with distutils, with "from packaging.version import Version" in ls_zprocess.py line 16, as from pull/86.

I've tried simply suppressing the VersionException, but this only leads me down a further rabbit hole of more errors. First a double import error

File "C:\Users\alexc\anaconda3\envs\py38\Lib\site-packages\lyse\analysis_subprocess.py", line 323, in do_analysis

exec(code, self.routine_module.__dict__)

File "C:\Users\alexc\labscript-suite\userlib\analysislib\example_apparatus\mloop_multishot.py", line 216, in <module>

import mloop_interface

RuntimeError: Double import! The same file has been imported under two different names, resulting in two copies of the module. This is almost certainly a mistake. If you are running a script from within a package and want to import another submodule of that package, import it by its full path: 'import module.submodule' instead of just 'import submodule.'

Path imported: C:\Users\alexc\anaconda3\envs\py38\Lib\site-packages\setuptools\_distutils\__init__.py

Traceback (first time imported, as setuptools._distutils):

------------

File "C:\Users\alexc\anaconda3\envs\py38\Lib\site-packages\lyse\analysis_subprocess.py", line 468, in <module>

qapplication.exec_()

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 46, in event

result = event.fn(*event.args, **event.kwargs)

File "C:\Users\alexc\anaconda3\envs\py38\Lib\site-packages\lyse\analysis_subprocess.py", line 323, in do_analysis

exec(code, self.routine_module.__dict__)

File "C:\Users\alexc\labscript-suite\userlib\analysislib\example_apparatus\mloop_multishot.py", line 216, in <module>

import mloop_interface

File "C:\Users\alexc\labscript-suite\userlib\analysislib\example_apparatus\mloop_interface.py", line 4, in <module>

from mloop.interfaces import Interface

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\mloop\interfaces.py", line 14, in <module>

import mloop.utilities as mlu

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\mloop\utilities.py", line 8, in <module>

import sklearn.preprocessing as skp

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\__init__.py", line 82, in <module>

from .base import clone

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\base.py", line 17, in <module>

from .utils import _IS_32BIT

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\utils\__init__.py", line 21, in <module>

from . import _joblib

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\utils\_joblib.py", line 7, in <module>

import joblib

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\__init__.py", line 113, in <module>

from .memory import Memory, MemorizedResult, register_store_backend

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\memory.py", line 32, in <module>

from ._store_backends import StoreBackendBase, FileSystemStoreBackend

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\_store_backends.py", line 15, in <module>

from .backports import concurrency_safe_rename

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\backports.py", line 22, in <module>

import distutils # noqa

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\labscript_utils\double_import_denier.py", line 57, in find_spec

spec = importlib.util.find_spec(fullname, path)

File "C:\Users\alexc\anaconda3\envs\py38\lib\importlib\util.py", line 103, in find_spec

return _find_spec(fullname, parent_path)

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\_distutils_hack\__init__.py", line 90, in find_spec

return method()

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\_distutils_hack\__init__.py", line 101, in spec_for_distutils

mod = importlib.import_module('setuptools._distutils')

File "C:\Users\alexc\anaconda3\envs\py38\lib\importlib\__init__.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\setuptools\__init__.py", line 8, in <module>

import _distutils_hack.override # noqa: F401

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\_distutils_hack\override.py", line 1, in <module>

__import__('_distutils_hack').do_override()

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\_distutils_hack\__init__.py", line 72, in do_override

ensure_local_distutils()

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\_distutils_hack\__init__.py", line 55, in ensure_local_distutils

importlib.import_module('distutils')

File "C:\Users\alexc\anaconda3\envs\py38\lib\importlib\__init__.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\_distutils_hack\__init__.py", line 90, in find_spec

return method()

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\_distutils_hack\__init__.py", line 101, in spec_for_distutils

mod = importlib.import_module('setuptools._distutils')

File "C:\Users\alexc\anaconda3\envs\py38\lib\importlib\__init__.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

------------

Traceback (second time imported, as distutils):

------------

File "C:\Users\alexc\anaconda3\envs\py38\Lib\site-packages\lyse\analysis_subprocess.py", line 468, in <module>

qapplication.exec_()

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 46, in event

result = event.fn(*event.args, **event.kwargs)

File "C:\Users\alexc\anaconda3\envs\py38\Lib\site-packages\lyse\analysis_subprocess.py", line 323, in do_analysis

exec(code, self.routine_module.__dict__)

File "C:\Users\alexc\labscript-suite\userlib\analysislib\example_apparatus\mloop_multishot.py", line 216, in <module>

import mloop_interface

File "C:\Users\alexc\labscript-suite\userlib\analysislib\example_apparatus\mloop_interface.py", line 4, in <module>

from mloop.interfaces import Interface

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\mloop\interfaces.py", line 14, in <module>

import mloop.utilities as mlu

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\mloop\utilities.py", line 8, in <module>

import sklearn.preprocessing as skp

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\__init__.py", line 82, in <module>

from .base import clone

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\base.py", line 17, in <module>

from .utils import _IS_32BIT

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\utils\__init__.py", line 21, in <module>

from . import _joblib

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\sklearn\utils\_joblib.py", line 7, in <module>

import joblib

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\__init__.py", line 113, in <module>

from .memory import Memory, MemorizedResult, register_store_backend

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\memory.py", line 32, in <module>

from ._store_backends import StoreBackendBase, FileSystemStoreBackend

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\_store_backends.py", line 15, in <module>

from .backports import concurrency_safe_rename

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\joblib\backports.py", line 22, in <module>

import distutils # noqa

------------

and on restarting the process

Traceback (most recent call last):

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\mloop\interfaces.py", line 90, in __init__

self.remaining_kwargs = mlu._config_logger(**kwargs)

File "C:\Users\alexc\anaconda3\envs\py38\lib\site-packages\mloop\utilities.py", line 81, in _config_logger

log.setLevel(min(file_log_level,console_log_level))

TypeError: '<' not supported between instances of 'str' and 'int'

Thanks,

Alex

[Johannes Schabbauer]

Hi Alex,

I suspect that the version error could be related to problems after python/setuptools changed their installation procedure. As I remember I also saw similar error, but could not track down the real cause or a good solution.

However, the following workarounds could make it work for you:

• For the version error, you can either just remove the check_version command in the mloop_mulishot.py file, or do the conda developer installation of labscript (there the version of labscript_utils is read correctly).
• For the double import error, you could add 'distutils' in the whitelist here (for this the developer installation might be better). In the pull request you linked, it seems to me that the package was removed from labscript, but for you tensorflow still imports the package. I am not aware if whitelisting the package as a workaround might cause other problems (So far it worked for me).
• For the last error I changed the console_log_level in the .ini file to the number of one logging level (e.g. 10 for DEBUG). I suspect this is caused by logging directly in M-LOOP, but did not investigate further.

The first two points might need some further investigation for a permanent solution.

Best wishes,

Johannes

[Alexander Chuang]

Hi Johannes,

Thanks for the tips! This fixed our issues completely. For reference, I just wanted to add that the first error is avoidable if one uses pip to install labscript-suite.

Best,

Alex

[dihm....@gmail.com]

Alex,

I'm glad Johannes was able to get you going again.

For everyone's information, I've just released a new version of labscript_utils (v3.2.2) that fixes the check_version error. Thank you for bringing it to our attention. It has apparently been a problem for over 2 years!

-David