Installer problem with importlib_metadata and pkg_resources

[Lars Kohfahl]

Hey,

as I am not sure if this is a bug or just me being able to install labscript correctly I will post my problem here and did not create an issue on bitbucket:

I am currently setting up a new computer with labscript. Labscript shall be installed in a "labscript"-env with python 3.6. I have installed all necessary dependencies but then the installer process raises the following error:

All dependencies satisfied

Enter custom installation directory or press enter
(C:\labscript_suite):
Adding to Python search path (C:\Users\QC-Eval\Anaconda3\envs\labscript\labscript_suite.pth)
Copying files
Traceback (most recent call last):
  File "setup.py", line 527, in <module>
    function(*args)
  File "setup.py", line 419, in install
    make_labconfig_file(install_folder)
  File "setup.py", line 328, in make_labconfig_file
    from labscript_utils.labconfig import LabConfig, default_config_path
  File "C:\Users\QC-Eval\Labscript\installer\labscript_utils\__init__.py", line 49, in <module>
    from labscript_utils.versions import get_version, VersionException, check_version
  File "C:\Users\QC-Eval\Labscript\installer\labscript_utils\versions.py", line 196, in <module>
    check_version('importlib_metadata', '0.17', '2.0')
  File "C:\Users\QC-Eval\Labscript\installer\labscript_utils\versions.py", line 174, in check_version
    version = get_version(module_name, project_name)
  File "C:\Users\QC-Eval\Labscript\installer\labscript_utils\versions.py", line 142, in get_version
    version = _get_metadata_version(project_name, import_path)
  File "C:\Users\QC-Eval\Labscript\installer\labscript_utils\versions.py", line 85, in _get_metadata_version
    dists = finder.find_distributions(name=project_name, path=[import_path])
TypeError: find_distributions() got an unexpected keyword argument 'name'

usage:
  python setup.py install
  python setup.py uninstall [<path>]
  python setup.py build
  python setup.py dist
  python setup.py clean

For me this looks like find_distributions is the function from importlib_metadata/importlib.metadata but in reality python tries to use (pkg_resources.)find_distributions which has no keyword argument name.

Any suggestions?

The problem occurs with py3.6 and py3.7

[Lars Kohfahl]

I forgot to say that it seems to be a problem not occuring when installing in the anaconda base environment.

[Lars Kohfahl]

Just saw the latest update from Chris in labscript utils. Maybe this will resolve my problem. I will first update all which our branch is behind and see what happens.

[Lars Kohfahl]

Okay, the update solved the problem. Thanks for being so fast with your update. I would suggest to soon tag this version of labscript_utils and add it to the installer as new installations will always get the newest version of importlib_metadata. And if I have understood correctly it is the new version causing the problems.

Lars

[Fred Jendrzejewski]

I now get the following error if I try to run runviewer:

python -m runviewer

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/fredjendrzejewski/labscript_suite/runviewer/__main__.py", line 24, in <module>
    check_version('labscript_utils', '2.10.0', '3')
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/versions.py", line 206, in check_version
    _initialise()
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/versions.py", line 49, in _initialise
    'importlib_metadata', '0.17', '2.0', version=importlib_metadata.__version__
AttributeError: module 'importlib.metadata' has no attribute '__version__'

Is this related ?

Cheers,

Fred

[Chris Billington]

Hi Fred,

This is because of some changes in Python 3.8. The importlib_metadata package was made part of the standard library, and dropped its __version__ attribute.

The pull request to add Python 3.8 compatibility to check_version() is here:

https://bitbucket.org/labscript_suite/labscript_utils/pull-requests/94

I haven't merged it yet. There may be other issues running with Python 3.8 as well. This one:

https://bitbucket.org/labscript_suite/blacs/pull-requests/76

Has been merged into BLACS, so you will want to update to get that too. Please report bugs if you hit any others!

It might be the case that not all of our dependencies have Python 3.8 builds in the conda repositories yet for all platforms. h5py seems to only have win-64 and linux-64 builds for example. pyzmq is missing a win-32 build as well. If support is missing for anything on your platform, you can either stick to Python 3.7 for a bit longer, or resort to installing the packages from pip (which can be messy since conda is not as good at managing packages installed by pip).

In the not too distant future, labscript suite components will be installable as conda packages. So it will be a bit more obvious whether we support a new Python version or not since we simply will not have built new packages for the new Python version until we've done some basic testing and merged the required changes.

-Chris

--
You received this message because you are subscribed to the Google Groups "The labscript suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/d0d22695-44bf-46b6-a5bf-df9771b72bd4%40googlegroups.com.

[Fred Jendrzejewski]

Thanks for the hints.

In the meantime I downgraded to python3.7 and then zprocess 2.18.1 started acting up again. I wanted to downgrade to the famous 1.3.3 version, but this does not seem to be compatible. The error it raises is:

 File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')
zprocess.utils.TimeoutError: Could not connect to server: timed out

This error persisted even if I ran

python -m zprocess.lock

in another terminal ...

Cheers,

Fred

On Tuesday, December 3, 2019 at 3:52:16 PM UTC+1, Chris Billington wrote:

Hi Fred,

This is because of some changes in Python 3.8. The importlib_metadata package was made part of the standard library, and dropped its __version__ attribute.

The pull request to add Python 3.8 compatibility to check_version() is here:

https://bitbucket.org/labscript_suite/labscript_utils/pull-requests/94

I haven't merged it yet. There may be other issues running with Python 3.8 as well. This one:

https://bitbucket.org/labscript_suite/blacs/pull-requests/76

Has been merged into BLACS, so you will want to update to get that too. Please report bugs if you hit any others!

It might be the case that not all of our dependencies have Python 3.8 builds in the conda repositories yet for all platforms. h5py seems to only have win-64 and linux-64 builds for example. pyzmq is missing a win-32 build as well. If support is missing for anything on your platform, you can either stick to Python 3.7 for a bit longer, or resort to installing the packages from pip (which can be messy since conda is not as good at managing packages installed by pip).

In the not too distant future, labscript suite components will be installable as conda packages. So it will be a bit more obvious whether we support a new Python version or not since we simply will not have built new packages for the new Python version until we've done some basic testing and merged the required changes.

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscri...@googlegroups.com.

[Chris Billington]

HI Fred,

Which component is failing to connect (i.e. could I see more lines of the traceback)? It could be zlog (the logging server that starts automatically) instead of zlock. You might try opening process explorer (this one is great if you don't already use it - will show you the command line of the processes etc) and killing Python processes to ensure another copy of zlog or zlock is not running.

The line to run to manually start zlock using settings configured from labconfig is:

python -m labscript_utils.zlock

Though python -m zprocess.zlock or python -m zprocess.locking (the latter is now an alias for the former) should both work fine if you are running zlock on the default port and not using encryption (labscript_utils.zlock is just a wrapper that sets these).

One frustrating issue I've been seeing here at NIST is that some computers are only able to be looked up by hostname for IPV6, the local DNS is not giving IPv4 addresses. This causes some computers to fail to connect to each other when given hostnames (though "localhost" should be fine...). I enabled IPV6 in zprocess to fix the issue, but hit some other bugs and reverted it for the time being. So the workaround for that (if you're running zlock on a different computer to everything else which it sounds like you're not) is to specify the zlock host in labconfig as an IPV4 address instead of a hostname. I hope to get zprocess working properly with IPV6 soon.

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/74d6f3aa-3faf-4140-9b1d-34c9212dca71%40googlegroups.com.

[Fred Jendrzejewski]

And here is the error message in all its gory detail:

(lss3) fredjendrzejewski@MacBook-Air-5 labscript_suite % python -m runviewer

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 343, in ensure_connected_to_zlog
    client.ping(timeout=0.05)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 116, in ping
    self._send(b'hello', timeout=timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 90, in _send
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 71, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout=timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')

zprocess.utils.TimeoutError: Could not connect to server: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/fredjendrzejewski/labscript_suite/runviewer/__main__.py", line 58, in <module>
    logger = setup_logging('runviewer')
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/setup_logging.py", line 37, in setup_logging
    ensure_connected_to_zlog()
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 353, in ensure_connected_to_zlog
    client.ping(timeout=15)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 116, in ping
    self._send(b'hello', timeout=timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 90, in _send
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 71, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout=timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')

zprocess.utils.TimeoutError: Could not connect to server: timed out

On Tuesday, December 3, 2019 at 4:43:36 PM UTC+1, Chris Billington wrote:

HI Fred,

Which component is failing to connect (i.e. could I see more lines of the traceback)? It could be zlog (the logging server that starts automatically) instead of zlock. You might try opening process explorer (this one is great if you don't already use it - will show you the command line of the processes etc) and killing Python processes to ensure another copy of zlog or zlock is not running.

The line to run to manually start zlock using settings configured from labconfig is:

python -m labscript_utils.zlock

Though python -m zprocess.zlock or python -m zprocess.locking (the latter is now an alias for the former) should both work fine if you are running zlock on the default port and not using encryption (labscript_utils.zlock is just a wrapper that sets these).

One frustrating issue I've been seeing here at NIST is that some computers are only able to be looked up by hostname for IPV6, the local DNS is not giving IPv4 addresses. This causes some computers to fail to connect to each other when given hostnames (though "localhost" should be fine...). I enabled IPV6 in zprocess to fix the issue, but hit some other bugs and reverted it for the time being. So the workaround for that (if you're running zlock on a different computer to everything else which it sounds like you're not) is to specify the zlock host in labconfig as an IPV4 address instead of a hostname. I hope to get zprocess working properly with IPV6 soon.

-Chris

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/74d6f3aa-3faf-4140-9b1d-34c9212dca71%40googlegroups.com.

[Chris Billington]

Excellent. This shows that zlog is not running, so runviewer tries to start it, but then it is still not running. Could you try running:

python -m labscript_utils.zlog

And posting presumably what will be an error showing zlog failing to start?

I recently changed the way that zlog decides where to write its own log file (a bit meta - the logging server has a log file), so perhaps there is a bug with that.

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/a826cd47-de24-488a-a1c6-cc9d234af685%40googlegroups.com.

[Chris Billington]

Ah, I think I see the problem.

`labscript_utils.zlog` configures zlog to write its log file to `<labscript_suite_install_dir>/logs`. However, that directory does not exist yet if you've just upgraded. I configured elsewhere in labscript_utils to create the directory if it doesn't exist, but zlog hits the error first. In my testing I still had a zlog server running from before the update so it didn't crash, and by the time it was restarted the directory existed, so I never saw the error.

Assuming this is the issue you're hitting, I'll fix it now...

-Chris

[Chris Billington]

Oh, I spoke to soon, I misread the code. I did indeed tell it to make the directory if it doesn’t exist:

    if directory is not None:
        logpath = os.path.join(directory, '%s.log' % name)
        if not os.path.exists(directory):
            os.mkdir(directory)
    else:
        logpath = None

So I’m out of ideas again - will hopefully get some clue what’s wrong from any error message zlog gives you.

-Chris

[Fred Jendrzejewski]

So I ran this and it does not complain to much. It 'only' gives a warning:

python -m labscript_utils.zlog
[2019-12-03 20:00:22,712] WARNING Can't open or do not have permission to write to log file /var/log/zlog.log. Only terminal logging will be output.
[2019-12-03 20:00:22,712] INFO This is zlog server, running on tcp://0.0.0.0:7340

Not sure if this is good or bad ...

On Tuesday, December 3, 2019 at 5:05:51 PM UTC+1, Chris Billington wrote:

Excellent. This shows that zlog is not running, so runviewer tries to start it, but then it is still not running. Could you try running:

python -m labscript_utils.zlog

And posting presumably what will be an error showing zlog failing to start?

I recently changed the way that zlog decides where to write its own log file (a bit meta - the logging server has a log file), so perhaps there is a bug with that.

-Chris

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/a826cd47-de24-488a-a1c6-cc9d234af685%40googlegroups.com.

[Chris Billington]

Hi Fred,

Something is fishy! Zprocess up to 2.18.0 tries to log to /var/log/zlog.log, but zprocess 2.18.1 doesn't try to make a log file unless requested to by labscript_utils.zlog (which doesn't ask it to - the change for that hasn't been merged yet). So based on that warning, it looks like you're running zprocess less than 2.18.1. Also, in zprocess 2.16 the bind address changed from 'tcp://0.0.0.0' to 'tcp://*' (which mean the same thing except the latter includes IPv6). So it seems based on that log line that you're running zprocess < 2.16. What do you get if you run in Python:

import zprocess

print(zprocess.__version__)

print(zprocess.__file__)

Also, since you are able to start zlog manually, is runviewer able to start whilst it is running? Do you see any output in the zlog terminal saying that it received requests from runviewer?

You might have a broken zprocess install. If you run in a terminal:

cd /Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/ 

ls | grep zprocess

What do you see? You should see the following:

zprocess
zprocess-2.18.1-py3.8.egg-info

If there is more than one zprocess folder there, you might try running rm -rf zprocess* in that directory (this will remove all traces of zprocess) and installing zprocess again. If it is a broken install, this is the kind of thing I'm trying to avoid by moving to conda packages :). If you do delete all traces of pip-installed zprocess, you might consider trying my experimental conda package for zprocess:

conda install -c cbillington zprocess

There is one other possibility for a broken install, and this would be if you had run pip install --user zprocess. This would have installed zprocess to some subdirectory of your home directory, I don't know which. But print(zprocess.__file__) from above should reveal if this is the case.

We'll get to the bottom of this!

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/6c9e4427-965d-486b-9f5b-8b5efa0f11e4%40googlegroups.com.

[Fred Jendrzejewski]

On Tuesday, December 3, 2019 at 8:46:14 PM UTC+1, Chris Billington wrote:

Hi Fred,

Something is fishy! Zprocess up to 2.18.0 tries to log to /var/log/zlog.log, but zprocess 2.18.1 doesn't try to make a log file unless requested to by labscript_utils.zlog (which doesn't ask it to - the change for that hasn't been merged yet). So based on that warning, it looks like you're running zprocess less than 2.18.1. Also, in zprocess 2.16 the bind address changed from 'tcp://0.0.0.0' to 'tcp://*' (which mean the same thing except the latter includes IPv6). So it seems based on that log line that you're running zprocess < 2.16. What do you get if you run in Python:

import zprocess

print(zprocess.__version__)

print(zprocess.__file__)

>>> import zprocess
>>> print(zprocess.__version__)
2.18.1
>>> print(zprocess.__file__)
/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/__init__.py

I downgraded at some point to a version which is running in our lab machines. Sorry for that. Running zlog with 2.18.1 works better.

Also, since you are able to start zlog manually, is runviewer able to start whilst it is running? Do you see any output in the zlog terminal saying that it received requests from runviewer?

Yes. The output is this one:

[2019-12-03 20:50:43,618] INFO This is zlog server, running on tcp://*:7340
[2019-12-03 20:50:48,391] INFO [127.0.0.1] said hello
[2019-12-03 20:50:48,393] INFO [127.0.0.1] Access confirmed for /Users/fredjendrzejewski/labscript_suite/runviewer/runviewer.log
[2019-12-03 20:51:04,428] INFO Opening /Users/fredjendrzejewski/labscript_suite/runviewer/runviewer.log
[2019-12-03 20:51:04,428] INFO New client (total: 1) for /Users/fredjendrzejewski/labscript_suite/runviewer/runviewer.log
[2019-12-03 20:51:04,439] INFO [127.0.0.1] Client done (remaining: 0) with /Users/fredjendrzejewski/labscript_suite/runviewer/runviewer.log
[2019-12-03 20:51:04,440] INFO No more clients, closing /Users/fredjendrzejewski/labscript_suite/runviewer/runviewer.log

And then the client shuts down with:

2019-12-03 20:51:04,425 ERROR runviewer: Got an exception:

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 302, in connect_to_zlock_server
    client.ping(timeout=0.05)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 105, in ping
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 85, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')

zprocess.utils.TimeoutError: Could not connect to server: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)

  File "/Users/fredjendrzejewski/labscript_suite/runviewer/__main__.py", line 62, in <module>
    import labscript_utils.h5_lock
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/h5_lock.py", line 104, in <module>
    connect_to_zlock_server()
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 312, in connect_to_zlock_server
    client.ping(timeout=15)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 105, in ping
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 85, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')

zprocess.utils.TimeoutError: Could not connect to server: timed out

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 302, in connect_to_zlock_server
    client.ping(timeout=0.05)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 105, in ping
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 85, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')

zprocess.utils.TimeoutError: Could not connect to server: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)

  File "/Users/fredjendrzejewski/labscript_suite/runviewer/__main__.py", line 62, in <module>
    import labscript_utils.h5_lock
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/h5_lock.py", line 104, in <module>
    connect_to_zlock_server()
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 312, in connect_to_zlock_server
    client.ping(timeout=15)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 105, in ping
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlock/__init__.py", line 85, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')

zprocess.utils.TimeoutError: Could not connect to server: timed out

You might have a broken zprocess install. If you run in a terminal:

cd /Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/ 

ls | grep zprocess

What do you see? You should see the following:

zprocess
zprocess-2.18.1-py3.8.egg-info

I actually get:

zprocess
zprocess-2.18.1.dist-info

If there is more than one zprocess folder there, you might try running rm -rf zprocess* in that directory (this will remove all traces of zprocess) and installing zprocess again. If it is a broken install, this is the kind of thing I'm trying to avoid by moving to conda packages :). If you do delete all traces of pip-installed zprocess, you might consider trying my experimental conda package for zprocess:

conda install -c cbillington zprocess

So weird stuf that broke even more things happened there. I will not post this now as I have to get the choreography of what I did right ...

There is one other possibility for a broken install, and this would be if you had run pip install --user zprocess. This would have installed zprocess to some subdirectory of your home directory, I don't know which. But print(zprocess.__file__) from above should reveal if this is the case.

We'll get to the bottom of this!

-Chris

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/6c9e4427-965d-486b-9f5b-8b5efa0f11e4%40googlegroups.com.

[Fred Jendrzejewski]

Actually with your experimental file I obtained still a very similiar error:

[Chris Billington]

Ok, interesting. So if you start zlog manually, it works but then it can't connect to zlock. What happens if you manually start both zlock and zlog, by running in separate terminals

python -m labscript_utils.zlock

and 

python -m labscript_utils.zlog

And then starting runviewer?

Your pip installed zprocess looks fine to me - not sure what the difference between dist-info and egg-info is, but in any case it doesn't look like multiple installs, and the filepath and version of what actually gets imported matches. So I think we've ruled out issues with zprocess versioning and installation.

If that works, then it seems like the problem is that runviewer is unable to automatically start the servers. Perhaps there is a macos-specific reason for this (maybe a security feature). Lacking  a mac, it might be difficult for me to test this. But if things work when you start them manually but not automatically, we can look closer at the code that starts them automatically to determine why it isn't working in this case (I don't think that code has changed recently).

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/443920d6-e8ac-4397-9597-72c7fd4a0ebe%40googlegroups.com.

[Chris Billington]

This Python bug:
https://bugs.python.org/issue37586

Seems to imply there might be an issue with how we are starting the zlock and zlog subprocesses in macos (possibly depending on macos version and Python version). We are starting them in such a way that they keep running after the parent process quits, which in unix is typically done by forking, calling os.setsid(), then forking again (although there is not a second call to fork() in zprocess.start_daemon(), it starts a new subprocess instead, which actually does involve a second fork())

Another question - did labscript programs ever work on your mac, and what version of OSX is it?

-Chris

[Chris Billington]

For what it's worth, I am able to start runviewer on a mac virtualbox with Python 3.7 and latest labscript suite components. But the version of OSX is 10.12.6 Sierra, so probably not the same as you're running, and not the one mentioned in the Python bug report.

[Fred Jendrzejewski]

Hi Chris,

I saw just now, that the answer that I tried to send yesterday was somehow dropped. Starting the two servers by hand is actually working for me quite nicely. So it really might be an OSX issue. Also before the upgrade I had did not have to run them manually ...

Thanks for the help,

Fred

[Chris Billington]

Good to hear!

You're welcome, but if you have time I'd like to get to bottom of it so that I can resolve the issue for osx. So some questions if you don't mind! If you don't have time that's ok. Anyone else with a mac feel free to chime in, otherwise I might harass mac-owning friends later to investigate.

1. What update are you referring to, before which you didn't need to start servers manually? osx, Python, zprocess, labscript_utils? Do you recall which version (even approximately, or as a duration of time since you last updated it) of the relevant component you were on previously?

2. Does the following code create a text file in your home directory?

import os

from subprocess import DEVNULL, Popen
import tempfile

def start_daemon(cmd):
    if not os.fork():
        os.setsid()
        Popen(
            cmd,
            stdin=DEVNULL,
            stdout=DEVNULL,
            stderr=DEVNULL,
            close_fds=True,
            cwd=tempfile.gettempdir(),
        )

HOME = os.getenv('HOME')
start_daemon(['python', '-c', f'open("{HOME}/yes_it_works.txt", "w")'])

Don't forget to delete the file if so, then...

3. What about if you replace DEVNULL with open(os.devnull, 'w'), does it create the file then?  (I realised we're passing devnull in 'write' mode as stdin to the processes, that might be not allowed!)

4. In labscript_utils.ls_zprocess you will see two functions connect_to_zlock_server() and ensure_connected_to_zlog(), which are the functions responsible for starting the two servers if they're not running. They both have lines like

zprocess.start_daemon(
    [sys.executable, '-m', 'labscript_utils.zlock', '--daemon']

)

What happens if you remove '--daemon' from both of the lists? Can you then start runviewer without having to manually start the servers?

5. What happens if you keep '--daemon', but replace zprocess.start_daemon with subprocess.Popen (you'll need to add import subprocess at the top of the file)?

The motivation for test 2 is to see if we can create daemon processes (processes that keep running after their parents exit) at all in your environment. Test 3 is one hypothesis about how we might be doing this wrong. Tests 4 and 5 are because I can see we're actually "daemonising" the processes twice - we're creating a daemon process that is creating another daemon process. The tests are to see what happens if we only do it once by removing each of the two steps one a time.

-Chris

--

You received this message because you are subscribed to the Google Groups "The labscript suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/2cf9b639-7f09-4cce-97c1-611775e16818%40googlegroups.com.

[Fred Jendrzejewski]

On Wednesday, December 4, 2019 at 9:46:16 PM UTC+1, Chris Billington wrote:

Good to hear!

You're welcome, but if you have time I'd like to get to bottom of it so that I can resolve the issue for osx. So some questions if you don't mind! If you don't have time that's ok. Anyone else with a mac feel free to chime in, otherwise I might harass mac-owning friends later to investigate.

1. What update are you referring to, before which you didn't need to start servers manually? osx, Python, zprocess, labscript_utils? Do you recall which version (even approximately, or as a duration of time since you last updated it) of the relevant component you were on previously?

No idea to be honest. But most likely the update of macOS to the new system ...

2. Does the following code create a text file in your home directory?

import os

from subprocess import DEVNULL, Popen
import tempfile

def start_daemon(cmd):
    if not os.fork():
        os.setsid()
        Popen(
            cmd,
            stdin=DEVNULL,
            stdout=DEVNULL,
            stderr=DEVNULL,
            close_fds=True,
            cwd=tempfile.gettempdir(),
        )

HOME = os.getenv('HOME')
start_daemon(['python', '-c', f'open("{HOME}/yes_it_works.txt", "w")'])

Don't forget to delete the file if so, then...

Yes.

3. What about if you replace DEVNULL with open(os.devnull, 'w'), does it create the file then?  (I realised we're passing devnull in 'write' mode as stdin to the processes, that might be not allowed!)

I did not try this given that 2.) worked.

4. In labscript_utils.ls_zprocess you will see two functions connect_to_zlock_server() and ensure_connected_to_zlog(), which are the functions responsible for starting the two servers if they're not running. They both have lines like

zprocess.start_daemon(
    [sys.executable, '-m', 'labscript_utils.zlock', '--daemon']

)

What happens if you remove '--daemon' from both of the lists? Can you then start runviewer without having to manually start the servers?

I think that I still have the same error

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 343, in ensure_connected_to_zlog
    client.ping(timeout=0.05)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 116, in ping
    self._send(b'hello', timeout=timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 90, in _send
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 71, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout=timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')
zprocess.utils.TimeoutError: Could not connect to server: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/fredjendrzejewski/labscript_suite/runviewer/__main__.py", line 58, in <module>
    logger = setup_logging('runviewer')
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/setup_logging.py", line 37, in setup_logging
    ensure_connected_to_zlog()
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 353, in ensure_connected_to_zlog
    client.ping(timeout=15)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 116, in ping
    self._send(b'hello', timeout=timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 90, in _send
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 71, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout=timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')
zprocess.utils.TimeoutError: Could not connect to server: timed out

5. What happens if you keep '--daemon', but replace zprocess.start_daemon with subprocess.Popen (you'll need to add import subprocess at the top of the file)?

The error code changed, still have an errorcode sooo....

/Users/fredjendrzejewski/miniconda3/envs/lss3/bin/python: Error while finding module specification for 'labscript_utils.zlog' (ModuleNotFoundError: No module named 'labscript_utils')

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 345, in ensure_connected_to_zlog
    client.ping(timeout=0.05)

  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 116, in ping
    self._send(b'hello', timeout=timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 90, in _send
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 71, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout=timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')
zprocess.utils.TimeoutError: Could not connect to server: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/fredjendrzejewski/labscript_suite/runviewer/__main__.py", line 58, in <module>
    logger = setup_logging('runviewer')
  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/setup_logging.py", line 37, in setup_logging
    ensure_connected_to_zlog()

  File "/Users/fredjendrzejewski/labscript_suite/labscript_utils/ls_zprocess.py", line 355, in ensure_connected_to_zlog
    client.ping(timeout=15)

  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 116, in ping
    self._send(b'hello', timeout=timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 90, in _send
    self._new_socket(timeout)
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/zlog/__init__.py", line 71, in _new_socket
    'tcp://%s:%s' % (self.host, str(self.port)), timeout=timeout
  File "/Users/fredjendrzejewski/miniconda3/envs/lss3/lib/python3.7/site-packages/zprocess/security.py", line 251, in connect
    raise TimeoutError('Could not connect to server: timed out')
zprocess.utils.TimeoutError: Could not connect to server: timed out

 

The motivation for test 2 is to see if we can create daemon processes (processes that keep running after their parents exit) at all in your environment. Test 3 is one hypothesis about how we might be doing this wrong. Tests 4 and 5 are because I can see we're actually "daemonising" the processes twice - we're creating a daemon process that is creating another daemon process. The tests are to see what happens if we only do it once by removing each of the two steps one a time.

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscri...@googlegroups.com.

[Chris Billington]

Thanks Fred.

Unfortunately that gives me no leads, since no changes to labscript code made it work, and no isolated tests of similar functionality didn't work. I'll see if I can commandeer a friend's mac and reproduce the issue, and continue to home in on the place that must exist between the isolated examples and the non-working real code.

Wouldn't be surprised if it is a new mac security feature to not allow spawning background processes in some cases, it seems to be the trend for apple to be locking things down a bit more. Might have to have labscript code talk with launchd to start a background service the official way.

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/38954e39-de42-4894-a9a7-f8d1bd2ea773%40googlegroups.com.

[Fred Jendrzejewski]

Hi,

thanks for all your help anyway. Should there be a bug report on this ?

Cheers,

Fred

On Friday, December 6, 2019 at 5:01:03 PM UTC+1, Chris Billington wrote:

Thanks Fred.

Unfortunately that gives me no leads, since no changes to labscript code made it work, and no isolated tests of similar functionality didn't work. I'll see if I can commandeer a friend's mac and reproduce the issue, and continue to home in on the place that must exist between the isolated examples and the non-working real code.

Wouldn't be surprised if it is a new mac security feature to not allow spawning background processes in some cases, it seems to be the trend for apple to be locking things down a bit more. Might have to have labscript code talk with launchd to start a background service the official way.

-Chris

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/38954e39-de42-4894-a9a7-f8d1bd2ea773%40googlegroups.com.

[Chris Billington]

Yup, reported here so I don't forget about it, but I'll probably investigate on Monday when I should be able to borrow a mac.

-Chris

To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/c903d155-16ab-47d9-8e3d-39f0c1ad72bd%40googlegroups.com.

[Chris Billington]

I tried to reproduce this on a mac with Catalina and was not able to - runviewer started up without error on a fresh install with Python 3.7.5.

So this is a mystery for now!