Installing on Windows with Anaconda

[Edvinas Gvozdiovas]

Hi All,

Using a fresh install of Anaconda, I am unable to install the labscript suite without errors on my machine. I am following the regular installation instructions provided by the labscript docs page. This happens near the very end with "labscript-profile-create".

Below is the anaconda config that gives me the error

# packages in environment at anaconda3\envs\py38:
#
# Name                    Version                   Build  Channel
blacs                     3.2.1                      py_0    labscript-suite
conda                     24.3.0           py38haa95532_0
conda-build               24.3.0           py38haa95532_0
conda-index               0.4.0              pyhd3eb1b0_0
conda-libmamba-solver     24.1.0             pyhd3eb1b0_0
conda-package-handling    2.2.0            py38haa95532_0
conda-package-streaming   0.9.0            py38haa95532_0
conda-verify              3.4.2                      py_1
desktop-app               0.4.1            py38h82bb817_0    labscript-suite
h5py                      3.9.0            py38hfc34f40_0
hdf5                      1.12.1               h51c971a_3
labscript                 3.3.1                      py_0    labscript-suite
labscript-c-extensions    1.1.0            py38h82bb817_0    labscript-suite
labscript-devices         3.2.0                      py_0    labscript-suite
labscript-suite           3.2.0                      py_0    labscript-suite
labscript-utils           3.3.1                      py_0    labscript-suite
numpy                     1.24.3           py38h79a8e48_1
pandas                    2.0.3            py38h4ed8f06_0
pip                       23.3.1           py38haa95532_0
pyqt                      5.15.10          py38hd77b12b_0
pyqt5-sip                 12.13.0          py38h2bbff1b_0
pyqtgraph                 0.13.1           py38haa95532_0
python                    3.8.19               h1aa4202_0
python-dateutil           2.8.2              pyhd3eb1b0_0
python-libarchive-c       2.9                pyhd3eb1b0_1
pyzmq                     25.1.2           py38hd77b12b_0
qt-main                   5.15.2              h19c9488_10
qtutils                   3.1.0                      py_0    labscript-suite
runmanager                3.2.1                      py_0    labscript-suite
runviewer                 3.2.2                      py_0    labscript-suite
scipy                     1.10.1           py38hdcfc7df_1
setuptools                68.2.2           py38haa95532_0
setuptools-conda          0.14.0                   py38_0    labscript-suite
setuptools-scm            8.0.4            py38haa95532_0
wheel                     0.41.2           py38haa95532_0
zprocess                  2.24.0                   py38_0    labscript-suite

The error occurs at the "labscript-profile-create" step:

labscript-profile-create
Traceback (most recent call last):
  File "C:\Users\dummy\anaconda3\envs\py38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\dummy\anaconda3\envs\py38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zprocess\makesecret.py", line 56, in <module>
    main()
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zprocess\makesecret.py", line 46, in main
    shared_secret = generate_shared_secret()
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zprocess\security.py", line 96, in generate_shared_secret
    _, client_secret = zmq.curve_keypair()
  File "zmq/backend/cython/utils.pyx", line 60, in zmq.backend.cython.utils.curve_keypair
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zmq\error.py", line 149, in _check_rc
    raise ZMQError(errno)
zmq.error.ZMQError: Not supported
Traceback (most recent call last):
  File "C:\Users\dummy\anaconda3\envs\py38\Scripts\labscript-profile-create-script.py", line 9, in <module>
    sys.exit(create_profile())
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\labscript_profile\create.py", line 74, in create_profile
    make_labconfig_file()
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\labscript_profile\create.py", line 45, in make_labconfig_file
    shared_secret = make_shared_secret(target_path.parent)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\labscript_profile\create.py", line 17, in make_shared_secret
    output = check_output(cmd, cwd=directory).decode('utf8')
  File "C:\Users\dummy\anaconda3\envs\py38\lib\subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\Users\dummy\anaconda3\envs\py38\lib\subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\dummy\\anaconda3\\envs\\py38\\python.exe', '-m', 'zprocess.makesecret']' returned non-zero exit status 1.

This seems related to zprocess and zmq. I have tried downgrading both to no avail but I will keep trying.

I also tried running zprocess.makesecret manually and this is what I got.

python -m zprocess.makesecret
Traceback (most recent call last):
  File "C:\Users\dummy\anaconda3\envs\py38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\dummy\anaconda3\envs\py38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zprocess\makesecret.py", line 56, in <module>
    main()
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zprocess\makesecret.py", line 46, in main
    shared_secret = generate_shared_secret()
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zprocess\security.py", line 96, in generate_shared_secret
    _, client_secret = zmq.curve_keypair()
  File "zmq/backend/cython/utils.pyx", line 60, in zmq.backend.cython.utils.curve_keypair
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zmq\error.py", line 149, in _check_rc
    raise ZMQError(errno)
zmq.error.ZMQError: Not supported

I saw others here had issues with zmq and zprocess recently, I will try a few combinations of package downgrades but so far it hasn't worked.

[Michael]

Hey Edvinas, 

I also recently encountered this error again on Friday, on both the regular install and the development install. This happened to me on an old fork recently updated to match the labscript repository, and a freshly pulled fork with the tags updated. 

I found that this goes away if you downgrade to pyzmq 25.1.0, or installing the dependencies before running the labscript install with the condition that pyzmq<=25.1.0.

--
You received this message because you are subscribed to the Google Groups "the labscript suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/c0c4eb58-3c8f-401d-b466-dd83988bbe5bn%40googlegroups.com.

[Edvinas Gvozdiovas]

Michael,

Thank you for the quick response. Strangely, downgrading to 25.1.0 didn't help. Thankfully, downgrading pyzmq to 23.2.0 was what got me through that.

Now I am at least running lyse, but lyse is giving me errors.

Traceback (most recent call last):

  File "C:\Users\dummy\anaconda3\envs\py38\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\zprocess\utils.py", line 122, in _reraise
    raise value.with_traceback(traceback)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\lyse\__main__.py", line 1853, in analysis_loop
    self.do_singleshot_analysis(filepath)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\lyse\__main__.py", line 1902, in do_singleshot_analysis
    self.shots_model.update_row(file, updated_row_data=updated_data[file])
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 214, in f
    return inmain(fn, *args, **kwargs)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 88, in inmain
    return get_inmain_result(_in_main_later(fn, False, *args, **kwargs))
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 150, in get_inmain_result
    raise value.with_traceback(traceback)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\qtutils\invoke_in_main.py", line 46, in event
    result = event.fn(*event.args, **event.kwargs)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\lyse\__main__.py", line 1416, in update_row
    self.dataframe.at[row_number, column_name] = value
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\pandas\core\indexing.py", line 2430, in __setitem__
    return super().__setitem__(key, value)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\pandas\core\indexing.py", line 2386, in __setitem__
    self.obj._set_value(*key, value=value, takeable=self._takeable)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\pandas\core\frame.py", line 4193, in _set_value
    self.loc[index, col] = value
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\pandas\core\indexing.py", line 849, in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\pandas\core\indexing.py", line 1835, in _setitem_with_indexer
    self._setitem_with_indexer_split_path(indexer, value, name)
  File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\pandas\core\indexing.py", line 1891, in _setitem_with_indexer_split_path
    raise ValueError(
RuntimeError: Analysis loop encountered unexpected exception. This is a bug and should be reported. The analysis loop is continuing, but lyse may be in an inconsistent state. Restart lyse, or continue at your own risk. Original exception was:

ValueError: Must have equal len keys and value when setting with an iterable

This error is connected to zprocess, qtutils and pandas, but thankfully not critical.

Thanks for the help.

[dihm....@gmail.com]

Michael and Edvinas,

The initial zmq error is a known issue with the zeromq==4.3.5 package as provided by the default conda channel (see here for discussion). Downgrading that package to 4.3.4 should fix that particular issue. I had hoped anaconda would be a little more proactive in fixing the build there, but it has been a while with zero response. We'll probably just have to pin the package down for the time-being.

As for the lyse error, that is quite a bit more problematic. I'd be curious to know more to ensure the issue is fixed. As a first round of advice that is unlikely to help, I would highly recommend that new installs of labscript use a newer version of python than 3.8 (which is end of life). 

-David

[Edvinas Gvozdiovas]

David,

Thank you for your response.

If I were a betting man, I'd say that there is a (possibly lucky) combination of packages with python>3.8 that we would find satisfactory. Since we have relatively a lot of people using labscript, I have the feeling we could pin down these combinations with the help of the community, and update the installation instructions on the labscript suite homepage.

I guess the question is - do we have a very good reason for sticking to py38?

Anyway, I get that lyse error persistently whenever I launch lyse and load an .h5 file for the first time; the error goes away with subsequent .h5 files.

-Edvinas

--
You received this message because you are subscribed to a topic in the Google Groups "the labscript suite" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/labscriptsuite/sr8x_K4MqHU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to labscriptsuit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/0d01707f-ea8a-4172-ac38-2c82d0ebffb9n%40googlegroups.com.

[Ian B. Spielman]

Edvinas,

The error chain briefly leaves lyse for qtutils with this

File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\lyse\__main__.py", line 1902, in do_singleshot_analysis
self.shots_model.update_row(file, updated_row_data=updated_data[file])

And then lastly for Pandas with this

File "C:\Users\dummy\anaconda3\envs\py38\lib\site-packages\lyse\__main__.py", line 1416, in update_row
self.dataframe.at[row_number, column_name] = value

Pandas then dies with this note:

ValueError: Must have equal len keys and value when setting with an iterable

So my question is what data are you putting into the data frame (either as a side effect of some lyse call or deliberately) / are you giving a specific index to modify that dataframe? What is the minimal lyse script that leads to this behavior? Since we both fresh and old installs of lyse in several labs it seems to me that this issue (which is a bug in lyse) is being triggered either by your script specifically or perhaps by something in the h5 file that is unusual in your lab.




— Ian

Ian B. Spielman

Fellow, Joint Quantum Institute
National Institute of Standards and Technology and the University of Maryland

----- WEB -----
http://ultracold.jqi.umd.edu

----- EMAIL -----
spie...@jqi.umd.edu

----- ZOOM -----
https://umd.zoom.us/j/7984811536

----- PHONE -----
(301) 246-2482

----- MAIL -----
UMD:
2207 Computer & Space Sciences Bldg.
College Park, MD 20742

NIST:
100 Bureau Drive, Stop 8424
Gaithersburg, MD 20899-8424 USA

----- OFFICE -----
UMD: Physical Sciences Complex, Room 2153
NIST: Building 216, Room B131

> --
> You received this message because you are subscribed to the Google Groups "the labscript suite" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/d97d62b0-63e4-4811-9957-6a9b5ef87be1n%40googlegroups.com.

[Edvinas Gvozdiovas]

Indeed, even this is enough to reproduce the error.

shot.save_result("R_fit", R_fit, overwrite=True)

If R_fit is a list, lyse will spit out that error because it has trouble displaying the saved result in the dataframe. The data that is saved is completely fine.

On Wed, Mar 27, 2024 at 2:26 PM Edvinas Gvozdiovas <brei...@gmail.com> wrote:

Actually, those save_results calls correctly save all of the input data to the .h5 file. The issue is not using save_results this way, I think this has to do with how these saved results are attempted to be displayed in the dataframe.

On Wed, Mar 27, 2024 at 2:09 PM Edvinas Gvozdiovas <brei...@gmail.com> wrote:

Hello Ian,

Thank you for looking into this. Trying this again with your questions in mind, I am getting this error with a specific lyse script that I wrote. If I comment out all the Run save_results calls, the error goes away. Here are the problematic lines:

shot.save_results('center_fit', center_fit,
                                  'R_fit', R_fit,
                                  'theta_fit', theta_fit,
                                  'offset_fit', offset_fit,
                                  'fit_error', fit_error,
                                  group=full_results_name, overwrite=True)

The reason this was happening is that center_fit and R_fit are lists containing two elements each. If I do this,

shot.save_results('center_fit', center_fit[0],
                                  'R_fit', R_fit[0],
                                  'theta_fit', theta_fit,
                                  'offset_fit', offset_fit,
                                  'fit_error', fit_error,
                                  group=full_results_name, overwrite=True)

The error goes away. This is somewhat unsurprising. I naively put all my save_result calls (which worked even with those two sequence variables) into a single save_results call without considering that this function accepts an infinite number of inputs.

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/36516D7C-87A7-48F8-B8D0-5D232D5CEDC3%40umd.edu.

[Edvinas Gvozdiovas]

Actually, those save_results calls correctly save all of the input data to the .h5 file. The issue is not using save_results this way, I think this has to do with how these saved results are attempted to be displayed in the dataframe.

On Wed, Mar 27, 2024 at 2:09 PM Edvinas Gvozdiovas <brei...@gmail.com> wrote:

Hello Ian,

Thank you for looking into this. Trying this again with your questions in mind, I am getting this error with a specific lyse script that I wrote. If I comment out all the Run save_results calls, the error goes away. Here are the problematic lines:

shot.save_results('center_fit', center_fit,
                                  'R_fit', R_fit,
                                  'theta_fit', theta_fit,
                                  'offset_fit', offset_fit,
                                  'fit_error', fit_error,
                                  group=full_results_name, overwrite=True)

The reason this was happening is that center_fit and R_fit are lists containing two elements each. If I do this,

shot.save_results('center_fit', center_fit[0],
                                  'R_fit', R_fit[0],
                                  'theta_fit', theta_fit,
                                  'offset_fit', offset_fit,
                                  'fit_error', fit_error,
                                  group=full_results_name, overwrite=True)

The error goes away. This is somewhat unsurprising. I naively put all my save_result calls (which worked even with those two sequence variables) into a single save_results call without considering that this function accepts an infinite number of inputs.

On Wed, Mar 27, 2024 at 11:19 AM Ian B. Spielman <spie...@umd.edu> wrote:

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/36516D7C-87A7-48F8-B8D0-5D232D5CEDC3%40umd.edu.

[Edvinas Gvozdiovas]

Hello Ian,

Thank you for looking into this. Trying this again with your questions in mind, I am getting this error with a specific lyse script that I wrote. If I comment out all the Run save_results calls, the error goes away. Here are the problematic lines:

shot.save_results('center_fit', center_fit,
                                  'R_fit', R_fit,
                                  'theta_fit', theta_fit,
                                  'offset_fit', offset_fit,
                                  'fit_error', fit_error,
                                  group=full_results_name, overwrite=True)

The reason this was happening is that center_fit and R_fit are lists containing two elements each. If I do this,

shot.save_results('center_fit', center_fit[0],
                                  'R_fit', R_fit[0],
                                  'theta_fit', theta_fit,
                                  'offset_fit', offset_fit,
                                  'fit_error', fit_error,
                                  group=full_results_name, overwrite=True)

The error goes away. This is somewhat unsurprising. I naively put all my save_result calls (which worked even with those two sequence variables) into a single save_results call without considering that this function accepts an infinite number of inputs.

On Wed, Mar 27, 2024 at 11:19 AM Ian B. Spielman <spie...@umd.edu> wrote:

To view this discussion on the web, visit https://groups.google.com/d/msgid/labscriptsuite/36516D7C-87A7-48F8-B8D0-5D232D5CEDC3%40umd.edu.