Installing labscript from scratch with a Python 3.7.1 virtual environment in Windows 10

[Rohit Prasad Bhatt]

Hi all,
We have started to set up labscript for our use. I installed Anaconda with Python 2.7 in root environment. But since we want to really use it in Python 3, I setup a Python 3.7.1 virtual enironment with conda and installed labscript via this environmnet. Now the problem is when I click on start menu icons for blacs, lyse etc., none of them launch but show error messages.

So I thought maybe I should use conda command line (by first going to the Python 3.7.1 virtual enironment and then lauch labscript modules). This works fine for lyse, runviewewr and runmanager. Blacs also launches but with some errors. The command I use to launch is "python -m blacs" etc.

Here are the list of errors :

1. When directly clicking on start menu icons :

A.  For Blacs
Traceback (most recent call last):
  File "C:\labscript_suite\blacs\__main__.py", line 41, in <module>
    from qtutils.qt.QtCore import *
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\qtutils\__init__.py", line 27, in <module>
    from qtutils.qt.QtCore import qInstallMessageHandler
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\qtutils\qt.py", line 86, in <module>
    from PyQt5 import QtGui, QtCore, QtWidgets
ImportError: DLL load failed: The specified module could not be found.


B.  For Lyse
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\core\__init__.py", line 16, in <module>
    from . import multiarray
ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\labscript_suite\lyse\__main__.py", line 23, in <module>
    import numpy as np
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\__init__.py", line 142, in <module>
    from . import add_newdocs
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\lib\__init__.py", line 8, in <module>
    from .type_check import *
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\lib\type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\core\__init__.py", line 26, in <module>
    raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

Original error was: DLL load failed: The specified module could not be found.


C.  For Runmanager
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\core\__init__.py", line 16, in <module>
    from . import multiarray
ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\labscript_suite\runmanager\__main__.py", line 33, in <module>
    import matplotlib
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\matplotlib\__init__.py", line 141, in <module>
    from . import cbook, rcsetup
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\matplotlib\cbook\__init__.py", line 33, in <module>
    import numpy as np
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\__init__.py", line 142, in <module>
    from . import add_newdocs
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\lib\__init__.py", line 8, in <module>
    from .type_check import *
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\lib\type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\numpy\core\__init__.py", line 26, in <module>
    raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

Original error was: DLL load failed: The specified module could not be found.


D.  For Runviewer
Traceback (most recent call last):
  File "C:\labscript_suite\runviewer\__main__.py", line 47, in <module>
    check_version('qtutils', '2.0.0', '3.0.0')
  File "C:\labscript_suite\labscript_utils\__init__.py", line 89, in check_version
    _reraise(exc_info)
  File "C:\labscript_suite\labscript_utils\__init__.py", line 69, in _reraise
    raise value.with_traceback(traceback)
  File "C:\labscript_suite\labscript_utils\__init__.py", line 52, in _get_version
    module = __import__(module_name)
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\qtutils\__init__.py", line 27, in <module>
    from qtutils.qt.QtCore import qInstallMessageHandler
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\qtutils\qt.py", line 86, in <module>
    from PyQt5 import QtGui, QtCore, QtWidgets
ImportError: DLL load failed: The specified module could not be found.



2. When launching from conda command line :

A.  For Blacs
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\zprocess\utils.py", line 26, in _reraise
    raise value.with_traceback(traceback)
  File "C:\labscript_suite\labscript_utils\connections.py", line 50, in __init__
    with h5py.File(h5file,'r') as hdf5_file:
  File "C:\labscript_suite\labscript_utils\h5_lock.py", line 54, in __init__
    _orig_init(self, name, mode, driver, libver, **kwds)
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\h5py\_hl\files.py", line 312, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\h5py\_hl\files.py", line 142, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\h5f.pyx", line 78, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = 'C:\Experiments\example_experiment\connectiontable.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)


I am trying to understand why I see these errors.

[Philip Starkey]

Hi Rohit,

While we don't officially support virtual envs at this stage, my experience is that they should work (I have the labscript suite installed in 2 virtual envs on my development machine, one for Python 3 and one for Python 2). It looks like there is something wrong with the paths of the shortcuts. Could you right-click on a shortcut, click properties and past the contents of both the "target" and "start in" fields. For reference, mine are "C:\Anaconda3\envs\labscript_py27\pythonw.exe C:\labscript_suite\labscript_py27\lyse\__main__.py" and "C:\labscript_suite\labscript_py27\lyse" (obviously path is specific to my machine, but you should get the idea). It's possible that the installer did not set those correctly, so let us know what it made for you!

The error you see for BLACS when launching from the anaconda prompt is to be expected at the moment. We currently don't have a nice way to generate the initial connection table h5 file for BLACS (the connection table defines the current set of hardware, and their connections to each other, and informs BLACS what device tabs to initialise). Usually this is done from within BLACS, but obviously this isn't possible unless you already have one! The current work around (which is actually documented, but I agree is a bit hard to find) is to compile the first connection table using runmanager, and manually copy it to the required location.

Before starting to do this I would suggest:

1. Updating the labconfig file: it is in labconfig/<PC-hostname>.ini and at least configuring an experiment name (unless you want to leave it as "example_experiment" - also note that this needs to be a valid Python module name, so no spaces etc.)
   
2. Creating a folder, that matches the experiment name, within the userlib/labscriptlib folder
3. Copy the "connectiontable_template.py" file from the labscript suite directory to userlib/labscriptlib/<experiment name>/connectiontable.py
   Note: This should agree with the path specific for "connection_table_py" in the labconfig file. By default it will.

Then, to compile the first connection table:

1. Using runmanager, select the newly created connectiontable.py file as the labscript file, and then compile a shot (no need to send it to BLACS at this stage, so you can untick the "run shots" checkbox) 
   
2. Find the compiled shot file (It will probably be in something like C:\Experiments\<experiment name>\connectiontable\<year>-<month>\<day>\<timestamp_connectiontable.h5>
3. Copy and rename this shot to: C:\Experiments\<experiment name>\connectiontable.h5 
4. Restart BLACS
5. BLACS should launch correctly. You will then be able to edit the connectiontable.py file as you see fit, and recompile it from within BLACS (rather than using runmanager). This removes the need for the above steps!

Sorry about the complicated first steps. It's on our list of things to make smoother!

Let us know if you run into any further problems.

Cheers,

Phil

[Rohit Prasad Bhatt]

Hi Phil,

Thanks for the detailed information. I will get back if I face any further issues.

Regards,

Rohit Prasad Bhatt

[Rohit Prasad Bhatt]

Hi Phil,

Sorry for getting back so late. It was a busy week and I could not work on debugging the issues.

Now I have tried your suggestions. But still BLACS launches with same errors. The reason I think is that runmanager still cannot provide any <timestamp_connectiontable.h5> file. In the ouput tab of runmanager, an error is dispalyed : "'generator' object has no attribute 'next' Compilation aborted." So the default folder in which I expect the <timestamp_connectiontable.h5> is created but it remains empty.

Is it because of the python version I am using?

Regards,

Rohit Prasad Bhatt

[Chris Billington]

Hi Rohit,

I noticed that we have not made a release of runmanager since it was ported to Python 3, which means that the labscript suite installer by default sets runmanager to a version not compatible with Python 3.

If you go into labscript_suite/runmanager, it is a mercurial repository, and you can update it to the most recent commit, which is compatible with Python 3. The command to update it is 'hg update'. You can also open the repository in tortoisehg and do it from there.

If you still have the issue, if you could send me the globals file and connection table .py file that caused the problem, I can try to reproduce the issue to investigate what's going on.

Thanks, 

Chris

--
You received this message because you are subscribed to the Google Groups "The labscript suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rohit Prasad Bhatt]

Hi Chris,

Yes updating worked. I updated all directories with C:\labscript_suite via right click context menu and choosing update from Tortoise Hg sub-menu.

Thanks.

Regards,

Rohit Prasad Bhatt

[Chris Billington]

Excellent, good to hear!

I've tagged the latest runmanager commit as version 2.2.0, so future labscript installations from the installer should get a Python-3 compatible runmanager. It was about time for a new release of runmanager in any case.

-Chris

[Rohit Prasad Bhatt]

Hi all,
The devices we have to control via labscript are : Pulseblaster USB, NI PXIe 6535 and NI PXIe 6738. I have seen that Chris has a fork repository in which there is a sub-folder called NI_DAQmx which includes the support code for our NI cards. However, being new to mercurial I am not sure how I could properly get that code in my local labscript installation folder and use it to include our NI cards in the connection table.

These NI cards are not included in the main labscript_devices folder in my local labscript installation folder.

Regards,

Rohit Prasad Bhatt

[Chris Billington]

Hi Rohit,

To get the NI_DAQmx branch, you can do the following on the command line in the labscript_devices folder.

hg pull --branch NI_DAQmx https://bitbucket.org/cbillington/labscript_devices

hg update default

hg branch beta-testing

hg merge NI_DAQmx

hg commit -m 'merge'

This will:

1. Pull my branch into your local repository (avoiding all the other branches I have in my fork, which you do not want)
2. Ensure you're at the current latest commit on the default branch
3. Make a new branch starting there called 'beta-testing'
4. Merge the NI_DAQmx branch into the new beta-testing branch
5. Commit the merge

It is good practice to create a new branch in this way for code you are using that has not yet been merged into mainline - that way the 'default' branch always corresponds to mainline labscript. However you can't just switch to the NI_DAQmx branch, because it has been in development for some time and so doesn't include other changes to labscript_devices in the meantime, hence the need to merge them locally to get something up to date and usable.

When the NI_DAQmx branch does get merged into mainline, you can then pull from the main repository and update to the default branch again. You can also then delete your 'beta-testing' branch (there is a mercurial extension called 'strip', you can enable it in the tortoisehg settings dialog, after which you can right click on commits in tortoisehg to delete them and their descendants.

As implied by the name, you'd be beta testing if you're using the unmerged branch. It's seen a fair bit of testing recently, but if you encounter any bugs, please let me know!

If your device is not one of the ones we've already added support for, see the readme file in NI_DAQmx/models/ for instructions on how to add support for a new DAQmx device. And see in NI_DAQmx/testing for an example of importing and using some of the devices (though not all the code in their is relevant, as it was mostly just me testing things).

Thanks,

Chris

[Rohit Prasad Bhatt]

Hi Chris,

Thanks for the suggestions. I executed the mercurial commands but the last command showed an error. Here is a snapshot of the mercurial console.

After this I checked my labscript installation folder and I do see that new sub-folders have been added but some files and folders are highlighted with a red exclamation mark. Here is a snapshot of my labscript installation folder:

But I still anyway went on to add the NI devices in the connection table. However, I encountered an error. The following are the details :

My Connection table :

from labscript import *
from labscript_devices.PulseBlasterUSB import PulseBlasterUSB
from labscript_devices.NI_DAQmx.models import NI_PXIe_6535

PulseBlasterUSB(name='pulseblaster_0')
NI_PXIe_6535(name='NI_DIO_card', parent_device=pulseblaster_0, clock_terminal='PXI1Slot3/PFI0', MAX_name='PXI1Slot3')

start()

stop(1)

The error when I press re-compile :

Traceback (most recent call last):

File "connectiontable.py", line 6, in <module>

File "C:\labscript_suite\labscript_devices\NI_DAQmx\models\NI_PXIe_6535.py", line 63, in __init__

NI_DAQmx.__init__(self, *args, **combined_kwargs)

File "C:\labscript_suite\labscript\labscript.py", line 161, in new_function

return_value = func(inst, *args, **kwargs)

File "C:\labscript_suite\labscript_devices\NI_DAQmx\labscript_devices.py", line 202, in __init__

IntermediateDevice.__init__(self, name, parent_device, **kwargs)

File "C:\labscript_suite\labscript\labscript.py", line 161, in new_function

return_value = func(inst, *args, **kwargs)

File "C:\labscript_suite\labscript\labscript.py", line 520, in __init__

raise LabscriptError('Error instantiating device %s. The parent (%s) must be an instance of ClockLine.'%(name, parent_device_name))

labscript.labscript.LabscriptError: Error instantiating device NI_DIO_card. The parent (pulseblaster_0) must be an instance of ClockLine.

Compilation failed. Please fix the errors in the connection table (python file) and try again

Regards,

Rohit Prasad Bhatt

[Chris Billington]

Hi Rohit,

The error when committing is that mercurial needs a username (I didn't anticipate this as it only occurs the first time you use mercurial, until you set a username). You can either set the username in the settings dialog of tortoisehg, or you can specify the username for just this commit, like:

hg commit -u 'Rohit Prasad' -m 'merge'.

Even though the merge hasn't been committed, the files on disk are what they are supposed to be, so it should be usable still. The exclamation point icons is just saying that these are files that have been merged but not committed yet.

As for the error compiling the connection table, as implied by the error message, an extra step is required. Because the PulseBlaster can have multiple outputs behaving as (somewhat) independent clocks, as well as just behaving as regular digital outputs, one must instantiate a ClockLine to say that one of its outputs will be a clock.  Your connection table would then look like this:

from labscript import *

from labscript_devices.PulseBlasterUSB import PulseBlasterUSB

from labscript_devices.NI_DAQmx.models import NI_PXIe_6535

PulseBlasterUSB(name='pulseblaster_0')

ClockLine(name='clockline', pseudoclock=pulseblaster_0.pseudoclock, connection='flag 0')

NI_PXIe_6535(name='NI_DIO_card', parent_device=clockline, clock_terminal='PXI1Slot3/PFI0', MAX_name='PXI1Slot3')

start()

stop(1)

Regards,

Chris

[Rohit Prasad Bhatt]

Hi Chris,

Thanks a lot. The commit worked with the command line.

Also the clockline error is resolved. I can now see all my cards in BLACS. Now I will get started by writing test experiments scripts. I also quite like the way the GUI design which clearly mentions which outputs support buffred execution.

Thanks again.

Regards,

Rohit Prasad Bhatt

[Chris Billington]

Excellent, good to hear!

[Chris Billington]

Hi Rohit,

For what it's worth, and for anyone else reading this who might have had the same issue, I've realised that the DLL loading errors you were getting were likely nothing to do with anything about your setup, and were probably in issue with the labscript suite installer and how it interacts with Conda. Basically, the shortcuts it creates did not properly set up the Conda environment. I've got a fix for this that will soon be included in the installer if everything looks good, so future installations of the labscript suite should work with Anaconda and Conda environments.

-Chris 

[Rohit Prasad Bhatt]

Hi Chris,
This also reminds me that Phil had asked me to post what paths in the start menu shortcuts the labscript installer created for me. So I have a Python 3.7.1 virtual environment which I have named "Labscript". Now here are the details of the shortcuts :

1. Blacs
Target : C:\ProgramData\Anaconda2\envs\Labscript\pythonw.exe C:\labscript_suite\blacs\__main__.py
Start in : C:\labscript_suite\blacs

2. Lyse
Target : C:\ProgramData\Anaconda2\envs\Labscript\pythonw.exe C:\labscript_suite\lyse\__main__.py
Start in : C:\labscript_suite\lyse

3. Runmanager
Target : C:\ProgramData\Anaconda2\envs\Labscript\pythonw.exe C:\labscript_suite\runmanager\__main__.py
Start in : C:\labscript_suite\runmanager

4. Runviewer
Target : C:\ProgramData\Anaconda2\envs\Labscript\pythonw.exe C:\labscript_suite\runviewer\__main__.py
Start in : C:\labscript_suite\runviewer

I was also (unneccessarily??) wondering whether my choice for the name of virtual environment was a bad one because there is an installation folder called "labscript" too. Would this ever cause a conflict?

Regards,

Rohit Prasad Bhatt

[Chris Billington]

Hi Rohit,

I can't think of any way the name of the virtual environment would matter, so I wouldn't be concerned about that.

If you were to prefix each "target" field with three extra paths (root env pythonw.exe, root env cwp.py wrapper script, env path), such that each looks like this (replace "lyse" with the name of each specific program), then I think the shortcuts might work:

C:\ProgramData\Anaconda2\pythonw.exe C:\ProgramData\Anaconda2\cwp.py C:\ProgramData\Anaconda2\envs\Labscript C:\ProgramData\Anaconda2\envs\Labscript\pythonw.exe C:\labscript_suite\lyse\__main__.py

This is what I'm changing the installer to do if it detects conda is in use.

-Chris

[Rohit Prasad Bhatt]

Hi Chris,

Yes, this fix worked really well. I can now launch everything from start menu. No more errors.

Thanks!

Regards,

Rohit Prasad Bhatt

[Rohit Prasad Bhatt]

Hi all,
I am trying to control NI PXIe 6738 in buffered mode from runmanager. I always end up with an error which complains about an overclocked sample rate. However, controlling the analog outputs in manual mode from BLACS directly works fine. I am attaching my connection table and test-experiment files.

Here is the error:
#################################
Exception in worker - Mon Feb 25, 18:28:58 :

Traceback (most recent call last):

  File "C:\labscript_suite\labscript_devices\NI_DAQmx\blacs_workers.py", line 330, in transition_to_buffered
    AO_final_values = self.program_buffered_AO(AO_table)
  File "C:\labscript_suite\labscript_devices\NI_DAQmx\blacs_workers.py", line 307, in program_buffered_AO
    None,
  File "<string>", line 3, in WriteAnalogF64
  File "<string>", line 2, in function
  File "C:\ProgramData\Anaconda2\envs\Labscript\lib\site-packages\PyDAQmx\DAQmxFunctions.py", line 62, in mafunction
    raise exception_class(errBuff.value.decode("utf-8"), f.__name__)
PyDAQmx.DAQmxFunctions.SampRateTooHighError: Specified sample rate is higher than the fastest rate supported by the device.
Property: DAQmx_SampClk_Rate
Corresponding Value:  1.0e6
Maximum Sample Rate:  400.0e3
Number of Channels: 2

Task Name: _unnamedTask<2>

Status Code: -200332
 in function DAQmxWriteAnalogF64
#################################

Regards,

Rohit Prasad Bhatt

[Rohit Prasad Bhatt]

Sorry I forgot the attachments.

[Chris Billington]

Hi Rohit,

The maximum clock speed of the 6738 depends on which outputs are in use, and I think the maximum of 1MHz is only accessible if you use a maximum of one analog output per 'bank' (i.e. only one out of each group of four AOs). Otherwise the maximum speed is lower.

The introspection code in the labscript DAQMx driver code doesn't know about this, and just thinks that the card is capable of 1MHz. But if you are using the device in a configuration that doesn't support that speed, which it sounds like you are, you can tell labscript that the maximum AO sample rate is lower, but passing the kaysoword argument max_AO_sample_rate=400e3 when you instantiate the NI_PXIe_6738 in your connection table.

I'm yet to decide how to take into account configuration-dependend capabilities like this, but setting the parameter manually at instantiation works.

For what it's worth, in fact, all that support for a particular NI_DAQmx device is in labscript, is a list of these keyword arguments. You can see for example in labscript_devices/NI_DAQmx/models/NI_PXIe_6738.py, that the only thing the subclass is doing is passing the list of capabilities as keyword arguments. But the user can override any of them, so in that way you can limit what labscript will assume about a device despite what capabilities are currently listed for it. For example, you might want to limit the voltage range of analog output, in order to have more precision given the limited bit-depth of the DACs, or for some other reason. 

Anyway, for your current issue, hopefully max_AO_sample_rate=400e3 solves the issue.

Regards,

Chris

[Chris Billington]

Alternatively, if instead of using ao0 and ao1, you use ao0 and ao4, then I think the card will be capable of 1MHz output rates and you also should no longer see the error. Also you will be able to use faster output rates, in the case that this benefits you.

[Rohit Prasad Bhatt]

Hi Chris,

Yes specifying the max sampling rate worked well. Actually I had started with testing the ramp functions where I did specify the sampling rate but it always showed this error. So I thought maybe it just has a wrong built-in value which I cannot overwrite. But I missed on the point that I could try to specify this as a class attribute itself.

Thanks for the help!

Regards,

Rohit Prasad Bhatt

[Chris Billington]

Hi Rohit,

Good to hear! Despite only using lower sample rates, labscript nonetheless tells the DAQmx driver that the maximum sample rate it might get is the highest it thinks it is capable of. Labscript might even generate some samples up to this sample rate even if you request a lower sample rate, because it might be necessary in order to clock some other device at the same time or in order to output the next instruction after the ramp at the right time. So just not using the higher sample rates in the ramps in your experiment script unfortunately wouldn't solve the problem, since labscript still needs to know what the actual limits are.

Anyway glad to hear it's working, and I might make this automatic in the future so that it doesn't cause the same issue for others, once I decide where the right place in the code to add this change would be.

-Chris