iohub on linux

[Daniel Shub]

I am having trouble using iohub.

from psychopy.iohub import launchHubServer
io=launchHubServer()
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/psychopy/iohub/client/__init__.py", line 1504, in launchHubServer
return ioHubConnection(ioConfig)
File "/usr/lib/python2.7/site-packages/psychopy/iohub/client/__init__.py", line 287, in __init__
raise RuntimeError("Error starting ioHub server: %s"%(self.iohub_status))
RuntimeError: Error starting ioHub server: ioHub startup timed out. iohub Server startup Failed. startup_read: Xlib.protocol.request.QueryExtension

startup_read: Xlib.protocol.request.QueryExtension

startup_read:



I am running Psychopy 1.82.02 on 64-bit Arch Linux with Python 2.7.10. Presumably, I am missing a dependency or have a version clash. Any ideas?

Thanks

Dan



This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please send it back to me, and immediately delete it.

Please do not use, copy or disclose the information contained in this
message or in any attachment. Any views or opinions expressed by the
author of this email do not necessarily reflect the views of the
University of Nottingham.

This message has been checked for viruses but the contents of an
attachment may still contain software viruses which could damage your
computer system, you are advised to perform your own checks. Email
communications with the University of Nottingham may be monitored as
permitted by UK legislation.

[Sol Simpson]

What version of python-xlib are you using and what version of the xlib lib is being used on your Arch Linux distro? 

Try updating the python-xlib package with the one you can download here, It works for me on the latest version of Mint Debian at least.

Thanks,

Sol

[Daniel Shub]

I am using libx11 version 1.6.3 and python-xlib version 0.15rc1 (I am pretty sure you sued this version when I had problems in the past (https://groups.google.com/forum/#!msg/psychopy-dev/Su_vcSsHV4k/j_gDLSR4QFcJ). My plan is to install some version of Debian in a chroot and make sure it is not my hardware/kernel. I might also be able to figure out what packages are different.
________________________________________
From: psycho...@googlegroups.com [psycho...@googlegroups.com] On Behalf Of Sol Simpson [s...@isolver-software.com]
Sent: Monday, September 21, 2015 2:02 PM
To: psychopy-dev
Cc: Danie...@nottingham.ac.uk
Subject: [psychopy-dev] Re: iohub on linux

What version of python-xlib are you using and what version of the xlib lib is being used on your Arch Linux distro?

Try updating the python-xlib package with the one you can download here<https://drive.google.com/open?id=0B-qFhchxISOSTjBkMjQ4VkpQam8>, It works for me on the latest version of Mint Debian at least.

--
You received this message because you are subscribed to the Google Groups "psychopy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-dev...@googlegroups.com<mailto:psychopy-dev...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

[Damien Mannion]

Daniel,

I am running iohub on Arch linux without any problems. This is inside a virtualenv so not sure how helpful it is, but here are the packages I have installed.

My process for getting iohub to work, in addition to the normal psychopy installation, is:
"pip install psutil msgpack-python greenlet gevent numexpr pyyaml lxml openpyxl future"

then manually install xlib and wx.

Cheers,

-damien

backports.ssl-match-hostname==3.4.0.2
certifi==2015.4.28
Cython==0.23
decorator==4.0.2
envoy==0.0.3
funcsigs==0.4
functools32==3.2.3.post2
future==0.15.0
gevent==1.0.2
greenlet==0.4.7
ipykernel==4.0.3
ipython==4.0.0
ipython-genutils==0.1.0
ipywidgets==4.0.2
jdcal==1.0
Jinja2==2.8
jsonschema==2.5.1
jupyter==1.0.0
jupyter-client==4.0.0
jupyter-console==4.0.0
jupyter-core==4.0.2
lxml==3.4.4
MarkupSafe==0.23
matplotlib==1.4.3
mistune==0.7
mock==1.3.0
msgpack-python==0.4.6
nbconvert==4.0.0
nbformat==4.0.0
nose==1.3.7
notebook==4.0.1
numexpr==2.4.3
numpy==1.9.2
openpyxl==2.2.5
pandas==0.16.2
path.py==7.6
pbr==1.5.0
pexpect==3.3
pickleshare==0.5
Pillow==2.9.0
plumbum==1.5.0
psutil==3.1.1
PsychoPy==1.82.1
ptyprocess==0.5
pyglet==1.3.0a1
Pygments==2.0.2
PyOpenGL==3.1.0
pyparsing==2.0.3
pyserial==2.7
python-dateutil==2.4.2
python-xlib==0.15rc1
pytz==2015.4
PyYAML==3.11
pyzmq==14.7.0
qtconsole==4.0.0
scipy==0.16.0
simplegeneric==0.8.1
six==1.9.0
tables==3.2.1
terminado==0.5
tornado==4.2.1
traitlets==4.0.0
veusz==1.23.1
wheel==0.24.0
wxPython==2.8.12.1
wxPython-common==2.8.12.1

[Damien Mannion]

I got curious about this because I was sure I'd seen that error before. Indeed, I get that error on one of my computers while it works fine on another computer (with the identical python environment).

After a bit of digging, I think the issue may arise in iohub/devices/mouse/linux2.py.

One the computer where iohub works, the line "Mouse._xscreen_count = self._xdll.XScreenCount(self._xdisplay)" works fine. On the computer where it doesn't, this line segfaults.

I'm not sure of the reason, but if I replace the ctypes calls with those from xlib, it works fine.

from pyglet.libs.x11 import xlib
Mouse._xdisplay = xlib.XOpenDisplay(None)
Mouse._xscreen_count = xlib.XScreenCount(Mouse._xdisplay)

Cheers,

-damien

[Daniel Shub]

That fixes the problem for me. If I make that change, I pass all the iohub unit tests. Nice job finding the segfault and a solution.
________________________________________
From: psycho...@googlegroups.com [psycho...@googlegroups.com] On Behalf Of Damien Mannion [damien.j...@gmail.com]
Sent: Wednesday, September 23, 2015 12:04 AM
To: psycho...@googlegroups.com
Subject: Re: [psychopy-dev] Re: iohub on linux

[Sol Simpson]

Thanks for finding a solution to this issue.

Did / will you make a pull request to have the code fix merged with github psychopy/psychopy? If not, I can update the code as you suggest and make a pull request.  

Thanks again.

[Damien Mannion]

Sol,

Thanks for finding a solution to this issue.

Did / will you make a pull request to have the code fix merged with github psychopy/psychopy? If not, I can update the code as you suggest and make a pull request. 

No, I haven't submitted a pull request - I wasn't sure if the pyglet dependency was trying to be avoided in iohub (I don't think it is used anywhere else?). In any case, just looking at it again now and it seems like the key component from pyglet is iohub's devices/xlib.py - so I think the pyglet import can be replaced with 'from .. import xlib'.

If you're happy with that, could you please update the code and submit a pull request.

Thanks,

-damien

[Sol Simpson]

No, I haven't submitted a pull request - I wasn't sure if the pyglet dependency was trying to be avoided in iohub (I don't think it is used anywhere else?).

Prior to iohub becoming part of psychopy, pyglet was not used at all within the iohub code. After moving to psychopy.iohub, pyglet is used in a few places other than devices/xlib.py. 

In any case, just looking at it again now and it seems like the key component from pyglet is iohub's devices/xlib.py - so I think the pyglet import can be replaced with 'from .. import xlib'.

If you're happy with that, could you please update the code and submit a pull request.

I can do that; it maybe late next week as I am away for ~ the next week. When I do the pull request I will include you so you get a notification about it. To be safe, it would be great If you could confirm the changed file still fixes the issue before Jon accepts the pull request.

Thanks again.

 

[Damien Mannion]

Sol,

I can do that; it maybe late next week as I am away for ~ the next week. When I do the pull request I will include you so you get a notification about it. To be safe, it would be great If you could confirm the changed file still fixes the issue before Jon accepts the pull request.

Great, thanks. I've just checked and can confirm that the following (on iohub/devices/mouse/linux2.py) allows iohub to work for me:

12a13,14
> from .. import xlib
>
36,37c38,39
<                 Mouse._xdisplay = self._xdll.XOpenDisplay(None)
<                 Mouse._xscreen_count = self._xdll.XScreenCount(self._xdisplay) 
---
>                 Mouse._xdisplay = xlib.XOpenDisplay(None)
>                 Mouse._xscreen_count = xlib.XScreenCount(self._xdisplay)

-damien

 

[Sol Simpson]

Can you update your iohub/devices/mouse/linux2.py file with the attached and let me know if it works for you. It just contains the changes you suggested, but I want to know it is OK for you before making the pull request. Thanks!

Sol

[Daniel Shub]

The fix works for me in that I can pass the iohub tests if I run as root in my regular environment. I fail the tests if I am not root or if I they are run in a chroot. I think these are different issues and I will dig into them more at some point.

Dan

---

From: psycho...@googlegroups.com [psycho...@googlegroups.com] on behalf of Sol Simpson [s...@isolver-software.com]
Sent: Tuesday, October 13, 2015 6:25 PM
To: psychopy-dev
Subject: Re: [psychopy-dev] Re: iohub on linux

--

You received this message because you are subscribed to the Google Groups "psychopy-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Shub]

If I understand the code correctly psychopy/iohub/client/__init__.py creates a file called .iohpid in "rootScriptPath", which seems to be the current working directory (although it may be more nuanced than that). I run into a problem if I run a script from a directory I do not have write access to. In fact, since my coder demo directory is write protected, the demo fails. The same goes for the iohub tests.

If I understand what goes into the .iohpid file, the Linux File System Hieracy (http://www.pathname.com/fhs/2.2/fhs-5.13.html) suggests that .iohpid should be written to /var/run/ and called iohub.pid.

As for the chroot issue. I am pretty sure my chroot does PID isolation with a cgroup (which means nothing to me yet) and this causes problems. I still need to look into that one.

Dan

---

From: psycho...@googlegroups.com [psycho...@googlegroups.com] on behalf of Daniel Shub [Danie...@nottingham.ac.uk]
Sent: Tuesday, October 13, 2015 9:08 PM
To: psycho...@googlegroups.com
Subject: RE: [psychopy-dev] Re: iohub on linux

[Sol Simpson]

OK, will change .iohpid save location to what you suggest when running on linux. Will change file name too.

Thanks for your help.

[Jeremy Gray]

hmmm, that link says "/var/run should be unwritable for unprivileged users (root or users running daemons); it is a major security problem if any user can write in this directory. " (http://www.pathname.com/fhs/2.2/fhs-5.13.html)

--Jeremy

--

[Sol Simpson]

Thanks Jeremy; Do you know where psychopy writes builder settings files, etc, to on linux? iohub should be able to use the same directory I would think.

Thanks again.

[Daniel Shub]

Oppps, missed that part of the specification. Maybe the location should be based on $XDG_RUNTIME_DIR

http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html#variables

Dan

---

From: Sol Simpson [s...@isolver-software.com]
Sent: Wednesday, October 14, 2015 7:09 PM
To: psychopy-dev
Cc: Danie...@nottingham.ac.uk

Subject: Re: [psychopy-dev] Re: iohub on linux

This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please send it back to me, and immediately delete it. 

Please do not use, copy or disclose the information contained in this
message or in any attachment.  Any views or opinions expressed by the
author of this email do not necessarily reflect the views of the
University of Nottingham.

This message has been checked for viruses but the contents of an
attachment may still contain software viruses which could damage your
computer system, you are advised to perform your own checks. Email
communications with the University of Nottingham may be monitored as
permitted by UK legislation.

[Jeremy Gray]

I think its in prefs cross-platform (and the contents do vary by platform)

>>> from psychopy import prefs
>>> prefs.paths['userPrefsDir']
'/Users/jgray/.psychopy2'

--Jeremy

[Jeremy Gray]

I think its in prefs cross-platform (and the contents do vary by platform)

Let me elaborate: I think a reasonable thing to use for a write-able directory for configuration-type files on all platforms is the value that is stored in prefs.paths['userPrefsDir']

 

>>> from psychopy import prefs
>>> prefs.paths['userPrefsDir']
'/Users/jgray/.psychopy2'

This is the value for me on a Mac. I think the analogous value on linux would be '/home/user/.psychopy2'. Many programs (like bash, git, and ssh) store stuff in the user home directory like this, so its very well-accepted as a practice.

However, for iohub that is not part of PsychoPy, something else is needed, like '/home/user/.iohub' for linux, or '/Users/user/.iohub'

[Damien Mannion]

Sol,

Can you update your iohub/devices/mouse/linux2.py file with the attached and let me know if it works for you. It just contains the changes you suggested, but I want to know it is OK for you before making the pull request. Thanks!

Yes, I've checked and updating that file makes iohub work fine.

Thanks,

Damien.

[Jonathan Peirce]

On 14/10/15 23:08, Jeremy Gray wrote:
>
> However, for iohub that is not part of PsychoPy, something else is
> needed, like '/home/user/.iohub' for linux, or '/Users/user/.iohub'

Actually although iohub began independently it is now a sub-package of
psychopy and isn't distributed in any other form at the moment. I guess
I'd leave it up to Sol whether to give it the same or different folder
for iohub config files.

Jon

--
Jonathan Peirce
University of Nottingham

http://www.peirce.org.uk

[Sol Simpson]

If it is OK to generalize the prefs.paths['userPrefsDir'] path to be used for creating experiment runtime related config / settings files, then I would just use prefs.paths['userPrefsDir']. 

If prefs.paths['userPrefsDir'] is meant to hold only Builder / Coder app related files, then I guess a different path should be created and used for exp. runtime files.

I'm leaning toward just using prefs.paths['userPrefsDir'], if others disagree please speak up. ;)

[Daniel Shub]

For Linux the XDG base directory specification (http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html) is fairly reasonable. It divides user-specific files into configuration, essential data, cache data, and runtime files as well as providing a hierarchy of where to look for files. I think Windows everything is supposed to get dumped into AppData. I am not sure what the model is on OS X.

From a practical standpoint, there are not many references to prefs.paths['userPrefsDir'] in the code base and my ~/.psychopy2 directory does not have too many files in it:

    appData.cfg: This is a configuration file and should be in $XDG_CONFIG_HOME/psychopy2/
    benchmarkReport.html: This seems like cache data and should be in $XDG_CACHE_HOME/psychopy2/
    firstrunReport.html: This seems like cache data and should be in $XDG_CACHE_HOME/psychopy2/
    monitors: This is a directory
        default.calib: This seems like essential data and should be in $XDG_DATA_HOME/psychopy2/
        testMonitor.calib: This seems like essential data and should be in $XDG_DATA_HOME/psychopy2/
    userPrefs.cfg: This is a configuration file and should be in $XDG_CONFIG_HOME/psychopy2/

The .ihpid file seems like it is a runtime file and should be in $XDG_RUNTIME_HOME/psychopy2/

I think we should consider defining, in addition to prefs.paths['userPrefsDir'] , prefs.paths['userDataDir'], prefs.paths['userCacheDir'], and prefs.paths['userRuntimeDir']. We should think about the name of userDataDir since it could be confusing to people (and possibly already exists). We could keep these new directories hidden from the user and on Windows and OS X just set them to be prefs.paths['userPrefsDir'] while on Linux we could use the correct directories. Then when we read and write files, we need to be careful to look in the correct directory.

---

From: psycho...@googlegroups.com [psycho...@googlegroups.com] on behalf of Sol Simpson [s...@isolver-software.com]

Sent: Thursday, October 15, 2015 4:30 PM

To: psychopy-dev
Subject: Re: [psychopy-dev] Re: iohub on linux

--
You received this message because you are subscribed to the Google Groups "psychopy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-dev...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please send it back to me, and immediately delete it. 

Please do not use, copy or disclose the information contained in this
message or in any attachment.  Any views or opinions expressed by the
author of this email do not necessarily reflect the views of the
University of Nottingham.

This message has been checked for viruses but the contents of an
attachment may still contain software viruses which could damage your
computer system, you are advised to perform your own checks. Email
communications with the University of Nottingham may be monitored as
permitted by UK legislation.