Hi Erik,
On Wed, Jun 1, 2016 at 6:03 PM, Erik Kastman <
erik.k...@gmail.com> wrote:
> Don’t kill yourself if you can’t find it fast, but I’ll take whatever you can give me. :)
cannot find it on my laptop -- it's probably somewhere on our old lab
computer. But I did discover an old message from Sol, it's so
extensive it's almost like a manual ;) that should explain everything
you need to know. Enjoy, and good luck!
Richard
----------
Devices can block ioHub Event Processing
Yes, this is totally the case and is critical to keep in mind when
developing a device for iohub. iohub uses gevent and greenlet instead
of python threads (for the majority of things). There are +'s and -'s
to this. IMO, for what iohub needs to do, the +'s outweigh the -'s. ;)
The reasoning is that, due to the Python GIL only allowing 1 python
thread to run at a time, having too many threads running in parallel
actually reduces the amount of 'real' work each thread can do. It can
be so bad in some use cases where single threaded computation is
actually much faster than breaking up the task into threads. Using
greenlets allows you to program in a fashion similar to threads, but
is much much more light weight. i.e. switching between greenlets is
/much/ faster than switching between threads, and since both run in a
serial nature in python, greenlets allow python to spend more time
doing real work and let time in context switching.
The potential negative to this is that you control the scheduling of
your own 'tasks' code in iohub, it is not done for you via thread
context switching. This can also be an advantage once you get use to
it actually. So you need to keep any calls made within your device to
be non-blocking and as fast as possible.
In iohub, each device has it's own greenlet that handles event
processing for that device. Basically this means that a devices
._poll() method, or ._handleNativeEvent method(), runs within a
tasklet for the device. Which of the two methods a device should use
depending on whether the native device needs to be polled for new
events, or can tell iohub when new events are available via a callback
function. Any device methods that can be called by the psychopy
processes script are run by the main iohub servers tasklet.
As you found out the hard way, if your device makes a call that is
blocking, then no other greenlet /tasklet can run until that blocking
call is done. This may sound serious at first, and it can be if how
the design works is not understood. In practice it is rarely a issue,
since devices will either have non-blocking / asynchronous versions of
blocking calls, are the native device delivers events in an event
driven fashion, so blocking is not an issue to begin with.
Finally, sometimes you have no choice but to run the native device
interface in a separate python thread though, and then use the iohub
device's callback approach, or use non blocking python queue calls, to
hand events to the iohub device tasklet. If the native device must be
polled for new events / data, and the device generates events at a
very slow rate, then using a thread may also make more sense then
using non blocking methods and having the device poll every msec or
something. If a thread is almost always in a blocked state, then the
issue with the python GIL is non significant.
You can also explicitly 'break up' an iohub device method's execution
into chunks if it takes a long time to run and you do not want to
block other devices for that long. This is often called cooperative
multitasking; you control when one tasklet yield execution to other
that are waiting to run. The easiest way to do this is to call:
gevent.sleep(0)
within your method at points where you want to allow other tasklets to
run before continueing that methods code. The call to gevent.sleep(0)
effectively:
tells the iohub scheduler to stop running your tasklet at that point
each other tasklet, that is waiting to run, gets a chance to run.
Then control is returned to the tasklet that called gevent.sleep and
execution continues.
If you actually want your device to sleep for x sec.msec during the
call to a function, then use gevent.sleep(x), this allows other
tasklets to run and the current tasklet will be scheduled to resume
after x amount of time.
So gevent.sleep is an example of a blocking python function
(time.sleep) that gevent has provided a non blocking asynchronous
version of, but that complexity is hidden from you. gevent provides an
async version of the python socket module as well for example.
For your example use case, you could made a non blocking version of
the method without using a thread by something like:
def provideStimulation(delay,duration):
## wait for a certain amount of time
gevent.sleep(delay)
## send a trigger (open valves)
# ... your value open code here....
## wait for a certain amount of time (stimulation)
gevent.sleep(duration)
## send another trigger (close valves)
# ... your close valve code here....
While the above method runs, other devices are not being blocked by
it, except at the two points of valve triggering. Note this highlights
the good side about tasklets; you decide when your method will stop
running and allow others to run, it is not arbitrarily decided by the
python GIL or OS thread scheduling.
If provideStimulation was called from your psychopy script, while it
would not block other iohub devices, it would block your psychopy
script until it returned. This is a design choice I made in the iohub
interface BTW, not something inherent to other packages used.
If you wanted the method to return back to your psychopy script in an
async manner (so it returned prior to the actual method completing on
the iohub server, you could change the method as follows:
def provideStimulation(delay,duration):
""" Async version (for psychopy script) of method
"""
def provideStimulationTask(delay,duration):
## wait for a certain amount of time
gevent.sleep(delay)
## send a trigger (open valves)
# ... your value open code here....
## wait for a certain amount of time (stimulation)
gevent.sleep(duration)
## send another trigger (close valves)
# ... your close valve code here....
gevent.spawn(provideStimulationTask,delay=delay, duration=duration)
Now the method will return very quickly back to your psychopy script
and the iohub process will run the provideStimulationTask function
asynchronously.
----------