USB polling rate

[Nicolas McNair]

Hi there,

I had a look around for this but couldn't find an explicit answer. Does Psychopy support USB polling at 1000Hz (either via psychopy.event or psychopy.iohub) or does it use the default 125Hz? We have some BlackBox USB response pads which are capable of supporting the higher polling rate, but obviously this depends on what is doing the actual polling.

Thanks very much!

Nick

[Jonathan Peirce]

At present you'd have to write the code to check the input values from the usb device yourself anyway, so I imagine the rate at which you check those values would be determined by your own code. It would be best to add this as an iohub device to make sure that you keep it separate from the psychopy drawing loop - or you could get tethered to the screen refresh rate for your polling

cheers
Jon

--
You received this message because you are subscribed to the Google Groups "psychopy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-user...@googlegroups.com.
To post to this group, send email to psychop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/053383ae-b64b-43fc-9708-c4a68b7c7874%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Jonathan Peirce
University of Nottingham

http://www.peirce.org.uk

This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please send it back to me, and immediately delete it. 

Please do not use, copy or disclose the information contained in this
message or in any attachment.  Any views or opinions expressed by the
author of this email do not necessarily reflect the views of the
University of Nottingham.

This message has been checked for viruses but the contents of an
attachment may still contain software viruses which could damage your
computer system, you are advised to perform your own checks. Email
communications with the University of Nottingham may be monitored as
permitted by UK legislation.

[Richard Höchenberger]

Hi Nicolas,

On Wed, Jan 20, 2016 at 12:01 AM, Nicolas McNair
<nicola...@gmail.com> wrote:
> Does Psychopy support USB polling at 1000Hz (either via psychopy.event or
> psychopy.iohub) or does it use the default 125Hz?

as far as I know, the polling rate has to be set by the respective USB
driver and cannot be easily altered from user space. That is, PsychoPy
should not be able to do anything about it.

I've never seen anyone raising the USB polling rate for anything other
than gaming mice (see e.g.
https://wiki.archlinux.org/index.php/Mouse_polling_rate) -- well,
exept for this request:
http://superuser.com/questions/369826/increase-usb-polling-rate-across-all-devices-in-linux

I don't know about the BBTK response pad; but usually, USB response
pads either send a time-stamped signal, so you don't need to poll
constantly; or they are simple serial devices operating on a virtual
COM port that constantly receives data, and the exact time of response
can be determined by finding the respective signal position in the
buffer and relating this to the baud rate.

All the best,

Richard

[Jon Peirce]

Actually, no this device by the BBTK guy doesn't have an on-board clock
so can't send time-stamped keypresses. I think it is the case with this
one that you need to poll the device frequently to look for changes in
the stream of data (relating to keypresses).

Jon

--
Jon Peirce
http://www.peirce.org.uk

[Richard Höchenberger]

On Thu, Jan 21, 2016 at 3:27 PM, Jon Peirce <jon.p...@gmail.com> wrote:
> this device by the BBTK guy doesn't have an on-board clock so can't send
> time-stamped keypresses

Oh, that's indeed a bit unfortunate :(
But browsing through
https://msdn.microsoft.com/en-us/library/windows/hardware/ff539317(v=vs.85).aspx,
I am wondering whether a USB device itself (!) could not request a
high(er) polling rate. Maybe the BBTK response pad is already running
at 1 kHz polling frequency for you, Nicolas? Although I'm not sure how
one could test this :)

Richard

[Jon Peirce]

I think the point is that even if it is running at 1kHz if you don't
look at it very often then it makes no difference. So the receiving
script needs to check it that often.

[Nicolas McNair]

Thanks very much for the replies!

Yes, as Jon said, its capable of reporting 1kHz via USB but this depends on whatever is checking the USB port.

In terms of checking input values, it simply emulates a keyboard press ('a', 's', 'd', and 'f' in the case of our 4-button response pads) so it doesn't actually need any special code to use it as such - but this will default to whatever Psychopy uses for polling a (USB) keyboard. Is that fixed to whatever the OS defaults to, Jon? I.e., I have a vague awareness that it is possible to change this (in Windows at least) via some registry hack or through various drivers floating around for gamers. If that worked, would that also result in psychopy adopting a higher polling rate? Or would I still need to code that specifically?

As a side note, they do also output a TTL pulse which would provide highly accurate timing of keypresses. But that results in a slightly more complicated set-up than I'd prefer - at least for some of the situations I'm hoping to use it in.

Thanks again!

[Richard Höchenberger]

Hi Nicolas,

On Fri, Jan 22, 2016 at 7:05 AM, Nicolas McNair <nicola...@gmail.com> wrote:
> In terms of checking input values, it simply emulates a keyboard press ('a',
> 's', 'd', and 'f' in the case of our 4-button response pads) so it doesn't
> actually need any special code to use it as such - but this will default to
> whatever Psychopy uses for polling a (USB) keyboard. Is that fixed to
> whatever the OS defaults to, Jon? I.e., I have a vague awareness that it is
> possible to change this (in Windows at least) via some registry hack or
> through various drivers floating around for gamers. If that worked, would
> that also result in psychopy adopting a higher polling rate? Or would I
> still need to code that specifically?

yes, you could try one of those "gaming mouse" driver hacks, although
I would not really recommend it, since these drivers are usually
unsigned and you would have to put Windows into Test Mode to
install/use these drivers; it's all kind of hackish and potentially
creates security risks.

If you decide to take this approach:
Since it's the OS USB driver itself that will acquire the response pad
data and pump it into the keyboard buffer, I reckon you would not need
to change anything about PsychoPy's standard keyboard devices. You
will definitely have to either A) poll the device yourself via
psychopy.event.getKeys() (and possibly a while loop); B) have PsychoPy
poll it via psychopy.event.waitKeys(); or C) you could try to use
ioHub's keyboard device to get non-blocking behavior.

> As a side note, they do also output a TTL pulse which would provide highly
> accurate timing of keypresses. But that results in a slightly more
> complicated set-up than I'd prefer - at least for some of the situations I'm
> hoping to use it in.

I personally would pick this approach. But it requires additional data
acquisition hardware and is not as convenient as simply using ioHub or
psychopy.event keyboard polling.

Cheers,

Richard

[Jonathan Peirce]

psychopy.event is using pyglet to check keypresses and I doubt pyglet is
doing anything fancy

iohub makes direct calls to the system and can make those calls at a
higher rate, automatically separated from the stimulus rendering
process, so if the box outputs signals that look like keypresses I'd say
there's a good chance that iohub will return those with high precision
but I'd suggest you test that! :-)

cheers
Jon

[Nicolas McNair]

Sorry for the late reply. According the Black Box people, the response pads request high speed polling from the OS when plugged in (USB 2.0 or higher). So they said as long as PsychoPy wasn't interfering in some way, it should be possible to poll at 1000Hz. I went ahead and tested this using a CRT monitor/PC (running Windows 7) set-up that I know, via independent testing, reports 100% accurate timing for visual stimulus presentation. I just used a simple PsychoPy script that recorded reaction times to a white screen flash across a number of trials. The response pad was connected to the stimulus presentation computer via USB and to the Black Box Toolkit (BBTK) via its high-speed serial connection (which provides sub-millisecond timing). The BBTK was also recording the onset of the flash via photodiode. Comparing the reported response pad RTs on the stimulus presentation computer to those on the BBTK, I get differences across all the trials of less than a millisecond. So it looks like using the Black Box response pads with PsychoPy can provide millisecond timing even when just connected via USB only! Also, this was just using the waitKeys function from psychopy.event. I haven't gotten around to testing iohub yet, but I'm guessing it should be the same.

Thanks again for your help!

Nick

[Jonathan Peirce]

Nick,

Thanks for the useful info. That means the communication side is working perfectly, which is great to hear.

So, the remaining issue is around how often the box is check. I think you'll find that when you check using the default option of one check per screen refresh you'll get some timing variability associated with that, due to the fact that the screen refresh is every 16.66ms (at 60hz). That will still be much better than the keyboard, with a lag of up to 30ms.

If you need to check keys while screen changes are occurring (which means you can't use waitKeys()) then from your tests I'd expect to see very good timing from iohub but not from the event module.

thanks again for sending back th results

Jon

--
You received this message because you are subscribed to the Google Groups "psychopy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-user...@googlegroups.com.
To post to this group, send email to psychop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/8f767c80-3266-4739-98c8-1a606695336e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nicolas McNair]

Hi Jon,

I tend to run experiments where stimuli remain on the screen until subjects respond or the stimuli is displayed shorter than any possible reaction time, so yeah I usually just have a screen flip followed by waitkeys(). We did some more testing and found that (unexpectedly and bizarrely) iohub actually produces worse times than the event module in this situation. We've tested it using the waitforkeys() and getpresses() keyboard functions from iohub, and both report differences from the 'true' reaction time of between 3 and 15ms. Here's a snapshot from both:

waitforkeys()

12.2
12.9
4.3
7.5
13.0
6.4
13.6
5.2
7.9
8.7

getpresses()

6.2
4.7
13.3
10.7
14.6
4.4
9.1
3.7
7.3
7.0

Not too sure why this is happening. It doesn't seem to be related to low-speed polling (since it goes over 8ms (125Hz), although perhaps it may miss a poll every now and then?), screen refresh time (we were using 85Hz), or two clocks being out of sync (since you would expect a constant difference or a constant change). It may also be that there's something odd we're doing with our script so I'll add that as soon as I can grab it off the computer.

Also, the testing we did here for iohub was still in a set up where we do a single screen flip and then poll for a keypress. So in terms of using iohub for situations where you're changing stuff on screen while recording response, I'm not precisely sure how the loss of timing accuracy when using the event module in these situations would compare - but these differences are rather large.

Ultimately it may not matter too much in most situations. Assuming you a) have enough trials and b) that the average of this timing difference is reasonably constant (or if it changes over time, you use adequate randomisation of trial conditions) then it should just be adding a constant to all of your conditions. But for some situations (e.g., looking at response-related potentials with EEG or correlating RTs with something else) I think it could be a bit of a problem.

Nick

[Marius Mather]

Nick and I have just been doing some testing using iohub's keyboard events and the results so far look less promising- iohub's recorded RTs are generally around 5-15ms slower than what's being measured with the Black Box Toolkit. This is on the same setup where psychopy.event was consistently showing ~1ms differences relative to what the BBTK recorded. Nick has the actual testing data and should be posting it shortly.

I've attached the simple script we were using to check iohub's keyboard event timing. All it does is set the screen to white at the start of the trial and wait for a keypress (currently it checks A, S, D, F and spacebar, since the usb response pad has its 4 buttons mapped to ASDF). If anyone is able to suggest improvements that might improve the timing accuracy, please do. Is there any better way to calculate the RT than running Computer.getTime() right after the first flip of the white display, then subtracting that from the event.time of the events returned by iohub's keyboard.getPresses() function?

Marius

[Richard Höchenberger]

Hi, nice detective work you've been doing there, this is indeed very
interesting!

On Mon, Feb 1, 2016 at 3:29 AM, Marius Mather <marius...@gmail.com> wrote:
> If anyone is able to suggest improvements that might improve the timing
> accuracy, please do. Is there any better way to calculate the RT than
> running Computer.getTime() right after the first flip of the white display,
> then subtracting that from the event.time of the events returned by iohub's
> keyboard.getPresses() function?

The following suggestions are completely untested.

Instead of doing:

----------
WINDOW.flip()
# Start timer right after the flip
trial_start_time = Computer.getTime()
----------

you should be able to use:
----------
# Start timer right after the flip
trial_start_time = WINDOW.flip()
----------

as ioHub and PsychoPy rely on the same time base. But this change
actually should not improve the timing much.

In line 36, you set:
----------
Computer.setPriority('high')
----------

which raises PsychoPy's priority throughout the whole experiment. It
has been suggested that this might not be a good idea, as it doesn't
let Python/the OS do housekeeping/garbage collection (not entirely
sure about the GC, would have to check the code). Your 1 ms waits
between keyboard polling might not give the system enough time to
breathe with PsychoPy running at high priority.

Instead, you should move that line to line 47, and ALSO raise the
ioHub server priority:
----------
Computer.setPriority('high')
IOHUB_SERVER.setPriority('high')
WHITE_SCREEN.draw(WINDOW)
WINDOW.flip()
----------

The priority should be lowered during the ITI:

----------
# ITI
BLACK_SCREEN.draw(WINDOW)
for i in range(N_ITI_FRAMES - 1):
WINDOW.flip()
if i == 0:
Computer.setPriority('low')
IOHUB_SERVER.setPriority('low')
----------

although I can see why you would want to keep it raised if you rely on
precise length of the ITI...

Let me know if that improves anything.

Btw, why would you want "for i in range(N_ITI_FRAMES - 1)" instead of
"for i in range(N_ITI_FRAMES)"?

Good luck,

Richard

[Richard Höchenberger]

Oh and I just realized I've never seen anyone using core.wait() with
hogCPUperiod=0. Could you maybe change

----------
core.wait(0.001, hogCPUperiod=0)
----------

to

----------
core.wait(0.001, hogCPUperiod=0.0005)
----------

and see if this changes anything?

Richard

[Nicolas McNair]

Thanks for that Richard. I'll let Marius respond to the particulars, since he wrote that code, but we'll give your suggestions a shot! I don't have the most recent version of my script for checking psychopy.event handy, but I've attached something very close to it for reference. 

[Nicolas McNair]

Hi Richard,

We've implemented all those changes but are still producing similar results. Time differences are anywhere up to 15ms delayed:

5.64
9.52
1.72
12.43
2.41
6.61
0.89
13.74
7.88
15.30

If anyone has some other suggestions for things to try, we'd be happy to give them a go. Otherwise I'm a bit stumped!

Nick

On Monday, February 1, 2016 at 6:59:12 PM UTC+11, Richard Höchenberger wrote:

[Richard Höchenberger]

Wow, this timing variance is just insane...

By the way, have you checked whether the process priorities are actually raised after the first setPriority() calls? You can test by printing the return value of getPriority().

    Richard

--
Sent from my cell phone.

[Nicolas McNair]

Yes they are (using setPriority('normal') instead of setPriority('low') to get the priority to change back down during the ITI). We also made sure we weren't picking up multiple presses (and then reporting the rt from the last of them).

[Richard Höchenberger]

Hi Nicolas, did you make any progress on this issue in the meantime?

Cheers,

Richard

[Nicolas McNair]

Hi Richard,

Unfortunately, no. We've tried a few things, but we always get differences between the psychopy.iohub times and the BlackBox times of between 1-15ms (iohub always being longer). The psychopy.event times have all been a millisecond or less different.

Cheers,

Nick

[Jon]

Hi Nick,

I think the priorities setting is a red herring here (you shouldn't be seeing such big and consistent diffs simply from thread priority).

Given the numbers what leaps to my mind is that they're suspiciously like a fraction of a frame interval. Are you sure that the reset/sync of the iohub clock in your code is tied correctly to the screen flip? If iohub measured t=0 from when you called stim.draw() but BBTK measures t=0 from the screen refresh then this is exactly the timing I'd expect.

cheers

Jon

[Nicolas McNair]

Just checked Marius' code, and he has waitBlanking on the window set to True and records the time on the clock immediately after the flip. So it should be giving the precise time of the flip, not the draw.

N

[Marius Mather]

Hi Jon,

I agree that it looks like the timing might be affected by screen refreshes but I'm a bit unsure about what could be done about it. 

> Are you sure that the reset/sync of the iohub clock in your code is tied correctly to the screen flip?

I don't reset or sync the iohub clock in my testing code, since the timestamps attached iohub keyboard events claim to be "in the same time base that is used by psychopy.core.getTime()". So I get the screen flip timestamp from the return value of the flip() call, and then subtract that from the keyboard event timestamp. So: trial_start_time = WINDOW.flip() to provide t=0 for the trial and then trial_rt = event.time - trial_start_time.

Unless iohub's clock is somehow out of sync with psychopy's, this should be the way to calculate the RT, no? Even then I'm not sure if that's the problem, since we've also tried getting the timestamp from iohub just after the flip like WINDOW.flip(); trial_start_time = Computer.getTime(), with the same results. Since iohub's data collection is running on a separate thread I can't see where the code would be getting blocked by a screen refresh, but there may be something I'm missing. If there are some unadvertised methods in the iohub classes that would allow me to have a look at the sync between the iohub clock and the psychopy.core clock, I could have a look at that as well.

Thanks,

Marius

[Sol Simpson]

Sorry for my absence in this discussion.

 We did some more testing and found that (unexpectedly and bizarrely) iohub actually produces worse times than the event module in this situation. .....
waitforkeys()

12.2

...

getpresses()

6.2

14.6

...

iohub is running in a completely separate process (not thread) than your experiment. Assuming you are running on a multi core computer, what your experiment process is doing should have little to no influence on what iohub is doing. 

What are the specs of the computer and OS you use to test?

On Windows, the iohub process still polls at regular intervals for mouse and keyboard events; the default polling rate is set in the psychopy/iohub/default_config.yaml.  In the current psychopy/psychopy master source the iohub kb / mouse polling rate is set to 0.01 sec (10 msec).

windows_msgpump_interval: 0.01

Given this polling interval, assuming

•  the actual keyboard events are random
•  the kb device has low / no debounce delay and 1 kHz polling

 I would expect the dt between when the event actually occurred to when it is timestamped by iohub to be up to 10 msec, 5 msec on average.

So if you have a keyboard that is capable of being polled at 1000hz, you should set this windows_msgpump_interval to 0.001 or even 0.0005 in that file, or via a custom iohub keyboard device config in/with your test script.

Once you do that the timing should be more like what you were expecting I think. Let us know how it goes.  

Another thing you may want to try doing is using KEYBOARD.getEvents(), which has less client side 'wrapping' of the keyboard event access. I would not expect there to be a significant difference between calling KEYBOARD.getEvents() and KEYBOARD.getPresses(), but maybe there is a small one.

 

As an aside, my memory was that the default iohub polling rate for the keyboard / mouse on Windows was set to 1 msec, not 10 msec, so I was a little surprised to see that myself when I checked today. Looking at the change history for that file, on April 24th, 2014, it was set to 1 msec by a commit I made. Then in Sept 2014 I made another commit that changed the default kb/mouse polling rate to 10 msec. I do not recall if there was some 'logic' behind me making this change, or if it was an unintentional change left over from some testing I was doing during development. Regardless, it is what it is now, which is 10 msec by default.

 It may also be that there's something odd we're doing with our script so I'll add that as soon as I can grab it off the computer.

Getting a copy of your test script would be great. If you could also email me directly I'll be more likely to see your reply sooner than later given my current schedule. 

 If there are some unadvertised methods in the iohub classes that would allow me to have a look at the sync between the iohub clock and the psychopy.core clock, I could have a look at that as well.

You can get the current iohub process time by calling:

ioserv_time = IOHUB_SERVER.getTime()

This sends a 'getTime' request from your experiment to the iohub process and returns after it gets the reply.  Therefore small variability seen if doing something like the following

dt = core.getTime() - IOHUB_SERVER.getTime()

will largely (fully) be due to the request-response delay in the iohub getTime() call, not because of sync issues between the psychopy and iohub time bases. Both psychopy and iohub use identical clocks (that is core.getTime()). You can test the general iohub round trip delay for the computer you are running on by running run.py in the psychopy\demos\coder\iohub\iohubdelaytest directory.

Just to be clear though, this topic is related to the amount of extra runtime delay iohub adds in accessing events compared to accessing the device directly, and is completely different than the keyboard event timestamp differences you are reporting. When using iohub,  events are processed and timestamped when they are received by the iohub process (via callback or polling depending on the device type), not when your experiment  asks for them. 

BTW, there was also some testing and a discussion in 2014 about keyboard event delays, I'll try and find it and send it to you. It was interesting and showed when iohub is useful for keyboard timestamping and when, as is the case in your flip() then wait for kb event test, it is basically the same as using event.getKeys(). 

Thank you

[Sol Simpson]

Here is the link to the keyboard event delay discussion / tests I mentioned. Thanks,

https://groups.google.com/d/msg/psychopy-dev/u3WyDfnIYBo/ZlzOSOCwusEJ

The images in the linked post seem to be broken URLs now. So I just sent a reply to the thread with the images, in the same order as they are discussed in the original post..

[Nicolas McNair]

Hi Sol,

The computer specs are:

CPU: Intel Xeon 3.6Ghz (4 cores)

RAM: 16Gb

Video: NVidia Quadro K620 (2Gb)

Monitor: Benq CRT @ 85Hz

SSD Harddrive running Windows 7 (64-bit) with PsychoPy 1.83.03 installed

I changed the windows_msgpump_interval in the default_config.yaml file to 0.001 and still got the following time errors:

2.42
0.81
10.33
9.07
1.72
6.35
5.00
2.82
10.03
1.63

Then changed it to 0.0005 and produced largely similar results:

7.81
13.78
4.27
13.53
5.04
6.73
10.17
2.65
8.16
13.39

I also tried using getEvents() instead of getPresses():

15.37
15.67
8.83
10.86
7.26
13.29
8.56
2.08
3.11
2.38

I also retested psychopy.event.waitKeys() and again got errors of 1ms or less.

The scripts are already attached to this thread, but I will e-mail them to you as well.

Thanks!

Nick

[Sol Simpson]

Hi Nick,

Thanks for the reply and extra info. 

To help me understand the situation better, can you please explain how you are calculating keyboard timestamp 'error' or 'usb polling rate' based on the test scripts you attached? They seem to use the kb event's timestamp and a known stim onset time (white screen I gather), i.e. win.flip() time,  to calculate RT's. In other words, what is being done to come up with the output like:

I changed the windows_msgpump_interval in the default_config.yaml file to 0.001 and still got the following time errors:

2.42

0.81

10.33

.....

1.63

It is not clear to me what is being done with this data that results in being able to state that all psychopy.event kb timestamps are within 1 msec of what is expected vs. iohub have larger and more variable results as given above.

It almost seems to me that what is being called 'error' in this discussion is really a measure of the amount of synchronization between a kb event timestamp and a screen.flip() time (or multiple thereof). In which case the results seem to just show that in the psychopy.event test script kb event times are being set / synchronized to win.flip() times, while the iohub kb event times are not, which is what would be expected. 

Unless the stim onset (win.flip) itself is being used to electronically trigger button(keyboard) events out of the blackbox USB response box, kb event times should be independent of the computers retrace onset and rate. The iohub data seems to make sense in this context.

Sorry if I'm missing something obvious here.

Thanks,

[Nicolas McNair]

Hi Sol,

Sure. We are getting the 'true' reaction times from an external testing device (the BlackBox) that measures the time of the screen flip via a photodiode and the key press (from the response pad) via a high speed serial connection. We have previously tested this particular computer/CRT set-up and found that there is essentially no delay between when the computer records the flip command and when the it is actually drawn on the monitor. The response pads are also connected to the computer via USB, simulating a keyboard. We then compare the time of the keypress to the monitor draw (the RT) as recorded by the computer to that of the BlackBox. The times I've reported are the differences between what the computer has recorded as the RT and what the BlackBox has recorded as the RT. The psychopy.event RTs were a millisecond or less different from the BlackBox RTs. The iohub times were all longer by the given amounts.

Cheers,

Nick

--
You received this message because you are subscribed to a topic in the Google Groups "psychopy-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/psychopy-users/0B14e8MsQWg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to psychopy-user...@googlegroups.com.

To post to this group, send email to psychop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/37ced79f-86d4-4528-862e-77ce6e7bd8ef%40googlegroups.com.

[Sol Simpson]

Hi Nick,

Thanks for the explanation; that makes sense. 

Given the two test scripts are somewhat different (other than the use of .event or .iohub), I thought I would modify BBTK_Test.py so that it simultaniously got the key event time from both event.getKeys() and iohub's keyboard.getKeys(). Please find it attached; the changes are minimal. This way the difference in actual event timestamps between the .event and .iohub based events can be calculated on a per trial basis.

Note that I did change the psychopy/iohub/default_config.yaml "windows_msgpump_interval" setting to be 0.001 as discussed earlier. Maybe double check that you are changing that in the correct path that your python env. is picking psychopy up from?

I see no real difference in kb timestamps between .event and .iohub frameworks when the script is doing the same trial pattern as you originally had: i.e. psychopy is not flipping each frame and can busy loop checking for keyboard events during the trial response period, 

Times are in sec.msec format....

io_evt_time pyglet_evt_time time_diff

5.408539 5.409144 0.000604

8.325651 8.326177 0.000526

11.227780 11.228298 0.000518

13.998863 13.999393 0.000531

16.806998 16.807533 0.000535

19.761224 19.761749 0.000525

22.561366 22.561885 0.000520

25.377539 25.378078 0.000539

28.177562 28.178110 0.000548

30.985706 30.986249 0.000544

The .event (pyglet based) times are consistently about 0.5 msec later than the iohub timestamps. This is exactly what was found in the similar test condition in the posting I linked to yesterday. 

This seems to suggest that timestamp differences are not the reason for the psychopy RT vs BBTK RT differences you were reporting. Maybe it is due to some difference in the two test scripts, other testing condition differences, or perhaps because the BBTK and psychopy + iohub do not like to run together at the same time. ;) No idea. Can you run the updated script and see what you get in terms of psychopy side RT differences and psychopy vs. BBTK RT differences?

I also added a variable, FLIP_EVERY_FRAME,  that lets you control if the display is flipped() every frame or not during the trial response period. This is the type of situation where iohub should have an advantage over .event, since win.flip() blocks the experiment script process until it returns. 

When FLIP_EVERY_FRAME is set to True, here is an example of what I get for key event timestamp differences between .event.getKeys() and iohub's getKeys():

io_evt_time pyglet_evt_time time_diff

5.516780 5.519467 0.002687

8.458923 8.469436 0.010513

11.360058 11.369394 0.009336

14.396208 14.402556 0.006348

17.281358 17.285924 0.004565

20.162508 20.169165 0.006657

23.125669 23.135642 0.009972

26.226827 26.235581 0.008754

29.160942 29.168828 0.007886

32.095104 32.102222 0.007118

As expected, iohub timestamps the same keyboard input earlier than psychopy.event because iohub still processing keyboard events when the experiment process is in a  win.flip() induced blocked state. The time difference between the two timestamps will vary each time you run it if you are manually making button responses, but should always be less than the retrace interval of your video card. Running it a few times now, I have seen differences as small as 0.6 msec and as large as 12.5 msec (my monitor is running at 60 Hz).This is also the same as what was found in the similar test condition in the posting I linked to yesterday. 

Anyhow, would be good to find out what happens when this is run on your test rig with the BBTK. Please let me know.

Thanks again,

Sol

[Nicolas McNair]

Hi Sol,

I've done some more testing. Running the script your provided produced the following errors in milliseconds (PsychoPy RT - BlackBox RT):

.event	.iohub
14.54057	14.23652
8.331786	8.066382
8.606956	8.317403
7.667577	7.390098
2.454527	2.186708
9.854391	9.560612
14.14973	13.88855
8.431116	8.169033
11.06313	10.76815
4.008676	3.721839

Here I get both .event and .iohub reporting the large, variable errors (iohub's is slightly smaller, around 0.3ms or so).

I then tried commenting out all the code referring to iohub and the reran the script, and got the errors of a millisecond or less:

0.52109 0.34109 0.1109 0.146149 0.554294 0.65358 0.436943 0.280299 0.608903 0.352158

I then reversed this, and commented out all the code relating to recording RTs via the event module (and decommented the .iohub code). This produced the following:

9.332096
11.16157
8.615619
2.290314
1.148599
10.1034
10.92779
4.160154
1.768414
12.09875

Finally, I went back to commenting out all the .iohub code, leaving only the .event keyboard monitoring. I then added back in only the code related to starting to iohub server:

from psychopy.iohub import launchHubServer

io = launchHubServer()

Doing this I get the following errors from the .event module:

4.322792
7.089614
4.395806
6.115912
8.145487
6.037905
7.621172
5.952045
14.07874
6.749861

So it seems that launching the iohub server is doing something that interferes with the timing for both iohub and event.

Cheers,

Nick

--

You received this message because you are subscribed to a topic in the Google Groups "psychopy-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/psychopy-users/0B14e8MsQWg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to psychopy-user...@googlegroups.com.
To post to this group, send email to psychop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/01ea1232-536c-4770-8385-c207d8414fd4%40googlegroups.com.

[Nicolas McNair]

P.S. Tested this on a second computer/CRT set-up and got the same results.

[Sol Simpson]

Sorry for the long reply here, but I think this is both important and very interesting to get to the bottom of.

 

So it seems that launching the iohub server is doing something that interferes with the timing for both iohub and event.

Obviously adding the iohub process is affecting the psychopy side vs BBTK side rt calculation.

However I do not think the test results show that " iohub server is doing something that interferes with the timing for both iohub and event.". Instead the results suggest that iohub is /not/ interfering with either .iohub and .event kb timestamping. Reasoning: Given you found the following results when psychopy + iohub + bbtk were running:

.event	.iohub
14.54057	14.23652
8.331786	8.066382
8.606956	8.317403
7.667577	7.390098
2.454527	2.186708
9.854391	9.560612
14.14973	13.88855
8.431116	8.169033
11.06313	10.76815
4.008676	3.721839

This means that both .event and .iohub were giving the about same timestamp to each trial kb event (within 0.5 msec of each other). The only way the iohub and psychopy key event timestamps could be ~=  is if both the psychopy and iohub processes were able to be constantly checking for keyboard events. This is the same, small, timestamp difference I found running the script on my PC. 

Do you agree? 

An example alternative explanation is that adding the iohub process, or even just adding any other process that is cpu intensive, is causing flip() timing issues on your PC. This would result in RT calculation differences / errors and not cause issues with .event vs .iohub kb timestamps. However this explanation has issues: if the iohub is causing inaccuracies in the reported flip time, why is it not also causing variability in the .event vs .iohub kb timestamps?

It would be great to try and figure out why adding the iohub process (or maybe it is just the act of adding another cpu intensive process??) into the mix causes issues in the rt error calculations on your setup.

Three question first please:

Q1: What is the exact Xeon model you are using?

Q2: What are you using to start your python script? an IDE, Coder, or from a windows cmd.exe?

Q3: When you described how the rt was being calculated by the BBTK, you said a high speed serial connection was being used to inform the BBTK of when the button press occurred. What is this source of the serial connection to the BBTK? The BB usb response box itself, or the psychopy computer? If the psychopy computer, does that mean there is some BB usb response box software running on the psychopy computer that feeds back keyboard events that the OS reports?

Some suggested ways to investigate what is actually happening; I would do these myself if I had a BBTK on hand but I do not. Some of these ideas are only worth trying if the serial connection reporting button press times back to the BBTK is from the psychopy pc. These I put a '*' at the start of.

1. Save out what the cpu core affinity is for the psychopy and iohub process. Add something like the following after you have created the iohub server connection
   
   
   # Note: If the cpu has hyperthreading enabled,
   # actual core count is 1/2 what is reported.
   print "Num CPU Cores:",Computer.getProcessingUnitCount()
   print "psycho, iohub proc affinities:", Computer.current_process.cpu_affinity(),Computer.iohub_process.cpu_affinity()
   
   
   
2. Force Windows to use different CPU cores for psychopy and iohub.Add something like the following after you have created the iohub server connection:
   
   
   # You may need to be running in an admin account for this to work....
   pucount = Computer.getProcessingUnitCount()
   
   # Assuming you have a 4 core cpu if computer reports 4 cores
   # then there is no hypertheading. If 8 cores are reported, then
   # hyperthreading is enabled.
   if pucount == 4:
       #try using the second core for psychopy and the third for iohub.
       psycho_cores = [1,]
       iohub_cores = [2,]
   elif pucount == 8:
       # If there is hyperthreading, then set each
       # hthread for the relevent core.
       psycho_cores = [2,3]
       iohub_cores = [4,5]
   else:
       raise ValueError("Reported core count != 4 or 8: %d"%pucount )
   
   Computer.setProcessAffinities(psycho_cores,iohub_cores)
   
   print "psycho, iohub proc affinities:", Computer.current_process.cpu_affinity(),Computer.iohub_process.cpu_affinity()
   
   
3. *What happens in terms of the .event vs .iohub timestamp difference for each trial if you run the exact same test script but do not have the BBTK 'feedback' connection attached and have ensured that any BBTK software processes or services / drivers associated with this feedback connection are disabled / not running on the psychopy computer.
   
   
4. What happens if you run the test script as you would normally (so no iohub) but first start a separate python script (say busywait.py) that simply busy loops for 30 seconds, or however long you need it to run so that you can collect the RT data from the test script that is also running. This busywait.py could be manually launched from a command prompt just before you start the test script, so it does not need to have any fancy multiprocessing code in it at all.
   
   
5. Can you run the psychopy\demos\coder\timingtimeByFrames.py demo?What are the results when you:
1. Run the script as is?
2. Add the line to start the iohub server at the start of the script and run it?
3. *Do 1 and 2 with the BBTK connection attached and client side software running
4. *Do 1 and 2 without the BBTK connection attached and client side software stopped / disabled.  
   
   
6. Can you run the psychopy\demos\coder\iohub\iohubdelaytest\run.py and send me the results? If the iohub request-response delays are larger than expected, this would suggest that the computer may have trouble running two high load processes smoothly or that other processes / services are interfering.
    
7. Try running the test on a different PC that has a different 4 core CPU.
   
   
8. Use something other than the BBTK to do the RT calculations:

• a Teensy 3 MCU and the iosync firmware and client side software.  This is what was used for these tests, and obviously iohub was not interfering with psychopy kb timing.
• Chris Rorden's StimSync software and hardware, which also uses a Teensy 3 and has better documentation on how to do retrace detection and keyboard / button press timing tests.

Thanks again.

PS: Another reason I do not think the issue is related to iohub causing 'issues with .event vs .iohub kb event time stamps' is because I tested the full end to end delay between when a keyboard event actually happens and when the event is time stamped by both .iohub and .event. I tested this using a standard keyboard as well as an usb device that, like the BB usb response pad, can be programmed to generate hid keyboard events extremely quickly following an event on the keyboard emulating device. I posted it before, but here is a link to the thread that describes the test and results in detail:

https://groups.google.com/forum/#!topic/psychopy-dev/u3WyDfnIYBo

The most important test condition for this discussion was the one called "waitkeys_keypress Test" with the "Teensy 3 ioSync Keyboard". It tested what amount of a keyboard event's delay (error between kb event timestamp vs. when the event was actually generated) is from software vs hardware. Timestamp accuracy (event delay) for both .event and .iohub were calculated in parallel. Here is the results plot, please see the original posting for the details on how this was done.

This is the most direct test I can think of that shows the delay that Windows and the psychopy software (.event or .iohub) adds to kb event timestamp error (uncorrected delay) is very low: under 2.5 msec max, avg of 1 and 1.5 msec for .iohub and .event respectively. 

It also shows the difference between .iohub and .event kb timestamps. is about 0.5 msec. this is the same difference we both find in .event vs .iohub kb timestamps using the current test script in all conditions.

[Sol Simpson]

Hi Nick, Just saw this now, so I guess suggested test #7 is already done. 

[Nicolas McNair]

Hi Sol,

I've gone through and addressed each question in turn below.

Q1: What is the exact Xeon model you are using?

> I wasn't able to do any testing on the Xeon computer (it's currently being used for running subjects). I was able to test on the second set-up, which had reported similar results. This has an Intel i7-4770 with 4 cores/8 threads, 16Gb RAM, and running the latest PsychoPy on a SSD.

Q2: What are you using to start your python script? an IDE, Coder, or from a windows cmd.exe?

> I've tried running the script from within the PsychoPy and from the command line and found no difference in the reported errors (i.e., there are still large errors using cmd).

Q3: What is this source of the serial connection to the BBTK?

> The BBTK is not connected to the PsychoPy computer. It collects the timing itself and then reports the results to a separate laptop.

1. Save out what the cpu core affinity is for the psychopy and iohub process + 2. Force Windows to use different CPU cores for psychopy and iohub

> Adding the code related to examining cores & affinities reported 8 cores. The affinities at the beginning for psychopy and iohub were [0,1,2,3,4,5,6,7]. After forcing windows to use different cores, the affinities were [2,3] for psychopy and [4,5] for iohub. This still resulted in timing errors.

4. What happens if you run the test script as you would normally (so no iohub) but first start a separate python script
> This did not make a difference to the output

5. Can you run the psychopy\demos\coder\timingtimeByFrames.py demo?
> Here's the results when run as is:

> Here's the results when run with iohub started:

There doesn't appear to be any significant difference between the two.

6. Can you run the psychopy\demos\coder\iohub\iohubdelaytest\run.py and send me the results?

> I wasn't able to get the iohubDelayTest demo to run. It produced a large number of errors:

##### Running: C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy-1.83.03-py2.7.egg\psychopy\demos\coder\iohub\ioHubDelayTest\run.py #####

Error during device creation ....

<type 'exceptions.TypeError'>

TypeError("'NoneType' object has no attribute '__getitem__'",)

['  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\server.py", line 591, in createNewMonitoredDevice\n    device_instance_and_config=self.addDeviceToMonitor(device_class_name,deviceConfig)\n',

 '  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\server.py", line 781, in addDeviceToMonitor\n    deviceInstance=DeviceClass(dconfig=device_config)\n',

 '  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\devices\\display\\__init__.py", line 62, in __init__\n    self._addRuntimeInfoToDisplayConfig()\n',

 '  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\devices\\display\\__init__.py", line 506, in _addRuntimeInfoToDisplayConfig\n    self._createPsychopyCalibrationFile()\n',

 '  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\devices\\display\\__init__.py", line 660, in _createPsychopyCalibrationFile\n    psychoMonitor.setSizePix(list(self.getPixelResolution()))\n',

 '  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\devices\\display\\__init__.py", line 193, in getPixelResolution\n    return self.getConfiguration()[\'runtime_info\'][\'pixel_resolution\']\n']

Error during device creation ....

<class 'psychopy.iohub.util.exception_tools.ioHubError'>

ioHubError:

Args: ('Error during device creation ....',)

['  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\server.py", line 520, in __init__\n    self.createNewMonitoredDevice(device_class_name,deviceConfig)\n',

 '  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\server.py", line 603, in createNewMonitoredDevice\n    raise ioHubError("Error during device creation ....")\n']

<class 'psychopy.iohub.util.exception_tools.ioHubError'>

ioHubError:

Args: ('Error during device creation ....',)

['  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\launchHubProcess.py", line 35, in run\n    s = ioServer(rootScriptPathDir, ioHubConfig)\n',

 '  File "C:\\Program Files (x86)\\PsychoPy2\\lib\\site-packages\\psychopy-1.83.03-py2.7.egg\\psychopy\\iohub\\server.py", line 524, in __init__\n    raise ioHubError("Error during device creation ....")\n']

Traceback (most recent call last):

  File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy-1.83.03-py2.7.egg\psychopy\demos\coder\iohub\ioHubDelayTest\run.py", line 268, in <module>

    runtime=ExperimentRuntime(module_directory(ExperimentRuntime.run), "experiment_config.yaml")

  File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy-1.83.03-py2.7.egg\psychopy\iohub\client\__init__.py", line 1603, in __init__

    self.hub = self._initalizeConfiguration()

  File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy-1.83.03-py2.7.egg\psychopy\iohub\client\__init__.py", line 1855, in _initalizeConfiguration

    self.hub = ioHubConnection(None, ioHubConfigAbsPath)

  File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy-1.83.03-py2.7.egg\psychopy\iohub\client\__init__.py", line 285, in __init__

    self.iohub_status = self._startServer(ioHubConfig, ioHubConfigAbsPath)

  File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy-1.83.03-py2.7.egg\psychopy\iohub\client\__init__.py", line 943, in _startServer

    isDataAvail=self._serverStdOutHasData()

  File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy-1.83.03-py2.7.egg\psychopy\iohub\client\__init__.py", line 1052, in _serverStdOutHasData

    raise e

pywintypes.error: (109, 'PeekNamedPipe', 'The pipe has been ended.')

Cheers,

Nick

[Sol Simpson]

So the BBTK is using the time of the physical button press and the time the light key triggers for the RT, so this should be the true RT. The psychopy RT is the time from when flip() returns to the timestamp of the button event, so it will be the 'true' RT + the delay from when the button event occurs to when it is actually timestamped by the psychopy software (either .iohub or .event). Do I have have this right?

To summarize the results you are finding:

1. when you run timingtimeByFrames it reports no dropped frames and low interframe interval variance when iohub is running and when it is not. So iohub is not affecting inter frame intervals when using that test. 
   
2. when you run the your test script and get both .event and .iohub kb event times for each trial response, they are the same (within 0.5 msec).
3. when you run 2) without iohub you get BBTK RT vs psychopy RT differences < 1 msec.
4. when you run 2) with iohub you get BBTK RT vs psychopy RT differences as large as 10 msec or so.

BTW, when you posted the results, you did not include what the RT differences are for conditions when FLIP_EVERY_FRAME = True. Can you please run the test script with FLIP_EVERY_FRAME = True, with and without iohub?

This pattern of results seems to raise more questions than are answered! ;)

1. Why in the test script are the .event and .iohub kb event times the same, but the calculated RT by BBTK vs psychopy different? The most likely answer would seem to be that iohub is somehow effecting the times returned by win.flip() by not the .event timestamping. This fits the data, but seems odd.
2. If the suggested answer to 1) is correct, the why does iohub not affect the timeByFrames script results at all.
3. If the BBTK is measuring the actual RT, and 
1. you get <= 0.65 msec difference between the BBTK RT and psycho RT with the test script when iohub is not running, 
2. you get .event kb times that are consistently about 0.3 msec later than the .iohub kb timestamp of the same event.
   

then this must mean that the delay between when the BB response device sends the button press event and when the event is timestamped psychopy is <= 0.65 msec for .event and <=0.35 msec msec for .iohub (3.1_time - 3.2_dt given above). This is relatively consistent with what was found in the  "waitkeys_keypress Test" with the "Teensy 3 ioSync Keyboard"  test and data I mentioned in my last post. It is about 1/2 msec faster actually.

So overall, everything shows that iohub is working fine with psychopy, except in the case of the BBTK RT test script. So what is actually changing when iohub is running in your test script that causes the RT differences to  go up to 10 msec but not effect the keyboard event timestamping of either .event or .iohub? 

Very puzzling.

Only other thing to try that I can think of would be to change the test script so that, while it is looping waiting for .getKeys() from .event and .iohub, insert a small sleep period of 0.5 or 1 msec using the OS sleep function. Perhaps having the psychopy process eat up 100% of a CPU core during this period, as it is now, is interacting with iohub running in some way. 

Regarding:

 I wasn't able to get the iohubDelayTest demo to run. It produced a large number of errors:

It is likely that the script does not have write permissions for the C:\Program Files (x86)\...., so please make a copy of the ioHubDelayTest folder and paste it somewhere that you have write permissions for (like your Desktop). Also ensure you are running the script with the ioHubDelayTest folder as the cwd; for example by going to the newly created test folder in a cmd.exe and starting it from there.

If that fixes the issue, then this may also mean that the process affinity test you did may not have actually change the process affinities. When you ran that test, did printing outComputer.current_process.cpu_affinity(),Computer.iohub_process.cpu_affinity() after the call to change the affinities actually return [2,3], [4,5]?

Thanks again,

Sol

[Nicolas McNair]

Hi Sol,

Yes that's right.

I've run the script using flip_every_frame. With both iohub and event running, then we get the same errors as before for iohub (average around 7ms or so). For the event times, apart from one or two trials were iohub and event coincided, the errors were all longer than iohubs by between 1-10ms. With just event running then the errors are between 1-10ms. So seems fairly additive.

Inserting a 1ms sleep into the while loop added a further small delay to iohub errors (now 9.2ms) and was pretty devastating for the event times (average error 24.8ms). That's with flip_every_frame set to false.

Moving iohubdelay and changing the cwd still did not allow me to run the test. The same error as before.

Cheers,

Nick

[Nicolas McNair]

Could initialising the iohub server cause psychopy to default back to using default (i.e., low-speed) USB polling of the keyboard? 

--

You received this message because you are subscribed to a topic in the Google Groups "psychopy-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/psychopy-users/0B14e8MsQWg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to psychopy-user...@googlegroups.com.
To post to this group, send email to psychop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/83677ccd-5cba-4275-8227-6922a72e0e27%40googlegroups.com.

[Jonathan Peirce]

There isn't a setting of polling rate that could have a default of low-speed. From PsychoPy's perspective it's just literally how often in the code the call to event.getKeys() is made. The waitKeys() code, which gave you the high precision, is essentially just repeatedly calling getKeys() and doing nothing else. But there isn't a polling "rate" as such.

You received this message because you are subscribed to the Google Groups "psychopy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-user...@googlegroups.com.

To post to this group, send email to psychop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/CABw5XkXZ2qMfmMf_f8s9GqUa-2RhvT61QQ%2BSNDRSFR8VahXxyQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

-- 
Jonathan Peirce
University of Nottingham

http://www.peirce.org.uk

This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please send it back to me, and immediately delete it. 

Please do not use, copy or disclose the information contained in this
message or in any attachment.  Any views or opinions expressed by the
author of this email do not necessarily reflect the views of the
University of Nottingham.

This message has been checked for viruses but the contents of an
attachment may still contain software viruses which could damage your
computer system, you are advised to perform your own checks. Email
communications with the University of Nottingham may be monitored as
permitted by UK legislation.

[Richard Höchenberger]

Just wondering, was this issue ever really (satisfactorily!) resolved?

Thanks,

Richard

[Richard Kirkden]

Hi Sol,

Please could you tell me how to set windows_msgpump_interval within my script? The computer I'm using is a client machine that runs PsychoPy from a server, so I don't have access to the default_config.yaml file. (I'm testing a button box that I've made which looks like a keyboard device to the PC. PsychoPy is not performing as well as E-Prime and I'm trying to figure out why. USB polling rate seems to be a possible cause.)

Thanks,

Richard.

On Monday, 15 February 2016 19:01:31 UTC, Sol Simpson wrote:

...

[Sol Simpson]

I do not think so; the results of the tests that Nicolas did do not seem to clarify what the iohub interaction actually is that he is seeing. See this post for a summary of what he tested and the results based on my understanding of things, the short of which are:

To summarize the results you are finding:

1. when you run timingtimeByFrames it reports no dropped frames and low interframe interval variance when iohub is running and when it is not. So iohub is not affecting inter frame intervals when using that test. 
2. when you run the your test script and get both .event and .iohub kb event times for each trial response, they are the same (within 0.5 msec).
3. when you run 2) without iohub you get BBTK RT vs psychopy RT differences < 1 msec.
4. when you run 2) with iohub you get BBTK RT vs psychopy RT differences as large as 10 msec or so.

The only thing I can think is that iohub is still pumping the Windows message queue in 10 msec intervals; which was the default config until this issue was raised.  This would explain why things run OK when both iohub and pyglet are being used to get events, but why there are up to 10 msec differences when iohub was used alone. I believe Nicolas confirmed he had checked / changed the iohub config so that it was pumping at 1 msec intervals and was still seeing the differences though. 

Note that in the latest psychopy master source, iohub is set to pump the Windows msg queue every 1 msec, so if this was the issue it should be fixed.

[Sol Simpson]

And to add, when the test you mention below was run by Nicolas with the addition of getting iohub keyboard events, the time difference between the event.getKeys() and iohub kb event timestamps were about 0.5 msec.

[Sol Simpson]

Please could you tell me how to set windows_msgpump_interval within my script? The computer I'm using is a client machine that runs PsychoPy from a server, so I don't have access to the default_config.yaml file  

If the server is running the latest version of psychopy, or at least the code as of March this year,  then it should already be pumping the windows msg queue every 1 msec. 

After starting the iohub server, you can check what config is being used by iohub by calling:

ioconf = io.getHubServerConfig()

Then check the contents of ioconf for what the value of the windows_msgpump_interval being used is.

If you actually need to change this setting, and can not update the default_config.yaml file in the psychopy.iohub package folder, please let me know how your script is starting iohub and I can send a modified example that changes the setting.

Thanks.

[Richard Höchenberger]

Thank you very much for your help on this, Sol.

As far as I can tell, your change has NOT yet been integrated into upstream master!

Also I am wondering: does this finding imply that all RTs collected with older versions of ioHub, i.e., with windows_msgpump_interval: 0.01 instead of 0.001, carry a jitter of up to 10 ms if one didn't happen to "pyglet-pump" for some reason at the same time? We are planning to run an RT study soon, using the PST SRbox; should I change the windows_msgpump_interval accordingly? Or can I safely leave it at 0.01?

Thanks,

    Richard

[Richard Kirkden]

Thanks for the swift reply. io.getHubServerConfig() returns windows_msgpump_interval=0.01 in PsychoPy v. 1.82.01 and v. 1.83.04.

It would be great if you could send me some code to modify windows_msgpump_interval within my script. I'm starting iohub using launchHubServer().

Will this code work in PsychoPy v. 1.82.01? This is the version that my employer presently makes available for client PCs, but I can ask them to update this if necessary.

[Sol Simpson]

After reviewing the launchHubServer() code, it does support changing a general iohub server level config setting, only settings for iohub devices can be set.

I also downloaded the 1.82.01 source and iohub is using the 10 msec msg pump interval.

So I think your only options are to have your employer either:

1. modify the psychopy/iohub/default_config.yaml file on the server and set windows_msgpump_interval = 0.001 
2. update the psychopy release to the latest source from psychopy/psychopy master branch.

Option 1 would probably be the safer choice unless they were planning to upgrade to a newer version anyhow.

Thanks.

[Sol Simpson]

Thanks for checking this out Richard; you are correct, it has only changed on the isolver/psychopy clone and I can find no record of making a pull request that included the update. My memory must really be shot. ;(

I just updated it to 1 msec directly in psychopy/psychopy master now. So using the latest psychopy/psychopy master source should now include the fix.

Also I am wondering: does this finding imply that all RTs collected with older versions of ioHub, i.e., with windows_msgpump_interval: 0.01 instead of 0.001, carry a jitter of up to 10 ms if one didn't happen to "pyglet-pump" for some reason at the same time?

On Windows OS yes, for psychopy/psychopy master source code between Sept 2014 and now.  Other OS's should not have this issue though.

 

We are planning to run an RT study soon, using the PST SRbox; should I change the windows_msgpump_interval accordingly? Or can I safely leave it at 0.01?

You should change it to 0.001 (1 msec) so kb and mouse event timestamps are as good as they can be. As you point out, this is especially true in experiments that do not repeatedly call a psychopy function that intern calls pyglet's dispatchEvents(), which is what pumps the msg queue on Windows.

 

[Richard Höchenberger]

Thanks Sol!

Just to be clear, did this "problem" only affect keyboards and mice, or ANY device (including serial, USB, ...)? Could explain why I didn't find significant effects in my last RT study using the serial response box *cough* ;)

    Richard

[Jonathan Peirce]

Does the PST serial box return the time from its own clock or does it also depend on the time it was noticed by the system?

--
You received this message because you are subscribed to the Google Groups "psychopy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-user...@googlegroups.com.
To post to this group, send email to psychop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/CAGm-8jNzfak%2Bu3aiiw_o7z4BzajXwFfQX9E2BMF1P1TdTCUP%2Bw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Richard Höchenberger]

On Tue, May 10, 2016 at 2:14 PM, Jonathan Peirce <jon.p...@gmail.com> wrote:

Does the PST serial box return the time from its own clock or does it also depend on the time it was noticed by the system?

The button events are time-stamped by ioHub (not the box itself!) based on iohub.Computer.getTime().

Thanks,

    Richard