iohub stimulus information

[Silvia]

Hello,

I have set up an reaction-time + eye-tracking experiment using iohub.  Everything is working well except for my data output - I get eye data and keyboard data sync'd and sent to the hdf5 file, but my stimulus information only appears in the log file (along with the keyboard data).

So I am wondering how to include my stimulus information (ie time and type of display) along with the keyboard and eye data using iohub?

Just as an aside - I thought of comparing keyboard times between the hdf5 file and the log file (and if there was a constant difference to simply subtract that from the stimulus times in the log file), but the difference in timing was not constant - and varied from 0 to 33 ms (which is long enough to be problematic).  I suspect the timing of iohub to be more accurate than the log file, but I wonder how I can verify that?

Thank you,

Silvia

PS, Here are some relevant snippets of my code:

...

## stuff relevant to setting up eyetracker & iohub ##

eyetracker = True

if eyetracker:

    try:

        from psychopy.iohub import EventConstants,ioHubConnection,load,Loader

        from psychopy.iohub.util import NumPyRingBuffer

        from psychopy.data import getDateStr

        # Load the specified iohub configuration file converting it to a python dict.

        io_config=load(file('SRR_eyelink_std.yaml','r'), Loader=Loader)

        # Add / Update the session code to be unique. Here we use the psychopy getDateStr() function for session code generation

        session_info=io_config.get('data_store').get('session_info')

        session_info.update(code="S_%s"%(getDateStr()))

        # Create an ioHubConnection instance, which starts the ioHubProcess, and informs it of the requested devices and their configurations.

        io=ioHubConnection(io_config)

        iokeyboard=io.devices.keyboard

        mouse=io.devices.mouse

        experiment = io.devices.experiment

        if io.getDevice('tracker'):

            eyetracker=io.getDevice('tracker')

            win.winHandle.minimize()

            eyetracker.runSetupProcedure() ##setup and calibration are here

            win.winHandle.activate()

            win.winHandle.maximize()

    except Exception, e:

       import sys

       print "!! Error starting ioHub: ",e," Exiting..."

       sys.exit(1)

    display_gaze=False

    x,y=0,0

...

## stuff relevant to looping through the stimulus table ##

startTime = globalClock.getTime()

t=0; DisplayClock.reset()

frameN=-1

trialCount = 0

i = 0;

if eyetracker:

    heldFixation = True #unless otherwise

    io.clearEvents('all')

    eyetracker.setRecordingState(True)

for i in xrange(0, len(stimulus_table)):

    stim = stimulus_table[i];

    txt = "%d" % stim

    cue.setText(txt)

    isi = 60 # mask time

    ISI = 72 #total time

    DisplayComponents=[]#to keep track of which have finished

    DisplayComponents.append(mask)

    DisplayComponents.append(circle)

    for thisComponent in DisplayComponents:

        if hasattr(thisComponent,'status'): thisComponent.status = NOT_STARTED

    

    #-------Start Routine "Display"-------

    continueRoutine = True

    while continueRoutine:

        # get current time

        t = DisplayClock.getTime()

        frameN = frameN + 1  # number of completed frames (so 0 is the first frame)

        # update/draw components on each frame

        

            

        #*cue* updates

        if frameN>=0 and cue.status==NOT_STARTED:

            #keep track of start time/frame for later

            cue.tStart=t#underestimates by a little under one frame

            cue.frameNStart=frameN#exact frame index

            cue.setAutoDraw(True)

        elif cue.status==STARTED and frameN>=(cue.frameNStart+12):

            cue.setAutoDraw(False)

    

        #*mask* updates

        if (cue.status==FINISHED) and mask.status==NOT_STARTED:

            #keep track of start time/frame for later

            mask.tStart=t#underestimates by a little under one frame

            mask.frameNStart=frameN#exact frame index

            mask.setAutoDraw(True)

                

        elif mask.status==STARTED and frameN>=(mask.frameNStart+isi):

            mask.setAutoDraw(False)

        if not continueRoutine:  # a component has requested a forced-end of Routine

            break

        continueRoutine = False  # will revert to True if at least one component still running

        for thisComponent in DisplayComponents:

            if hasattr(thisComponent, "status") and thisComponent.status != FINISHED:

                continueRoutine = True

                break  # at least one component has not yet finished

        

        # refresh the screen

        if continueRoutine:  # don't flip if this routine is over or we'll get a blank screen

            win.flip()

    trialCount=trialCount + 1

for thisComponent in DisplayComponents:

    endTime=globalClock.getTime()

    logging.flush()  #store info to log

    if hasattr(thisComponent,"setAutoDraw"): thisComponent.setAutoDraw(False)

    

if eyetracker:

    eyetracker.setRecordingState(False)

    io.quit()

win.close()

core.quit()

[Michael MacAskill]

> On 2/02/2016, at 11:30, Silvia <silvia.l...@gmail.com> wrote:
>
> I have set up an reaction-time + eye-tracking experiment using iohub. Everything is working well except for my data output - I get eye data and keyboard data sync'd and sent to the hdf5 file, but my stimulus information only appears in the log file (along with the keyboard data).
>
> So I am wondering how to include my stimulus information (ie time and type of display) along with the keyboard and eye data using iohub?

Hi Silvia,

You haven't really told us anything about your stimulus information and where it comes from. But assuming you are using a TrialHandler to iterate over your trials (whether in your own code or by a loop in Builder), you need to let ioHub know of the existence of that TrialHandler so that it will integrate its variables into its HDF data store. e.g. from <http://www.psychopy.org/api/iohub/starting.html>:

> createTrialHandlerRecordTable(trials)
> Create a condition variable table in the ioHub data file based on the a psychopy TrialHandler. By doing so, the iohub data file can contain the DV and IV values used for each trial of an experiment session, along with all the iohub device events recorded by iohub during the session. Example psychopy code usage:
>
> # Load a trial handler and
> # create an associated table in the iohub data file
> #
> from psychopy.data import TrialHandler,importConditions
>
> exp_conditions=importConditions('trial_conditions.xlsx')
> trials = TrialHandler(exp_conditions,1)
>
> # Inform the ioHub server about the TrialHandler
> #
> io.createTrialHandlerRecordTable(trials)
>
> # Read a row of the trial handler for
> # each trial of your experiment
> #
> for trial in trials:
> # do whatever...
>
>
> # During the trial, trial variable values can be updated
> #
> trial['TRIAL_START']=flip_time
>
> # At the end of each trial, before getting
> # the next trial handler row, send the trial
> # variable states to iohub so they can be stored for future
> # reference.
> #
> io.addTrialHandlerRecord(trial.values())


Regards,

Michael



--
Michael R. MacAskill, PhD 66 Stewart St
Research Director, Christchurch 8011
New Zealand Brain Research Institute NEW ZEALAND

Senior Research Fellow, michael....@nzbri.org
Te Whare Wānanga o Otāgo, Otautahi Ph: +64 3 3786 072
University of Otago, Christchurch http://www.nzbri.org/macaskill

[Silvia]

Hi Michael, 

Thank you for your prompt response.  Actually I am not using a TrialHandler - is this be necessary?  I have a particular method of setting up the stimulus table in Psychopy ahead of running the experiment by shuffling a pattern of stimuli in a particular way, then building a stimulus table from that. I am not sure about loading the table in from an excel spreadsheet as I am unaware how to shuffle the pattern appropriately for each subject using that method.

Currently I am iterating over my trials as follows (from the second snippet posted earlier):

i = 0;

...

for i in xrange(0, len(stimulus_table)):

    stim = stimulus_table[i];

    txt = "%d" % stim

    cue.setText(txt)

where that last line determines what is displayed during the "cue" component to begin each trial.  I hope that makes it a bit clearer..?  I am not even that concerned about keeping a list of what is presented, just keeping track of the timing of the "cue" component within the "Display" routine.

Thanks again,

Silvia

[Michael MacAskill]

Hi Silvia,

Using a spreadsheet is optional, you can certainly pass a TrialHandler a list of stimulus values directly. e.g. here is an example from the TrialHandler.py demo:

stimList = []
for ori in range(90,180,30):
for sf in [0.5, 1.0, 2.0]:
stimList.append(
{'sf':sf, 'ori':ori} #this is a python 'dictionary'
)

#organise them with the trial handler
trials = data.TrialHandler(stimList,10,extraInfo= {'participant':"Nobody",'session':1})


But just sticking with your own code,it could be simplified markedly by being a bit more Pythonic:


for stim in stimulus_table:
cue.setText(stim)

> I am not even that concerned about keeping a list of what is presented, just keeping track of the timing of the "cue" component within the "Display" routine.

Not really sure what you mean by that. I guess the time it is displayed? If so, just after the relevant win.flip() command, you can access the time and store it in whatever fits the way you are working.

Using a TrialHandler, it would look something like this:

e.g:

stimulus_table = [] # this list will contain dictionaries rather than strings directly
for text in ['cat', 'dog', 'rat']:
stimulus_table.append({'stim': text,
'cue_time': ''}) # leave this blank to be filled in later


trials = data.TrialHandler(trialList= stimulus_table)

timer=core.clock()

for trial in trials:
cue.setText(trial['stim'])
cue.draw()
core.wait(blah blah some random period)
win.flip()

trial['cue_time'] = timer.getTime()


# when finished, export the data in a flat file (but you could do this with ioHub I guess):
trials.saveAsWideText('datafile.csv')


Regards,

Michael

> On 4/02/2016, at 10:56, Silvia <silvia.l...@gmail.com> wrote:
>
> Hi Michael,
>
> Thank you for your prompt response. Actually I am not using a TrialHandler - is this be necessary? I have a particular method of setting up the stimulus table in Psychopy ahead of running the experiment by shuffling a pattern of stimuli in a particular way, then building a stimulus table from that. I am not sure about loading the table in from an excel spreadsheet as I am unaware how to shuffle the pattern appropriately for each subject using that method.
>
> Currently I am iterating over my trials as follows (from the second snippet posted earlier):
>
> i = 0;
> ...
> for i in xrange(0, len(stimulus_table)):
> stim = stimulus_table[i];
> txt = "%d" % stim
> cue.setText(txt)
>
> where that last line determines what is displayed during the "cue" component to begin each trial. I hope that makes it a bit clearer..? I am not even that concerned about keeping a list of what is presented, just keeping track of the timing of the "cue" component within the "Display" routine.
>
> Thanks again,
>
> Silvia

[Sol Simpson]

Sorry if this reply is too late to help any....

So I am wondering how to include my stimulus information (ie time and type of display) along with the keyboard and eye data using iohub?

The trialhandler with iohub integration approach suggested by Michael is definitely a useful one.

If you are going to be working with the hdf5 file and are comfortable accessing / filtering events from it using an api like pytables or other,

then another simple way to store experiment generated events to the hdf5 file, like stim onset, is to send iohub experiment messages when

the event occurs.

# Assuming earlier code:
#     1) created a psychopy window called 'win'.
#     2) Started iohub and have called it 'io'

mystim.draw()
flip_time = win.flip()

# send iohub an experiment message event
# that will have a timestamp equal to the current
# psychopy.core.getTime()
io.sendMessageEvent("STIM ONSET")
 
# If you want to use the exact time that
# the call to flip returned as the msg time:
io.sendMessageEvent("STIM ONSET", sec_time=flip_time)

Now you will have keyboard events, eye tracker events, and experiment message events all in the hdf5 file. 

All event times use the same time base (equal to psychopy.core.getTime()), so calculating the time difference

between any event type in the hdf5 file is done by simple subtraction (i.e. evt1.time - evt2.time).

It is really important to remember that all events generated by iohub 

use the same clock as psychopy.core.getTime(), which gives the time since 

the experiment script started. This is the same timebase that win.flip() uses 

when it returns the time that the flip() occurred. Therefore, you should never compare an 

iohub event time with the time returned by a Clock() instance if you create them in your experiment. 

I suspect the timing of iohub to be more accurate than the log file, but I wonder how I can verify that?

For a description of tests I did that calculate the difference between when an actual real world keyboard event occurred

and the timestamp assigned to that event by psychopy.iohub and psychopy.event check out this discussion:

https://groups.google.com/d/msg/psychopy-dev/u3WyDfnIYBo/ZlzOSOCwusEJ

The important points from this are:

1. 'Normal' keyboards have built a delay (probably because of the keyboards debounce logic) that can be very large and also variable. Without using extra testing hardware, it is not possible to know what the delay for a given keyboard event is. So using normal keyboards as input devices for RT calculation is going to result in poor event timestamping accuracy no matter what software is used.
2. The difference in accuracy between psychopy.iohub kb event vs. psychopy.event based timestamps depends on the experiment design and coding implementation:
1. In experiments where psychopy is constantly calling win.flip(), the iohub keyboard event times are more accurate than psychopy.event.getKeys() times. The increase in accuracy can be up to 1 video retrace interval. 
2. In experiments where you make one display change and then wait until the user responses, so psychopy is not constantly calling win.flip(), there is no difference in accuracy between the two kb event detection methods. 

Thanks