How can we synchronize Pyphotometry recordings with a camera feed?

[félix Leroy]

Hi,

I wish to sync the recordings done with Pyphotometry with a USB camera imaging the animal's behavior.

How could we do so?

best regards,

félix

[Thomas Akam]

Hi Félix,

The easiest way would probably be to generate sync pulses that are sent both to a digital input on the pyPhotometry board and to the camera.  If you are using a camera like a Point Grey which has digital inputs you could send the sync pulse electrically to the camera, if you are using a camera that does not have digital inputs you can send the sync pulse to an LED that is in the field of view of the camera, so that the sync pulses are recorded on the video itself.  You could generate the sync pulses e.g. using an arduino or a micropython/pyControl board.

You will then have the times of the sync pulses recorded on both the photometry system and the video, and can convert the time of any behavioural event of interest into the time reference frame of the photometry data by linearly interpolating between the pulse times on either side of the behavioural event.  E.g. if you know from the video that the behavioural event of interest occured 25% of the way between sync pulse 4 and 5, you can then use the times of sync pulse 4 and 5 recorded by the photometry system to work out the time of the behavioural event in the time reference frame of the photometry data.

For this approach to work it is critical that you know which sync pulse in the photometry data corresponds to which sync pulse in the video.  When we use sync pulses to align behavioural and physiology data we use sync pulses with random inter-pulse intervals (e.g 200 - 2000ms) which ensures that there is a completely unambiguous match between the sequence of inter-pulse-intervals recorded on each system, so you can always work out which pulse coresponds to which even if some pulses are missing, e.g. due to forgetting to turn one system on till after the start of the session.

pyControl has functionality both for automatically generating sync pulse trains with random inter pulse intervals and analysis code for transforming times from one systems reference frame to annother using the sync pulse times.  This is currently not documented but I will try and write some documentation for it soon.

best,

Thomas

[Yangfan Peng]

Hi Thomas,
sorry for bringing up this old thread. I have a question regarding camera syncing with pycontrol: I currently have a digital_out pulse that triggers each frame of the camera at 100fps (trigger on falling edge):

cam_trigger = Digital_output(pin = board.port_1.DIO_A,inverted = True)
v.cam_freq(100)

...

cam_trigger.pulse(v.cam_freq)

Is this a safe method to keep the frames in sync with the timepoints of the pycontrol? I was unsure whether the first frame corresponds to milisecond 0 or milisecond 10 of the pycontrol time. Or is this not a safe method to keep camera and behaviour in sync and I need to introduce a LED in the field of view? Thanks for your help!

Best

Yangfan

[thoma...@neuro.fchampalimaud.org]

Hi Yangfan,

I don’t think that is a very safe method of synchronising a camera with pyControl behavioural data because pyControl does not record the times of the pulses generated by Digital_output.pulse(), so if they did drift relative to the pyControl clock for any reason you would not know about it. To trigger a camera with regularly timed pulses I suggest doing it using a timer as shown below, using the output_event argument to ensure the timer events get written to the data file.  That way you have the time of each pulse recorded in the pyControl data file using the same clock as all the other behavioural data.

best,

Thomas

# In hardware definition.

cam_trigger = Digital_output(pin = board.port_1.DIO_A,inverted = True)

# In task file.
def run_start():
    # Set timer to trigger first pulse.
    set_timer('camera_pulse', 1*ms, output_event=True) 

def all_states(event):
    if event == 'camera_pulse':
        # Turn on camera pulse and set timers to turn off current pulse and trigger the next one.
        cam_trigger.on()
        set_timer('camera_pulse', 10*ms, output_event=True)
        set_timer('camera_pulse_off', 5*ms)
    elif event == 'camera_pulse_off': 
        cam_trigger.off()

​

[Yangfan Peng]

Thanks so much for the rapid reply! I will give this a try.