Adding text events in plugin for GUI v1.0.0

[Lowell Thompson]

Hello,

I am updating a custom plugin that our lab created for communicating with an older computer (no ZMQ libraries) to work with GUI v1.0.0. This plugin worked fine with v0.6.7. For the most part this was straight forward and most components are functioning fine. However, I have some issues and questions regarding the most appropriate method to add text events.

We used to add text events like so:
 TextEventPtr textEvent = TextEvent::createTextEvent(getMessageChannel(),
                                                                                sampleNum,
                                                                                messageText);
addEvent(textEvent, 0);

This method produces negative/wrong timestamps when analyzing offline (saved as nwb). Instead, it seems to work fine using a timestamp instead of a sample number as the second argument (I think I got this idea from the messageCenter source code). This timestamp method works when the timestamp is based on CoreServices::getSystemTime(). However, when I try to add a text event that is added at the same timestamp as a received ttl event, I run into a new issue. Using event-->getTimestampInSeconds() always returns -1. Okay, so even if that doesn't work I can just calculate it manually similar to the latency histogram plugin: event-->getSampleNumber() / stream-->getSampleRate() * 1000.0f. These functions return reasonable values. However, all these "ttl-locked" text events still have negative numbers offline, unlike those that I added using a system time. Note that the timestamps in both cases print to the console with reasonable non-negative numbers.

So the core questions are:

1. Should I be creating the text event using a sample number or a timestamp as the second argument? When using addEvent, should I use 0 as the second input argument or a sample number? Using a sample number as the second argument results in no text events being saved.
2. Why would using the system timestamp work fine, but not the calculated timestamps? Any ideas on how to resolve saving these ttl-locked text messages correctly?
3. Why does event-->getTimestampInSeconds() produce a value of -1 for ttl events, but the other event-related functions work fine?
   

Any help is appreciated and I'm happy to share links to the updated plugin code if necessary.

Thanks,
Lowell

[Anjal Doshi]

Hi Lowell,

We've re-architected TEXT events in GUI v1.0 to no longer be stream-specific. The goal of TEXT events in the Open Ephys GUI is to allow you to send custom messages with an associated software timestamp, which helps you annotate your data with "events" that occur during your experiments (e.g., a mouse entering a maze).

With v1.0, we introduced the Message Window functionality. When you open this window, it saves and uses the time the window was opened, rather than the time the "send" button is pressed. Thus, the window can be launched when an interesting event occurs, and the user can take as much time as needed to enter custom text describing the event. This means the GUI can now send TEXT events that have occurred in the past while data acquisition is active. To support this, we updated the TEXT events API to use system time in milliseconds when creating messages, rather than relying on sample numbers. This allows the Record Node to use system time information, convert it to a sample number relative to the main sync stream set in the Record Node, and then convert that relative sample number to the timestamp that is ultimately saved.

I agree that the `TextEvent::createTextEvent` method asking for a sample number is misleading; it seems we forgot to update the method signature to request system time in milliseconds.

To answer your questions:

1. Should I be creating the text event using a sample number or a timestamp as the second argument? When using addEvent, should I use 0 as the second input argument or a sample number? Using a sample number as the second argument results in no text events being saved.

> Use system time as the second argument when creating the text event. When using `addEvent`, the sample number should be 0 if you want to add it to the start of the buffer. Alternatively, you can use an actual sample number that falls within the block size of the buffer. If the sample number is outside that range, the text events will not be saved.

2. Why would using the system timestamp work fine, but not the calculated timestamps? Any ideas on how to resolve saving these ttl-locked text messages correctly?

> The system timestamp works because the RecordNode expects the text event time to be based on system time. If you calculate the timestamp based on the TTL event's sample number, the RecordNode's time offset calculation becomes incorrect, leading to an incorrect saved timestamp. We recommend getting the TTL event's sample number (for TTL-locked text messages) and the system time when the TTL event was received. Then, use the system time as the second argument when creating the text event, and include the sample number in the text event's message. This way, you can reconstruct the events offline using the sample number from the message and calculate the exact timestamp based on the TTL event's timestamps.

3. Why does `event->getTimestampInSeconds()` produce a value of -1 for TTL events, but the other event-related functions work fine?

> This is because the TTL event's timestamp in seconds is calculated and set by the RecordNode, depending on whether there are multiple streams that need to be synchronized online. If your plugin is not downstream of a record node, that information is never set.

Please let us know if anything is unclear or if you require further assistance.

Best,

Anjal

[Lowell Thompson]

Thank you for all your clarifications! Made the appropriate changes and it seems to be working well, just needed to set the Acquisition Board to use a TTL sync timestamp rather than the hardware timestamp in the Record Node as well.

Lowell

[Anjal Doshi]

Hi Lowell,

We've re-architected TEXT events in GUI v1.0 to no longer be stream-specific. The goal of TEXT events in the Open Ephys GUI is to allow you to send custom messages with an associated software timestamp, which helps you annotate your data with "events" that occur during your experiments (e.g., a mouse entering a maze).

With v1.0, we introduced the Message Window functionality. When you open this window, it saves and uses the time the window was opened, rather than the time the "send" button is pressed. Thus, the window can be launched when an interesting event occurs, and the user can take as much time as needed to enter custom text describing the event. This means the GUI can now send TEXT events that have occurred in the past while data acquisition is active. To support this, we updated the TEXT events API to use system time in milliseconds when creating messages, rather than relying on sample numbers. This allows the Record Node to use system time information, convert it to a sample number relative to the main sync stream set in the Record Node, and then convert that relative sample number to the timestamp that is ultimately saved.

I agree that the `TextEvent::createTextEvent` method asking for a sample number is misleading; it seems we forgot to update the method signature to request system time in milliseconds.

To answer your questions:

1. Should I be creating the text event using a sample number or a timestamp as the second argument? When using addEvent, should I use 0 as the second input argument or a sample number? Using a sample number as the second argument results in no text events being saved.

> Use system time as the second argument when creating the text event. When using `addEvent`, the sample number should be 0 if you want to add it to the start of the buffer. Alternatively, you can use an actual sample number that falls within the block size of the buffer. If the sample number is outside that range, the text events will not be saved.

2. Why would using the system timestamp work fine, but not the calculated timestamps? Any ideas on how to resolve saving these ttl-locked text messages correctly?

> The system timestamp works because the RecordNode expects the text event time to be based on system time. If you calculate the timestamp based on the TTL event's sample number, the RecordNode's time offset calculation becomes incorrect, leading to an incorrect saved timestamp. We recommend getting the TTL event's sample number (for TTL-locked text messages) and the system time when the TTL event was received. Then, use the system time as the second argument when creating the text event, and include the sample number in the text event's message. This way, you can reconstruct the events offline using the sample number from the message and calculate the exact timestamp based on the TTL event's timestamps.

3. Why does `event->getTimestampInSeconds()` produce a value of -1 for TTL events, but the other event-related functions work fine?

> This is because the TTL event's timestamp in seconds is calculated and set by the RecordNode, depending on whether there are multiple streams that need to be synchronized online. If your plugin is not downstream of a record node, that information is never set.

Please let us know if anything is unclear or if you require further assistance.

Best,

Anjal

--
You received this message because you are subscribed to the Google Groups "Open Ephys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-ephys+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/open-ephys/159d08bd-01cc-4728-93ea-86627d756c06n%40googlegroups.com.