Thoughts on implementing autorepeat

[Filip Filmar]

Let's activate ui-inp...@fuchsia.dev

tl;dr: how to refactor the input pipeline for key auto-repeat support

Hi folks.

I've been thinking as to how to implement the auto-repeat functionality for the input pipeline's keyboard. This is a quick note to jot down my thoughts, and also invite you to comment on the approach.

As you may expect, key autorepeat generates repeated identical key events while the user actuates and holds a specific key. Typically, the key autorepeat is engaged after the user keeps a key actuated for some time ("auto-repeat delay"), while the key events are then generated periodically if the key continues to be actuated ("auto-repeat rate").

A possible implementation approach would be to kick off an auto-repeat generator task following each key press, which would then periodically inject key events (key down, wait, key up) based on the autorepeat settings.

This I think leads to a need to modify how the input pipeline stages are connected together. The rest of this note is only about that part of the work.   The actual auto-repeat generation is treated as a black box, and is out of scope of this note. Similarly, the auto-repeat settings storage and retrieval is out of scope.

The current InputHandler allows a single event to be unpacked into 0..n other events, but does not allow delayed event generation:

// Extra comments removed. #[async_trait] pub trait InputHandler {     async fn handle_input_event(         &mut self,         input_event: input_device::InputEvent,     ) -> Vec<input_device::InputEvent>; }

The events are currently processed in a loop like so, repeatedly feeding the generated events into the pipeline until they are fully processed:

pub async fn handle_input_events(mut self) {         while let Some(input_event) = self.input_event_receiver.next().await {             let mut result_events: Vec<input_device::InputEvent> = vec![input_event];             for input_handler in &mut self.input_handlers {                 let mut next_result_events: Vec<input_device::InputEvent> = vec![];                 for event in result_events {                     next_result_events.append(&mut input_handler.handle_input_event(event).await);                 }                 result_events = next_result_events;             }         }     }

Note that this does not allow injecting events that did not originate from the driver.  So even though the InputHandler API is declaratively serial, the events are consumed by synchronously awaiting on each stage. This leaves no place to inject events that are sourced from within the input pipeline, which auto-repeat needs to work.

To allow for auto-repeat, I was thinking that the individual stages of the pipeline should instead be coupled through a producer-consumer queue like async_channel.  This would allow the autorepeat stage to generate events and "float" them downstream just as usual.  The latter pipeline stages would be none the wiser.

However, since not all stages need this specific approach, I would keep the InputHandler interface around.  Instead of refactoring InputHandler, I was considering an adapter around InputHandler that instantiates a producer-consumer queue and is applied at the time the input pipeline is instantiated. This means any "regular" pipeline stages do not need to care about the details of inter-stage communication.

The only exception to this rule for the time being would be autorepeat, which would need direct access to the producer-consumer queue in order to inject generated events.

F

[Alice Neels]

First, let's make sure we're building it in the right place. Does autorepeat want to live fully inside the input pipeline? Or should it be something that happens in IME service (or some other service) after the input pipeline has "sent" the event off? Do autorepeat events actually propagate through the pipeline or are they something we generate before dispatching to UI frameworks? Would other input pipeline stages be interested in autorepeat events? I imagine we really don't want autorepeat to trigger new keyboard shortcuts, for example.

-A

--
All posts must follow the Fuchsia Code of Conduct https://fuchsia.dev/fuchsia-src/CODE_OF_CONDUCT or may be removed.
---
You received this message because you are subscribed to the Google Groups "ui-input-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ui-input-dev...@fuchsia.dev.
To view this discussion on the web visit https://groups.google.com/a/fuchsia.dev/d/msgid/ui-input-dev/CAGEh6bhVy7_QCRpv_EVTKfx2embO2UtiPfykNbu3wN5XAgs1Jw%40mail.gmail.com.

[Filip Filmar]

On Mon, Aug 16, 2021 at 2:56 PM Alice Neels <nee...@google.com> wrote:

First, let's make sure we're building it in the right place. Does autorepeat want to live fully inside the input pipeline? Or should it be something that happens in IME service (or some other service) after the input pipeline has "sent" the event off? Do autorepeat events actually propagate through the pipeline or are they something we generate before dispatching to UI frameworks? Would other input pipeline stages be interested in autorepeat events? I imagine we really don't want autorepeat to trigger new keyboard shortcuts, for example.

All good points!  Definitely should be addressed in a design document.  

I think one guiding principle could be consistency, as a way to handle questions such as these.  To wit, I think we probably want similar behavior of autorepeat across all uses, and that means a single set of related settings (delay and rate).  We also have the user expectations from experience with existing systems.  And I'd also like to check if our PMs have specific expectations.

HTH,
F