Intent to Implement: WebHID (Human Interface Device)

[Matt Reynolds]

Contact emails

mattre...@chromium.org, rei...@chromium.org

Explainer

WebHID Explainer

Summary

A HID (Human Interface Device) is a type of device that takes input from or provides output to humans. It also refers to the HID protocol, a standard for bi-directional communication between a host and a device that is designed to simplify the installation procedure. Mice, keyboards, touchscreens, gamepads, and many other devices are typically implemented using HID. Our WebHID proposal would provide low-level access to these devices so that web applications can access them even when they are not supported well by the host OS.

Motivation

Many popular and useful devices use HID. For some devices, generic HID drivers provided by the operating system can support all important use cases and no additional device-specific support is needed. However, some devices may provide additional capabilities outside of the features supported by the generic driver. There are many devices that use HID but do not fall into the categories of devices that are supported by generic drivers.

Risks

Interoperability and Compatibility

We haven't received feedback yet, but HID support is not provided in any other browsers and is unlikely to be adopted. Chrome already has support for HID through the chrome.hid API exclusive to Chrome Apps, and this API aims to bring that functionality to the web. Some types of HID devices are already widely supported on the web platform through pointer, keyboard, and gamepad APIs.

Edge: No signals

Firefox: No signals

Safari: No signals

Web developers: No signals

Ergonomics

With WebHID we are trying to address issues with the ergonomics of the chrome.hid API. The chrome.hid API similarly allows access to HID subsystem devices but treats HID reports as opaque binary blobs. We aim to improve on this by providing utility methods to get and set the values of individual fields within the report.

Activation

I expect that this API will typically be used through a library or polyfill that encapsulates device-specific behavior to provide developers a cleaner interface. These libraries will not be available at first, which may slow adoption.

Debuggability

Chrome already provides some debugging support for HID through the chrome://device-log dashboard.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Link to entry on the feature dashboard

https://www.chromestatus.com/feature/5172464636133376

Requesting approval to ship?

No.

[Philip Jägenstedt]

Hi Matt,

This looks cool, and as usual no LGTM required here to implement. Do you have a sense for how this will interact with Web Bluetooth and USB? The explainer mentions that HID is a protocol for both Bluetooth and USB. Does this mean that the same device might show up in multiple APIs, and is this a good thing or a problem?

I presume that this has already been answered for USB cameras and microphones that might show up in both Web USB and navigator.mediaDevices.enumerateDevices, just curious.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2Bqnouc%3DESV8Spawf8m1OkuWWadXMyM2e8igoHHrcEFUvvx%2B6A%40mail.gmail.com.

[Matt Reynolds]

Hi Philip,

I've thought about how we might expose HID functionality through WebUSB and Web Bluetooth, but for the initial version of the API it will be simpler to keep them separate. I'm not sure how permissions should be handled when the page has permission to access the device through USB but not through HID, for instance.

It is likely that some devices will be accessible through both WebHID and either WebUSB or Web Bluetooth (or even Gamepad). I think this is a good thing because there isn't always a one-to-one mapping between a WebUSB device and a WebHID device, even when they refer to the same physical device. For instance, a single USB device may expose multiple HID devices.

It's also not required that a HID device be a USB or Bluetooth device, although these are by far the most common. HID can be implemented over many protocols including PCI and I2C. We use the OS's APIs to enumerate HID so these devices would be made accessible over WebHID even if they were not accessible through other APIs.

- Matt

[Gary Kačmarčík (Кошмарчик)]

I assume that the `oninputreport` event is non-cancelable - otherwise the interaction with the cancelable keyboard events would be... complicated.

My general concern is there needs to be more details about how these events might affect the existing input events from these devices. My hope is that the answer is "No impact at all", but there needs to be a section exploring that. Please let me know if you have any questions.

For example, using this API would it be possible to send an event to a keyboard (or other HID device) that would trigger the keyboard to generate a trusted keyboard event?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BqnoufS6pmq3sOJmu9xSPm7V4YvP_RzvjvtpgzNp5B-2GR81A%40mail.gmail.com.

[Matt Reynolds]

Hi Gary,

Can you explain more about the interaction with keyboard events?

I expect we will avoid the issue by not allowing keyboards to be exposed through this API as it could allow a tab to observe keystrokes without keyboard focus (i.e., you could turn it into a keylogger). For similar reasons we will also exclude mice and security keys. But even if they were made accessible, I don't expect it would have any impact on existing events.

Some devices offer functionality over HID that could potentially be used to attack the host. For example, it's possible for a keyboard with macro functionality to have its macros reprogrammed over HID, leading to unexpected input the next time the macro is triggered. Some devices receive firmware updates over HID, and could potentially be reprogrammed to do any number of terrible things. To mitigate this, we should exclude any devices that look like keyboards or mice by examining the HID report descriptors during enumeration. For known exploits, we can implement a blocklist that bans access to particular devices or disallows specific reports.

- Matt

[Rick Byers]

Sounds exciting! I'm also interested if we have any customers (publicly nameable or otherwise) who say they need this, and which devices they are most interested in. Presumably anyone who currently consumes chrome.hid would, do we know what the most popular apps that use that API are?

Before I started work on pointer events I always thought it was a huge shame developers had to wait for many years to get access to stylus properties. I think we've fixed that for stylus devices, but it only took us about a decade from the relatively wide adoption of such devices in specific verticals like art and graphic publishing. Hopefully whatever the next most popular unusual input devices are won't have to wait at all ;-)

Rick

On Fri, Sep 28, 2018 at 6:48 AM Philip Jägenstedt <foo...@chromium.org> wrote:

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYfJXBUCUwLfSFeDaTk%3DsbN4aotzkYL009WkEioA%3DhhHXA%40mail.gmail.com.

[Kane York]

On Friday, September 28, 2018 at 11:15:20 AM UTC-7, Matt Reynolds wrote:

Hi Gary,

Can you explain more about the interaction with keyboard events?

I expect we will avoid the issue by not allowing keyboards to be exposed through this API as it could allow a tab to observe keystrokes without keyboard focus (i.e., you could turn it into a keylogger). For similar reasons we will also exclude mice and security keys. But even if they were made accessible, I don't expect it would have any impact on existing events.

Some devices offer functionality over HID that could potentially be used to attack the host. For example, it's possible for a keyboard with macro functionality to have its macros reprogrammed over HID, leading to unexpected input the next time the macro is triggered. Some devices receive firmware updates over HID, and could potentially be reprogrammed to do any number of terrible things.

As an even worse example of this, my recent research on the Nintendo Switch Joy-Cons shows that they are both

1. a device that fits the exact use-case of this API: accessing additional functionality currently unavailable to web content, such as an IR camera / rangefinder, fully controllable NFC radio, and multiple accelerometers; as well as
2. a device that receives firmware updates over HID, exposes several kilobytes of writable flash storage, and has unique identifiers that cannot be reset by clearing cookies in the browser.

 

To mitigate this, we should exclude any devices that look like keyboards or mice by examining the HID report descriptors during enumeration. For known exploits, we can implement a blocklist that bans access to particular devices or disallows specific reports.

I don't think that blocking just keyboards or mice will solve the issue - there's a lot of devices that can be broken or tampered with just by sending raw HID reports.

I'd like to see some device safety guarantees implemented before this ships.

An alternate model, where extensions using the chrome.hid API function as downloadable device drivers that expose functionality to webpages, might not be as scary here - they can filter down to a known-safe API, while still retaining the flexibility to be updated quickly for new hardware features.

[Gary Kačmarčík (Кошмарчик)]

On Fri, Sep 28, 2018 at 11:15 AM Matt Reynolds <mattre...@google.com> wrote:

Hi Gary,

Can you explain more about the interaction with keyboard events?

Some keyboard events are cancelable, which means that they impact follow-on events. As long as these HID events are passive, then there shouldn't be a problem with listening to them.

I'm more concerned about the possibility of using this API to make the device trigger new events, since those would be indistinguishable from native events.

I expect we will avoid the issue by not allowing keyboards to be exposed through this API as it could allow a tab to observe keystrokes without keyboard focus (i.e., you could turn it into a keylogger). For similar reasons we will also exclude mice and security keys. But even if they were made accessible, I don't expect it would have any impact on existing events.

I suspect that not exposing "keyboards" will cause problems since some devices (like remotes) may present themselves as keyboards.  If you limit API access to the currently active foreground tab, then you eliminate background keyloggers.

Some devices offer functionality over HID that could potentially be used to attack the host. For example, it's possible for a keyboard with macro functionality to have its macros reprogrammed over HID, leading to unexpected input the next time the macro is triggered. Some devices receive firmware updates over HID, and could potentially be reprogrammed to do any number of terrible things. To mitigate this, we should exclude any devices that look like keyboards or mice by examining the HID report descriptors during enumeration. For known exploits, we can implement a blocklist that bans access to particular devices or disallows specific reports.

Have you considered breaking up the functionality between (1) actions that are safe to expose to the web and (2) powerful actions that only a trusted Web Platform app should be able to do?  Things like updating the firmware are useful things to do as long as the appropriate permissions/protections are in place.

As Kane mentions, there should also be a section talking about the potential fingerprinting risk.

[Kane York]

Agree on a read/write split; "View raw events from input devices" and "Send raw events to input devices" have massively different security risk profiles (though largely similar privacy risk profiles).

--
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/OaDCpCaEe_4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGnkXoEJZ_1zCwW-9xJsryYmMu0nDr_oLpHB775sQ%2B3JcaahTA%40mail.gmail.com.

[Philip Jägenstedt]

Thanks Matt, sounds like there are separate APIs at the OS level, which makes my question moot.

[Thiemo Nagel]

Joining in with Kane: Please make sure this gets a privacy review.

[aannik...@gmail.com]

Any time frame when this support will be available in Chrome browser ?