Intent to Implement: Generic Sensor & Ambient Light Sensor APIs

[Bhaumik, Rijubrata]

Contact emails

rijubrat...@intel.com, to...@sensors.codespeaks.com

Specs

Generic Sensor API: https://w3c.github.io/sensors/

Ambient Light Sensor API: https://w3c.github.io/ambient-light/

Tag Review

Generic Sensor API: https://github.com/w3ctag/spec-reviews/issues/110

Ambient Light Sensor API: https://github.com/w3ctag/spec-reviews/issues/115

Summary

The Generic Sensor API defines a framework for exposing MEMS sensors to the Open Web Platform in a consistent way. It does so by defining a blueprint for writing specifications of concrete sensors along with an abstract Sensor interface that can be extended to accommodate different sensor types (such as a gyroscope, magnetometer, or an altimeter). The AmbientLightSensor is the first concrete implementation of a sensor in terms of the Generic Sensor API. More will follow.

Motivation

Currently, not all MEMS sensors available on a device are exposed to web applications while those are readily available to native apps. This incentivizes developers to pick native solutions where web applications would have been a better choice.

In the cases where such sensors are exposed to the web (for example through the DeviceOrientation or DeviceLightEvent APIs), the APIs are inconsistent, don’t meet key use-cases, and are generally too high-level (and thus in opposition to the spirit of the Extensible Web Manifesto).

This new API addresses these concerns, those brought up by W3C TAG in the Sensor section of the Extensible Web Report Card and issues specific to the DeviceLightEvent such as not providing a value until a change causes the devicelight event to fire.

Overall, this API lowers developers’ cognitive load by offering a consistent API across sensors, makes it simpler and faster to specify and implement new sensors (such as barometers/altimeters which are now appearing on high-end devices) and opens up a number of uses cases by providing lower-level access to the sensors.

Interoperability and Compatibility Risk

Firefox: shipped DeviceLightEvent. Positive public signals for moving to Ambient Light Sensor.

Edge: has implemented DeviceLightEvent. Positive public signals for moving to Ambient Light Sensor.

Safari: No negative signals.

Ongoing technical constraints

None.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

OWP launch tracking bug

crbug.com/606766

Link to entry on the feature dashboard

https://www.chromestatus.com/features/5298357018820608

Requesting approval to ship?

No.

[Kostiainen, Anssi]

[+cc folks who have indicated interest]

Hi,

Just in case someone is wondering what do MEMS (Micro Electro Mechanical Systems) sensors mean:

Basically, in this context, MEMS is just a catch-all term for various small sensors that are embedded in all modern mobile devices, phones, tablets, laptops etc. This includes accelerometers, gyroscopes, digital compasses, ambient light sensors, etc.

The Generic Sensor API defines a framework (think an abstract base class) that is extended by such concrete sensors, e.g. AmbientLightSensor to begin with. This means the API surface across various sensors will be more consistent.

The feedback from TAG review seem to signal the feature is good for implementation.

Thanks,

-Anssi

On 26 Apr 2016, at 17:23, Bhaumik, Rijubrata <rijubrat...@intel.com<mailto:rijubrat...@intel.com>> wrote:


Contact emails
rijubrat...@intel.com<mailto:rijubrat...@intel.com>, to...@sensors.codespeaks.com<mailto:to...@sensors.codespeaks.com>

Specs
Generic Sensor API: https://w3c.github.io/sensors/
Ambient Light Sensor API: https://w3c.github.io/ambient-light/

Tag Review
Generic Sensor API: https://github.com/w3ctag/spec-reviews/issues/110
Ambient Light Sensor API: https://github.com/w3ctag/spec-reviews/issues/115

Summary
The Generic Sensor API defines a framework for exposing MEMS sensors to the Open Web Platform in a consistent way. It does so by defining a blueprint for writing specifications of concrete sensors along with an abstract Sensor interface that can be extended to accommodate different sensor types (such as a gyroscope, magnetometer, or an altimeter). The AmbientLightSensor is the first concrete implementation of a sensor in terms of the Generic Sensor API. More will follow.

Motivation
Currently, not all MEMS sensors available on a device are exposed to web applications while those are readily available to native apps. This incentivizes developers to pick native solutions where web applications would have been a better choice.
In the cases where such sensors are exposed to the web (for example through the DeviceOrientation or DeviceLightEvent APIs), the APIs are inconsistent, don’t meet key use-cases, and are generally too high-level (and thus in opposition to the spirit of the Extensible Web Manifesto).

This new API addresses these concerns, those brought up by W3C TAG in the Sensor section of the Extensible Web Report Card<https://extensiblewebreportcard.org/#toc-28> and issues specific to the DeviceLightEvent such as not providing a value until a change causes the devicelight event to fire<https://bugzilla.mozilla.org/show_bug.cgi?id=1057185>.

Overall, this API lowers developers’ cognitive load by offering a consistent API across sensors, makes it simpler and faster to specify and implement new sensors (such as barometers/altimeters which are now appearing on high-end devices) and opens up a number of uses cases by providing lower-level access to the sensors.

Interoperability and Compatibility Risk
Firefox: shipped DeviceLightEvent. Positive public signals for moving to Ambient Light Sensor.
Edge: has implemented DeviceLightEvent. Positive public signals for moving to Ambient Light Sensor.
Safari: No negative signals.

Ongoing technical constraints
None.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Yes.

OWP launch tracking bug

crbug.com/606766<https://bugs.chromium.org/p/chromium/issues/detail?id=606766>

[Alex Russell]

Hey all,

Your local TAG member here! We've taken a first read here: https://github.com/w3ctag/spec-reviews/issues/115#issuecomment-203822542

We've discussed a few additional points at a recent all.:

• It isn't clear how to subclass this interface yet. I think this is important and I'm excited and optimistic that once it's sorted out, making new sensor types (both from the user and system perspective) will be a powerful way to add new capabilities.
• There's an open question about delivering sensor readings to workers. Avoiding main thread jank, in general, means doing less work there. Making it possible to ask for permission to use a sensor from the document context but handle sensor readings in workers seems like a good way to start.
• Are there explainer documents for the Ambient Light (or Generic Sensors) APIs? Seeing example code might make it easier for everyone to evaluate the proposals.

Overall the direction looks great and I'm excited that this is coming to the platform.

Regards

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Alexander]

Hi,

Based on feedback we’ve received from initial review for the patches that Riju sent and constraints that were identified by Alex Russel and Tim Volodine (e.g., subclassing, addition of new sensor types, managing different sensor configurations, etc), we’ve created design for Generic Sensor API and implemented prototype that is based on it.

The proposed design provides:

1. Sharing code between the concrete sensor implementations (for example: adding a new sensor on Android requires ~30 loc on platform side and ~150 loc on blink side). This reduces the code duplication and also simplifies maintenance of the newly added code.

2. Support for multiple JS Sensor instances that can have different configurations and life-time.

3. Support for both “slow” sensors that provide periodic updates (e.g. AmbientLight, Proximity), and “fast” streaming sensors that have low-latency requirements for sensor reading updates (Gyroscope, LinearAcceleration).

4. Shared memory based IPC mechanism for sensor reading update. It is beneficial for performance-sensitive streaming sensors (Gyroscope, LinearAcceleration).

Please find class and sequence diagrams for the intended design in the attachment.

Comments and questions are welcome.

Contact emails:

alexander...@intel.com

mikhail.p...@intel.com

Best regards,

Alexander Shalamov

[Reilly Grant]

The abstraction looks good but I'm concerned that the use of shared memory here is a bit of premature optimization. Sensor readings are going to be very small messages so the overhead of signaling negates the benefit of writing the data to shared memory.

[Alexander]

Thank you for looking Reilly.

We used shared memory to reduce IPC latency for sensors like Gyroscope, LinearAcceleration, and took after the approach that is already used (https://chromium.googlesource.com/chromium/chromium/+/refs/heads/trunk/content/browser/device_sensors/device_inertial_sensor_service.cc#82).

However, we agree that shared memory IPC increases implementation complexity and latency is not critical for AmbientLightSensor which is planned to be implemented first.
We can base our implementation on mojo message pipe IPC for simplicity, yet will keep design flexible, so that we can introduce shared memory IPC for senors that might require it.

[Rick Byers]

Seems strange that avoiding IPC would be necessary for eg. Gryroscope but not for touch events (both have low-latency 60fps requirements).  For input we've found IPC latency not to be a bottleneck, though we do need the blink scheduler to treat the input IPC messages as "high priority" to avoid blocking on lower priority tasks (thread contention, especially on the blink thread, is the real latency problem in general).  That said, if there's convincing data that avoiding IPC entirely can yield lower latency I'm sure the input team would be very interested in that.  +tdresser.

[Michael van Ouwerkerk]

See also the Intent to implement and ship Device Motion API, where shared memory was introduced.

/m

[Reilly Grant]

I took a look at the mechanism used for device orientation, which is simply reading from the shared memory buffer at 60Hz. This certainly seems reasonable for some sensor types (though if we haven't yet we should look at synchronizing this timer with the rendering pipeline so that the event is delivered at a useful time). For sensors that update less frequently this won't be appropriate so we shouldn't bake a preference for one method or the other into the generic infrastructure. Ideally we would have something that adaptively scales the sensor read rate based on how much the value is actually changing.

DataPipe is probably not what you want to use in this instance now that I understand the goal as it uses normal IPC messages to signal changes to the data. The benefit of the existing shared memory implementation is that it avoids IPC entirely at the expense of a high-frequency timer in the renderer.

[Mikhail Pozdnyakov]

Hi Reilly, others,

Our way of shared memory usage for streaming sensors is basically the same as the one used for device orientation: clients simply read from shared buffer at frequency each of them is given (and for discrete sensors client read from shared buffer on the update signal).

The data source is a single instance within browser process (listening to a platform sensor) it just writes sensor data to the shared buffer immediately after it gets it, and it is not affected by increasing number of clients.

On the other hand using of IPC messages for sensor reading updates would bring drawbacks as the data source would have to send message to each client (frame) separately.

Besides sending IPC message brings additional latency comparing to writing to/reading from shared buffer.

We measured latency between the moment a single sensor data update is received from the platform and the moment this data becomes available on blink side.
This includes both mojo data handling and IPC data transmission itself. The average latency is equal to ~0.3ms for shared memory mechanism, and ~1ms for IPC messages (Chromium build on Nexus 7).

BR,
Mikhail

[Alexander]

Hi all,

We are planning to start upstreaming Generic Sensor API enablers early next week, if anyone wants to check prototype code that we've mentioned before, WIP CL is https://crrev.com/2051083002 .

BTW, we've noticed that device_sensors (https://crrev.com/2037513002/) is being modified to use mojo IPC. There might be some naming / folder conflicts, however, we think it would be easy to solve.

Best regards,
Alexander & Mikhail