Intent to Prototype: Confirmation of Action API

[Sara Tang]

Contact emails

sar...@microsoft.com

Explainer

https://github.com/WICG/aom/blob/gh-pages/notification-api.md

Specification

Summary

This effort aims to create a JavaScript API so that developers can better notify AT users of actions/changes to a webpage not necessarily tied to UI elements.

Blink component

Blink>Accessibility

Motivation

Currently the only mechanism available today that communicates content changes in a web app down to the accessibility layer is via ARIA live regions. One major limitation to ARIA live regions is that they assume the change to a webpage is tied to a DOM element. This leads to content authors employing various inefficient or inconsistent tricks and hacks to notify of changes that are not associated with the DOM. We propose a separate notification API to address these scenarios, called Confirmation of Action.

Initial public proposal

https://github.com/WICG/aom/blob/gh-pages/notification-api.md

TAG review

TAG review status

Pending

Risks

Interoperability and Compatibility

Gecko: No signal

WebKit: No signal

Web developers: Positive (https://github.com/w3c/aria/issues/832)

Other signals:

Debuggability

TBD

Is this feature fully tested by web-platform-tests?

No

Flag name

--enable-blink-features=ConfirmationOfAction

Requires code in //chrome?

False

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1291098

Estimated milestones

No milestones specified

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5745430754230272

This intent message was generated by Chrome Platform Status.

[Sara Tang]

+Daniel Libby

---

From: Sara Tang
Sent: Friday, January 28, 2022 4:26 PM
To: blin...@chromium.org <blin...@chromium.org>
Subject: Intent to Prototype: Confirmation of Action API

 

[Yoav Weiss]

On Sat, Jan 29, 2022 at 1:27 AM 'Sara Tang' via blink-dev <blin...@chromium.org> wrote:

+Daniel Libby

---

From: Sara Tang
Sent: Friday, January 28, 2022 4:26 PM
To: blin...@chromium.org <blin...@chromium.org>
Subject: Intent to Prototype: Confirmation of Action API

 

Contact emails

sar...@microsoft.com

Explainer

https://github.com/WICG/aom/blob/gh-pages/notification-api.md

Specification

Summary

This effort aims to create a JavaScript API so that developers can better notify AT users of actions/changes to a webpage not necessarily tied to UI elements.

Blink component

Blink>Accessibility

Motivation

Currently the only mechanism available today that communicates content changes in a web app down to the accessibility layer is via ARIA live regions. One major limitation to ARIA live regions is that they assume the change to a webpage is tied to a DOM element. This leads to content authors employing various inefficient or inconsistent tricks and hacks to notify of changes that are not associated with the DOM. We propose a separate notification API to address these scenarios, called Confirmation of Action.

Initial public proposal

https://github.com/WICG/aom/blob/gh-pages/notification-api.md

TAG review

 

Just wanted to note that it seems worthwhile to file for an early TAG review. 

TAG review status

Pending

Risks

Interoperability and Compatibility

Gecko: No signal

WebKit: No signal

Web developers: Positive (https://github.com/w3c/aria/issues/832)

Other signals:

Debuggability

TBD

Is this feature fully tested by web-platform-tests?

No

Flag name

--enable-blink-features=ConfirmationOfAction

Requires code in //chrome?

False

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1291098

Estimated milestones

No milestones specified

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5745430754230272

This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CH2PR00MB06809C5589E8FD6848CF5E09F2239%40CH2PR00MB0680.namprd00.prod.outlook.com.

[Sara Tang]

Good suggestion Yaov! I've opened one here: Review request for Confirmation of Action API · Issue #713 · w3ctag/design-reviews (github.com)

---

From: Yoav Weiss <yoav...@chromium.org>
Sent: Monday, January 31, 2022 6:33 AM
To: Sara Tang <Sara...@microsoft.com>
Cc: blin...@chromium.org <blin...@chromium.org>; Daniel Libby <dli...@microsoft.com>
Subject: [EXTERNAL] Re: [blink-dev] Re: Intent to Prototype: Confirmation of Action API

 

[Roberto Clapis]

Hi All,

During a discussion about this proposal a few concerns were raised:

• What pipeline of data would be used to pass the new messages to a potential screen-reader? Would screen-readers need to implement a new API or would this use pre-existing ones?
• Does this new API allow pages to have a more direct or a less restricted way to pass data to a screen reader?
• Would this API allow potential attackers to use different character sets or might this allow them to pass potentially malformed data to screen readers that was not possible to pass before?
• If a pre-existing channel is used to communicate with the screen reader (e.g. already existing APIs) how would a user distinguish this new mechanism from content on the page?

Thanks in advance,

Rob

[Roberto Clapis]

There is one additional question that was brought forward during the discussion:

• What information can be read by the users of this API? This is mentioned in the security concerns but it doesn't seem to be specified elsewhere. Is this just about learning of an existence of a AT or is this some additional info?

[Sara Tang]

Hi Roberto, thanks for your feedback 🙂 Responses inline:

---

From: Roberto Clapis <cl...@google.com>
Sent: Tuesday, February 8, 2022 9:05 AM
To: blink-dev <blin...@chromium.org>
Cc: Roberto Clapis <cl...@google.com>; Sara Tang <Sara...@microsoft.com>; blin...@chromium.org <blin...@chromium.org>; Daniel Libby <dli...@microsoft.com>; yoav...@chromium.org <yoav...@chromium.org>
Subject: Re: [EXTERNAL] Re: [blink-dev] Re: Intent to Prototype: Confirmation of Action API

 

There is one additional question that was brought forward during the discussion:

• What information can be read by the users of this API? This is mentioned in the security concerns but it doesn't seem to be specified elsewhere. Is this just about learning of an existence of a AT or is this some additional info?

• Here are some security concerns and possible mitigations. These are also re-iterated in the "Privacy and Security Considerations" section of the proposal:
• - readback: Do not readback AT configuration settings. Doing so makes the user an easier target for fingerprinting.
• - authoritative-sounding notifications: announcements can be crafted to deceive the user. We should suppress notifications when focus moves outside of the web content.
• - Maybe only offer this feature to Secure Contexts (instead of 3^rd party browsing contexts)

On Tuesday, February 8, 2022 at 11:06:43 AM UTC+1 Roberto Clapis wrote:

Hi All,

During a discussion about this proposal a few concerns were raised:

• What pipeline of data would be used to pass the new messages to a potential screen-reader? Would screen-readers need to implement a new API or would this use pre-existing ones?

• A small nuance: screen-readers do not implement APIs, they consume ones that are exported by the Web Platform.
• - In the case of Windows systems, we use the UIA notifications API to pass information along to screen-readers.
• - In the case for other systems, we can hijack the existing ARIA live regions implemenation. In the case where the confirmation of action API is called without a DOM element/ARIA node, we can attach the announcement to an internal "root" node instead.

• Does this new API allow pages to have a more direct or a less restricted way to pass data to a screen reader?

• Less restrictive; possible restrictions we'll need to employ are listed in the next response.

• Would this API allow potential attackers to use different character sets or might this allow them to pass potentially malformed data to screen readers that was not possible to pass before?

• Here are some possible mitigations we have for this scenario:
• - Truncating strings, employing a max queue length
• - Restricting to alphanumeric input.
• - Running the announcement-text through a HTML-parser/DOM-parser/setInnerHtml or similar JS API

• If a pre-existing channel is used to communicate with the screen reader (e.g. already existing APIs) how would a user distinguish this new mechanism from content on the page?

• I don't think it's necessary for the user to distinguish between different screen-announcing APIs. Is there a particular scenario you are thinking of where a distinction would be needed? 

Thanks in advance,

Rob

[Roberto Clapis]

> - readback: Do not readback AT configuration settings.

This would be ideal. What would the major downsides of this be?

> - authoritative-sounding notifications

Do we currently (before this proposal) inform the user on which page/origin is causing a certain ARIA node to be read?

> We should suppress notifications when focus moves outside of the web content.

+1 on that

> Maybe only offer this feature to Secure Contexts (instead of 3rd party browsing contexts)

+1 on this too

Thanks for the clarification on the APIs. If I understand correctly this proposal won't change the API we use or the way we communicate with it, but it might end up opening a more direct channel to use it.
My proposal would be to then validate messages encoding (UTF-wise) and limit them in length and character set. Alphanumeric and punctuation should be sufficient for the use case we're trying to address, right?

I don't think running the message through parsers would be beneficial, so we can skip that part.

[David Tseng]

Hi there,

A few additional questions/comments:
- this api will be a separate channel from ordinary live regions. If we mix the two techniques, wouldn't the output be interleave? For example, an iframe with a live region, and another iframe wherein we call document.ariaNotify?
- the way Chrome (at least) works is based on the serialization of a tree structure. This api appears to be a imperitive call, making it seem like the call gets honored immediately. However, the a11y tree gets pushed only on layout clean.
- what happens if I repeatedly call document.ariaNotify? Would these announcements queue up? Would they flush tts? Would it trigger a refresh of the accessibility tree?
- live regions have the aria-live polite|assertive properties which roughly (though not really for all screen readers) map to queue vs flushing tts. Is there something equivalent for this api?
- many screen readers actually suppress live regions (e.g. NVDA) or give users a way to turn them off. This api might reduce the ability for screen readers to do this depending on how it gets implemented.
- at least via the current proposed change, it looks like notify is trying hard to be a direct tts channel to the screen reader. It has very little to do with ARIA?

[Sara Tang]

Hi David,

Responses inline 🙂. I've also linked the design doc if you'd like to take a further look: https://docs.google.com/document/d/1pz7KQUgd3zjjH7aTWFTFBEps_GrxXF3pZjaV8H76WjA/edit?usp=sharing

Cheers,

Sara

---

From: David Tseng <dts...@google.com>
Sent: Monday, March 7, 2022 4:44 PM
To: blink-dev <blin...@chromium.org>
Cc: Sara Tang <Sara...@microsoft.com>
Subject: [EXTERNAL] Re: Intent to Prototype: Confirmation of Action API

 

Hi there,

A few additional questions/comments:

- this api will be a separate channel from ordinary live regions. If we mix the two techniques, wouldn't the output be interleave? For example, an iframe with a live region, and another iframe wherein we call document.ariaNotify?

In the Windows implementation, we are indeed using a different channel than live regions. However, the final invocation out of Chromium for both this API and live regions is into UIA, which has its own queuing mechanism. While it is possible for notifications to interleave, they will never interrupt each other (unless specifically intended)

The plan for other systems is to hook our API into the existing ARIA live region machinery. It will have a similar outcome as above, where ariaNotify notifications may interleave with live ARIA region ones, but they won't interrupt or interfere with each other.

- the way Chrome (at least) works is based on the serialization of a tree structure. This api appears to be a imperitive call, making it seem like the call gets honored immediately. However, the a11y tree gets pushed only on layout clean.

- what happens if I repeatedly call document.ariaNotify? Would these announcements queue up? Would they flush tts? Would it trigger a refresh of the accessibility tree?

This is a good point! We mark the a11y tree as dirty whenever document.ariaNotify is called and store unqueued notifications on AXTreeData.

- live regions have the aria-live polite|assertive properties which roughly (though not really for all screen readers) map to queue vs flushing tts. Is there something equivalent for this api?

Yes! One of the goals for this API was to have more indicative properties to describe how notification behave. Currently, we have any combination of the following two properties (naming tbd):

• PlaceInQueue: (front | back | newest-only)
• front: place the notification in the front of the queue
• back: place the notification in the back of the queue
• newest-only: place the notification in the back of the queue, but also drop all existing notifications from the same element (good for progress bars, latest status, etc.)
• Interrupt: (true | false)
• true: if a notification is currently speaking and shares the same source as the one being queued, interrupt it and move to the next notification in the queue
• false: let the current notification speak to completion

- many screen readers actually suppress live regions (e.g. NVDA) or give users a way to turn them off. This api might reduce the ability for screen readers to do this depending on how it gets implemented.

Another good point! I think it makes sense that if live regions are suppressed, we would want this API to also be suppressed.

On non-Windows systems, since we will be hooking directly into the existing ARIA live regions implementation, we will share the same behavior as ARIA live regions (aka. suppress live regions => suppress JS API).

I'm not so sure in the Windows implementation, I'll do more research on how live regions are suppressed in screen readers.

- at least via the current proposed change, it looks like notify is trying hard to be a direct tts channel to the screen reader. It has very little to do with ARIA?

That is correct, but we decided to keep the ARIA-naming because many web authors associate accessibility with ARIA. Thus, this naming most succinctly expresses the intention of the API.

[Sara Tang]

+ Cynthia Shelly

---

From: Sara Tang <Sara...@microsoft.com>
Sent: Wednesday, March 9, 2022 3:32 PM
To: David Tseng <dts...@google.com>; blink-dev <blin...@chromium.org>
Subject: Re: [EXTERNAL] Re: Intent to Prototype: Confirmation of Action API

 

[David Tseng]

Thanks for the responses here and sorry for the late reply.

IIUC, this looks mostly like syntactic sugar. The imperitiveness of the api is somewhat of an illusion since this takes the same code path as ordinary live regions and for most platforms and even most clients on Windows, it re-uses the same mechanics of delivering the announcement as a live region.

So, we are basically reduced to the two lines:
someElement.textContent = 'foo';

vs

document.ariaNotify('foo');

(assuming someElement is a live region).

The key diff is some number of queueing modes to reorder the announcements for the current callstack and the aforementioned syntax cleanup. The other diff is not having to add a DOM element which is somewhat negligible if it is positioned offscreen.

Is that a fair assessment of the difference? If so, could we consider introducing additions to live regions to get the queueing behaviors introduced?

[Sara Tang]

Hi all!

Letting everyone know that we are picking up this initiative again :) Please find an updated explainer here:  MSEdgeExplainers/Accessibility/AriaNotify/explainer.md at main · MicrosoftEdge/MSEdgeExplainers (github.com). I've already updated the ChromeStatus.

There is also an updated design doc in the works, but still waiting on reviews from third parties.

Thanks!

Sara

[Ethan Jimenez]

Updating Chrome status entry and subject of this thread.

--Ethan