Accessibility integration with new single-page app navigation API

[Domenic Denicola]

Hi Chrome Accessibility folks,

Nate and I are working to implement the app history API, which is a new way for single-page apps to deal with history and navigation.

One of the most consistent pieces of feedback we've gotten from web developers is that they'd like this new API to have first-class accessibility integration. In particular, they want to be able to convert a cross-document navigation into a same-document navigation, but have the browser announce to assistive technology as if it were a cross-document navigation. You can see more background on this by following the links from this issue.

For today's single-page apps, which use history.pushState(), there is no such AT integration. But we believe the app history API has the right affordances to make it possible: it has no backward-compatibility constraints, and it has appropriate hooks for the web developer to signal both the start and the end of the navigation.

We'd love any thoughts from you all, both on whether this sounds like a good idea, and if so, how we could get started on implementing it.

[Dominic Mazzoni]

Thanks for reaching out! At a high level it definitely sounds like a great idea.

I see some discussion of placing the user's keyboard focus.

For a while I've wanted a JavaScript API that kind of does the equivalent of following a same-page link or calling SetSequentialFocusNavigationStartingPoint, without affecting the url fragment. Right now if a developer wants to move accessibility focus to a certain part of the page after a in-page navigation, their only choices are to either update the fragment (which is not robust and has other undesirable side effects) or to actually call focus(), which leads to adding tabindex on something like a heading where it doesn't really belong.

I could see this as something that could be solved orthogonally, or something integrated into app history, where each navigation would also have a target "starting element". Navigating back could also default to returning focus to the element that initiated the navigation, if any.

What's nice about these ideas is that for the most part the underlying platform APIs already exist, so many of the ideas we might come up with could be directly implemented by user agents without requiring assistive technology to also buy in to a new API. We should be able to prototype these ideas quickly.

- Dominic

--
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-accessib...@chromium.org.

[Domenic Denicola]

Yeah, we're definitely interested in the focus problem as well. What's tricky there is that there are conflicting opinions on what is best: some developers we've talked to said that they prefer the cross-document behavior, where focus just resets to the body element, whereas others (backed by research) indicate that something more app-specific would be best. Maybe by default the focus should be reset to the body, but there should be an easy way for a SPA router to make a more intelligent decision? This intelligent override could either be done using app history somehow, or with a new API (like setSequentialFocusNavigationStartingPoint) that people use together with app history.

The point about navigating back restoring focus is a very interesting one; it'd be great to solve this as well.

Do you have any pointers on where to start prototyping the navigation-announcement part? E.g., where it is done for cross-document navigations? We were thinking that, since that has fewer outstanding questions, we could start there.

[Dominic Mazzoni]

On Fri, Mar 5, 2021 at 11:12 AM Domenic Denicola <dom...@chromium.org> wrote:

Do you have any pointers on where to start prototyping the navigation-announcement part? E.g., where it is done for cross-document navigations? We were thinking that, since that has fewer outstanding questions, we could start there.

Most platforms have a "load complete" notification that we send to assistive technology. While you'll find some code in Blink accessibility that fires this event, it's a little misleading because it's actually just a hint. In practice, what actually happens is that every update sends the value of AXObject::IsLoaded(), and the state transition (from false to true), or the first notification on a document where the value is true, is what triggers the event. So practically we'd either need to toggle it from false to true or introduce some new mechanism to trigger that event.

Focus events are similar. Even though there's code to fire a focus event from Blink, it won't work if it doesn't match what AXObjectCacheImpl::FocusedObject() returns. If you added some code to override that and then fire a focus event that should work.

Finally, there's the kScrolledToAnchor event, which is used for same-page links. That one is very simple to call and on all platforms except macOS it should "just work", last I checked. On macOS, VoiceOver seems to ignore it unless it comes in immediate response to the user clicking on a same-page link, unfortunately. We could explore coming up with a different hack to make macOS work.

Hope that helps!

[Fran007 Cortes]

--

[Fran007 Cortes]

[Domenic Denicola]

Hi all,

I'm returning to trying to implement this and got a bit stuck. You can see my progress and some more background in this document.

I'd welcome any general help; in particular I think it's likely an expert in accessibility code who is familiar with the Blink/a11y boundary would be able to fix this 10x quicker than I could. But in terms of tactical questions, I'm wondering:

• What code actually tells AT about the new page navigation? Is it this code? I.e., is my goal to somehow end up calling AddEvent(tree->root(), Event::LOAD_COMPLETE), and then everything will work? Or is it going to be more complicated than that?
• What does Dominic mean above when he says that the code in Blink accessibility that fires the load-complete event is a "hint"? I wrote a CL that tried to use that infrastructure and it ended up doing nothing, from what I could see. Where in the code does that event get handled, or ignored?

- Dominic

 

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-accessibility+unsub...@chromium.org.

[Daniel Libby]

On Tuesday, January 11, 2022 at 12:12:45 PM UTC-10 dom...@chromium.org wrote:

Hi all,

I'm returning to trying to implement this and got a bit stuck. You can see my progress and some more background in this document.

I'd welcome any general help; in particular I think it's likely an expert in accessibility code who is familiar with the Blink/a11y boundary would be able to fix this 10x quicker than I could. But in terms of tactical questions, I'm wondering:

• What code actually tells AT about the new page navigation? Is it this code? I.e., is my goal to somehow end up calling AddEvent(tree->root(), Event::LOAD_COMPLETE), and then everything will work? Or is it going to be more complicated than that?

Yes that is the code. AXTreeData::loaded is populated by this code in the renderer process:

https://source.chromium.org/chromium/chromium/src/+/main:content/renderer/accessibility/blink_ax_tree_source.cc;drc=1f97794f93e8a29186ed208d22027a43608446cb;l=315 

In theory this should just work if the state transitions are done correctly on that object.

• What does Dominic mean above when he says that the code in Blink accessibility that fires the load-complete event is a "hint"? I wrote a CL that tried to use that infrastructure and it ended up doing nothing, from what I could see. Where in the code does that event get handled, or ignored?

This is related to the previous point - the kLoadEvent that is sent directly from Blink (i.e. not derived from the AXTreeData state) is not mapped to platform accessibility API events currently (though it is read in other places as a 'hint', AFAICT). 

[Domenic Denicola]

On Tue, Jan 11, 2022 at 5:29 PM 'Daniel Libby' via Chromium Accessibility <chromium-ac...@chromium.org> wrote:

On Tuesday, January 11, 2022 at 12:12:45 PM UTC-10 dom...@chromium.org wrote:

Hi all,

I'm returning to trying to implement this and got a bit stuck. You can see my progress and some more background in this document.

I'd welcome any general help; in particular I think it's likely an expert in accessibility code who is familiar with the Blink/a11y boundary would be able to fix this 10x quicker than I could. But in terms of tactical questions, I'm wondering:

• What code actually tells AT about the new page navigation? Is it this code? I.e., is my goal to somehow end up calling AddEvent(tree->root(), Event::LOAD_COMPLETE), and then everything will work? Or is it going to be more complicated than that?

Yes that is the code. AXTreeData::loaded is populated by this code in the renderer process:

https://source.chromium.org/chromium/chromium/src/+/main:content/renderer/accessibility/blink_ax_tree_source.cc;drc=1f97794f93e8a29186ed208d22027a43608446cb;l=315 

In theory this should just work if the state transitions are done correctly on that object.

I'm not sure how to do that in this case. My understanding is in the cross-document case, a new tree gets created with loaded false initially, then set to true later.

But in the same-document navigation case, it doesn't make sense to set loaded to false, and then just set it back to true later within the same document... does it?

 

To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/chromium-accessibility/wDUwToU8LdM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to chromium-accessib...@chromium.org.

[Domenic Denicola]

On Wed, Jan 12, 2022 at 12:24 PM Aaron Leventhal <aleve...@google.com> wrote:

NVDA doesn't currently listen to load complete events. We've recently talked to them about changing that (because of bfcache), and this could provide another reason.

Other ATs may listen to load complete events, however. It may make sense to identify one and get this working with that first.

Oh, that is good to know! I guess NVDA isn't the best testing platform for this then...

On the other hand, it seems like we should be able to get the cross-document experience in NVDA somehow anyway: i.e., have it read the new title and contents of the page as-if we performed a cross-document navigation. Maybe this ties into your below point?

 

Maybe what we should do is create a new event, e.g. kLoadCompleteInSamePage, and then finding something to map that to on each platform (or inventing a new API event for each), and then working with each AT to support it.

I think this makes sense, although I'm not sure I understand the full idea. So in Blink we send this new event to ui/accessibility code, and then we do something like:

• Do the same thing currently done for kLoadComplete for Android and Aura Linux (ChromeOS?)
• Try to do something intelligent on other platforms, e.g. maybe read the title and contents in NVDA somehow, like I mentioned above?
• Work with platforms and ATs to do something less hacky by sending them a proper event, in the longer term?

Aaron

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-accessib...@chromium.org.