WebAccessibilityAPI

[Dominic Mazzoni]

I'm starting a new thread for this. Here are just a few initial questions about:

https://wiki.mozilla.org/Accessibility/WebAccessibilityAPI

Overall this looks very comprehensive and well-designed. I didn't realize that you were working on an API simultaneously for exposing the accessibility tree, and also for allowing authors to extend it. Neat idea.

Here are my first questions, in no particular order.

Why do you have an AccessibleSource object to be filled in, rather than allowing authors to extend AccessibleElement directly? Is the main issue that AccessibleSource can provide attributes like name, description, value but it can't override the tree-walking interfaces?

What do you want to do about aria-owns? I've heard that you're considering having the browser implement aria-owns internally by rearranging the accessibility tree exposed to AT rather than forcing each AT to keep track of that relation. If so, when walking the AccessibleElement tree would you get the rearranged tree?

Along those lines, why do appendChildren and insertBefore return a new element? Why not allow the author to reparent an existing element somewhere else? That would essentially be an alternative way of implementing aria-owns.

Have you thought about extending this to support multiple frames? If this is really supposed to represent the same interface that AT sees, then it'd be great if it could represent the full composed accessibility tree that spans multiple frames.

What do you want to do to represent the huge number of roles? The more common ones have an ARIA role and it'd make sense to use the same name where it exists, but there are lots of HTML elements that map to a unique native role. Would every one of those have to be part of this spec? Part of the challenge I see there is that the set of roles on each native platform is so different - I'm afraid that the list of roles necessary would be the superset of roles needed by all platforms, which will only grow.

Finally, what's the use case for allowing the author to extend the taxonomy? While menuitemcheckbox makes sense, most permutations of existing roles don't make much sense.

[Alexander Surkov]

On Wed, Mar 18, 2015 at 2:36 AM, 'Dominic Mazzoni' via Browser Accessibility Development <browser-acce...@googlegroups.com> wrote:

I'm starting a new thread for this. Here are just a few initial questions about:

https://wiki.mozilla.org/Accessibility/WebAccessibilityAPI

Overall this looks very comprehensive and well-designed. I didn't realize that you were working on an API simultaneously for exposing the accessibility tree, and also for allowing authors to extend it. Neat idea.

thank you, it's great to hear

 

Here are my first questions, in no particular order.

Why do you have an AccessibleSource object to be filled in, rather than allowing authors to extend AccessibleElement directly? Is the main issue that AccessibleSource can provide attributes like name, description, value but it can't override the tree-walking interfaces?

I think yes. If we didn't want to create a11y trees in JS then we could cut off 'readonly' part from attributes of AccessibleElement interface.

But even keeping in mind JS trees I'm not confident about AccessibleSource and I think it's felt a bit like hypostatization issue but there are reason behind the approach. AccessibleSource is semantically different than AccessibleElement since AccessibleSource is *description* of AccessibleElement, and thus it doesn't have to look exactly same as AccessibleElement. For example, if AccessibleElement has 'states' attribute which is an ValueSet object then AccessibleSource may have 'state' attribute which is an array of states for web author convince. Another example is AccessibleSource may skip 'name' property if 'labelledby' relation was provided, AccessibleElement has to return both name and labelledby relation. Another reason is technical one, afaik WebIDL doesn't allow to combine interface and callback interface into one entity, so if it's something implemented in c++ then it cannot be implemented in js. Not sure if that was necessary part to answer your question :)

 

What do you want to do about aria-owns? I've heard that you're considering having the browser implement aria-owns internally by rearranging the accessibility tree exposed to AT rather than forcing each AT to keep track of that relation. If so, when walking the AccessibleElement tree would you get the rearranged tree?

Right, I tend to think if aria-owns altered the accessible tree then it would be easier to implement aria-owns, both on browser and AT side since you don't have to care about stuff like group position, relations, eventing etc when aria-owns used. AccessibleElement is supposed to reflect accessible tree hierarchy, so walking AccessibleElement hierarchy should make you walk the rearranged tree I think.

 

Along those lines, why do appendChildren and insertBefore return a new element? Why not allow the author to reparent an existing element somewhere else? That would essentially be an alternative way of implementing aria-owns.

These methods take accessible source and return new accessible element for them. Do you think to make them accepting AccessibleElements or DOM nodes additionally what would lead to reparenting? It sounds flexible. Alternatively, we could have aria-owns approach by specifying nodes in relations part.

 

Have you thought about extending this to support multiple frames? If this is really supposed to represent the same interface that AT sees, then it'd be great if it could represent the full composed accessibility tree that spans multiple frames.

Originally I supposed we should have seamless tree per tab until you said about process per frame initiatives. Maybe we should split trees not running process boundaries but I didn't think of it much. Do you have ideas on it?

 

What do you want to do to represent the huge number of roles? The more common ones have an ARIA role and it'd make sense to use the same name where it exists, but there are lots of HTML elements that map to a unique native role. Would every one of those have to be part of this spec? Part of the challenge I see there is that the set of roles on each native platform is so different - I'm afraid that the list of roles necessary would be the superset of roles needed by all platforms, which will only grow.

I'd say that other specifications should describe their accessible vocabularies, for example, HTML should provide its mappings. I wish to end up with something platform neutral, I agree that ARIA roles is a good base for it. If we can't organize them as superset of platform roles then the solution is less valuable for automated testing but it's still ok for web AT. Having taxonomies we can have many roles as we like.

 

Finally, what's the use case for allowing the author to extend the taxonomy? While menuitemcheckbox makes sense, most permutations of existing roles don't make much sense.

I wanted to have API that can be used to describe different kinds of content and would provide AT a hint what new content is about. Say, you want to describe a music sheet, and you add bunch of roles and some taxonomy for them, for example, you claim that role="note" is inherited from a button role and thus you can play the note by pressing it. It should be better than nothing. Also it should be nice for backward compatibility, for example, you can say that new 'switch' role can be described as 'checkbox' for those who doesn't know what 'switch' is. On the other hand it has some tech sugar, if AT wants to handle certain set of roles same way, it can check role taxonomy instead of listing all possible roles. For example, they can do a11ydoc.taxonOf("role", aRole).is("menuitem") to figure out if given role has a menuitem in its base and then run a common procedure for menuitems.

 

--
You received this message because you are subscribed to the Google Groups "Browser Accessibility Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to browser-accessibil...@googlegroups.com.
To post to this group, send email to browser-acce...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/browser-accessibility-dev/CAFz-FYyh2Pi-XgYuKkzjMLgbo4awKb2uY4eDPxjiPtBvP9AL5w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Dominic Mazzoni]

To move forward with this idea, what would you think of splitting it into phases? We'd try to keep the big picture in mind but we wouldn't need to reach consensus on every last detail before making some progress on some of the earlier parts.

For example we could split it up like this:

Phase 1: Reading the accessibility tree from within an HTML document

Phase 2: Extending the semantics of a single DOM element

Phase 3: Building an accessible tree

Phase 4: (Privileged) accessing the accessibility tree for a whole tab or more

Phase 5: Virtual cursor and content traversal

Phase 6: Custom taxonomies

Or we could do it in a very different order, focusing on getting to accessing the accessibility tree for a whole tab sooner and extending semantics later.

I guess the question is which use-case do you think would be the best one to start with? Extending the semantics of the DOM, or accessing the accessibility tree for a whole tab? My proposal would be to pick the first use-case and then focus only on the interfaces for that use-case.

More below:

 

Why do you have an AccessibleSource object to be filled in, rather than allowing authors to extend AccessibleElement directly? Is the main issue that AccessibleSource can provide attributes like name, description, value but it can't override the tree-walking interfaces?

I think yes. If we didn't want to create a11y trees in JS then we could cut off 'readonly' part from attributes of AccessibleElement interface.

But even keeping in mind JS trees I'm not confident about AccessibleSource and I think it's felt a bit like hypostatization issue but there are reason behind the approach. AccessibleSource is semantically different than AccessibleElement since AccessibleSource is *description* of AccessibleElement, and thus it doesn't have to look exactly same as AccessibleElement. For example, if AccessibleElement has 'states' attribute which is an ValueSet object then AccessibleSource may have 'state' attribute which is an array of states for web author convince. Another example is AccessibleSource may skip 'name' property if 'labelledby' relation was provided, AccessibleElement has to return both name and labelledby relation. Another reason is technical one, afaik WebIDL doesn't allow to combine interface and callback interface into one entity, so if it's something implemented in c++ then it cannot be implemented in js. Not sure if that was necessary part to answer your question :)

Yes, this basically makes sense. In Chromium code we have a similar split: AXNode and AXNodeData. Anything can provide an AXNodeData, and AXNode wraps an AXNodeData and also includes the tree relationships.

I wonder if a similar naming convention might work: what about AccessibleElementData, or something along those lines? I think the name should convey that this acts as a simple object containing the underlying data for one element (though of course it can defer computation lazily).

What do you want to do about aria-owns? I've heard that you're considering having the browser implement aria-owns internally by rearranging the accessibility tree exposed to AT rather than forcing each AT to keep track of that relation. If so, when walking the AccessibleElement tree would you get the rearranged tree?

Right, I tend to think if aria-owns altered the accessible tree then it would be easier to implement aria-owns, both on browser and AT side since you don't have to care about stuff like group position, relations, eventing etc when aria-owns used. AccessibleElement is supposed to reflect accessible tree hierarchy, so walking AccessibleElement hierarchy should make you walk the rearranged tree I think.

Perfect, I like that.

Along those lines, why do appendChildren and insertBefore return a new element? Why not allow the author to reparent an existing element somewhere else? That would essentially be an alternative way of implementing aria-owns.

These methods take accessible source and return new accessible element for them. Do you think to make them accepting AccessibleElements or DOM nodes additionally what would lead to reparenting? It sounds flexible. Alternatively, we could have aria-owns approach by specifying nodes in relations part.

No, I'd like to stay away from using relations to modify the tree. I think they should accept AccessibleElements and that would be a way for an author to quickly reparent an existing node instead of using aria-owns.

Have you thought about extending this to support multiple frames? If this is really supposed to represent the same interface that AT sees, then it'd be great if it could represent the full composed accessibility tree that spans multiple frames.

Originally I supposed we should have seamless tree per tab until you said about process per frame initiatives. Maybe we should split trees not running process boundaries but I didn't think of it much. Do you have ideas on it?

I think part of the issue is that there are multiple uses for this API. For the purpose of extending the accessibility of HTML and building custom accessibility trees, it doesn't seem necessary to go beyond one HTML document.

However, for the purpose of accessing the accessibility tree for testing or for building new AT, it should be possible to see the whole tree. In fact for the automation API on Chrome OS we're going one step forward and exposing one tree for the whole desktop including all native UI.

[Alexander Surkov]

I agree the implementation will likely be done in parts. As long as we agreed on key parts then I'm good to polish docs in parts too. The way you put items into parts looks good as well, I would probably bump up content traversal since it includes text stuff which is needed for tree building in general.

Data might be too static name. If JS is used to create accessible elements then they may need to implement interactions like invoking accessible actions.

I think I agree that changing accessible hierarchy by means of relations is rather ARIA markup restriction than nice concept. But I like the idea of describing the children in declarative way (likewise we could do that with relations, that's why I was thinking of them). So I'm good as long as we can do something like

<div id="div1"></div>

<div id="div2"></div>

var acc = {

  children: [ "div2" ]

};

document.getElementById("div1").accessibleSource = acc;

to avoid extra JS calls of appendChild, etc.