Improving AXBoundsForRange support

105 views
Skip to first unread message

Євген Рисай

unread,
Aug 4, 2020, 11:15:27 AM8/4/20
to chromium-ac...@chromium.org
Hi all,

I'm working on enabling support for AXBoundsForRange parameterized attribute for text fields --
right now it is explicitly disabled (returning nil) on everything except static text elements.
After some progress I'm facing a consistency issue which I'm not sure how to go about.
The crux of the issue is that related AX attributes are computed assuming different string values.
For example:
- AXValue, AXStringForRange assume that queried string is BrowserAccessibility::GetValue() or
BrowserAccessibility::GetInnerText() if the latter is empty
- implementation of AXBoundsForRange relies on internal ax tree and judging from the name of invoked function assumes the supplied range corresponds to inner text (to be honest it is also not at all obvious that implementation of BrowserAccessibility::GetInnerTextRangeBoundsRect() is consistent with AXNode::GetInnerText() )
- AXRangeForLine seems to combine both assumptions: it uses the length of the string returned by AXValue for upper bound of last line, but relies on line offsets computed from ax tree (which I suppose should correspond to line breaks of inner text string).

These inconsistencies manifest themselves when I'm trying to determine the precise frame of text fragments within a contenteditable field (for example email body in Gmail interface).

An easy solution is to use inner text consistently for these range- and line-based attributes. This approach fails however as inner text does not reflect visual line breaks, so for example in Gmail subsequent paragraphs in the mail body would look as if the user has missed a space between sentences to AX software. 

I think that the way to resolve this issue is to implement an algorithm that produces a string mostly following the steps of GetInnerText(), but also adding newlines using info from the accessibility tree. Then the implementation of attributes such as AXStringForRange, AXLineForIndex, AXRangeForLine, AXBoundsForRange would need to be synced to the aforementioned algorithm to provide a consistent view into the text field's content.

Another possible approach (which I haven't explored, but it seems complex and brittle) is to try synchronizing with renderer --- when we use non-empty "kValue" attribute for the element's AXValue try to trace back from what inline text boxes Blink has built it.


Reply all
Reply to author
Forward
0 new messages