Label guessing code

[Aaron Leventhal]

We have a request to provide assistive technology
vendors with a guess as to what might be labelling
a control, when the <label for> or aaa:labelledby
markup is not being used to explicitly set a label
in HTML. This would help prevent the need for
every AT to implement this separately. It may also
be more performant for us to provide this.

We've already been reasonably successful now and
providing a guess as to whether a table is for
layout or data, and the community helped with that.

Are there any thoughts on a good heuristic, given
an HTML form control with no label, to guess where
its label is? We have to keep different languages
in mind.

- Aaron

[Aaron Leventhal]

Also, as with our other guessing code, we will
indicate to the AT that this is in fact a guess.

- Aaron

[Lukas Loehrer]

Aaron Leventhal writes ("Label guessing code"):

> We have a request to provide assistive technology
> vendors with a guess as to what might be labelling
> a control, when the <label for> or aaa:labelledby
> markup is not being used to explicitly set a label
> in HTML. This would help prevent the need for
> every AT to implement this separately. It may also
> be more performant for us to provide this.

I can only speak for left-to-right languages.

I was planning to use nsiAccessible.getAccessibleToRight() and
friends for this purpose once these functions are implemented.
Especially for radiobuttons and checkboxes, getting the thing visually
to the right should work fine for many cases.

For forms that are outlined with tables, looking in the previous cell
for a label might often work, however if the previous cell contains a form
element too, chances are the label is in the cell above the original form
element.

It would indeed be very useful if Firefox could provide this
information to the AT.

I guess there is a similar problem with finding table headings if
no TH element is used.

Best regards, Lukas

[Gijs Kruitbosch]

So do we want to implement this separately from the thing (I sadly
forgot its name) Alex Faaborg mentioned at CSUN? Because it seemed to me
that they're already doing it, and Alex said it'd be released as Open
Source "soon" (hopefully)? I'm guessing it would be worthwhile to build
on what they have, or at least figure out what algorithm they're using
(which I'm hoping/guessing they won't mind telling you about). CC-ing
Alex so we can get a bit more substantial info on that.

~ Gijs

[Aaron Leventhal]

Good point Gijs -- they had some label guessing
code in JS. Although we'll be in C++, we could use
some of the ideas.

I wonder, what are all the evil things authors do
to position labels?

Table cells, but what else?

Maybe it's not that complicated, and we can always
improve it as we evolve FF3.

- Aaron

[Al Gilman]

At 7:05 PM +0200 9 04 2007, Lukas Loehrer wrote:
>Aaron Leventhal writes ("Label guessing code"):

>> We have a request to provide assistive technology
>> vendors with a guess as to what might be labelling
>> a control, when the <label for> or aaa:labelledby
>> markup is not being used to explicitly set a label
>> in HTML. This would help prevent the need for
>> every AT to implement this separately. It may also
>> be more performant for us to provide this.
>

>I can only speak for left-to-right languages.

If you deal with the outer-loop direction (nextLine from nowLine) and
the inner-loop direction (nextChar from nowChar) you will be in
pretty good shape (although there are BIDI situations where there is
a locally-scoped variation in the inner-loop direction). By
locally-scoped, I mean to a textrange that is smaller than a line of
wrapped text.

>I was planning to use nsiAccessible.getAccessibleToRight() and
>friends for this purpose once these functions are implemented.
>Especially for radiobuttons and checkboxes, getting the thing visually
>to the right should work fine for many cases.

These are cases where the actual control is compact and it makes
sense to align them vertically within groups.

On the other hand, won't you more often find the best label for
a text box in the preceding text, not subsequent text?

Of course the common case is the search form where the text field for
search terms is unlabeled and the submit button may say GO and the
group of two controls is identified as SEARCH with presentational
markup that positions it in the right place.

WASP style should be a good clue to heuristics. Might as well reward
what is usable.

http://www.webstandards.org/learn/tutorials/accessible-forms/

>For forms that are outlined with tables, looking in the previous cell
>for a label might often work, however if the previous cell contains a form
>element too, chances are the label is in the cell above the original form
>element.

Is there anyone on the list who has played with natural language
processing in this regard? For naive forms, the strongest clue that
I can think of as to what a field is about is the field name used in
reporting results.

If you had an NLP tool that would estimate the meaning relationship
between the field name and the sundry text chunks near the control,
that should give you a good leg up on binding the likeliest one as
"label -- guessed".

>It would indeed be very useful if Firefox could provide this
>information to the AT.
>
>I guess there is a similar problem with finding table headings if
>no TH element is used.

s/no TH/no TH or HEADERS

IIRC the best current practice is to use the first row and column as
the guessed headers. Even if the were not planned by the author
that way, they become effective nicknames for the rows and columns.

Al

>
>Best regards, Lukas
>_______________________________________________
>dev-accessibility mailing list
>dev-acce...@lists.mozilla.org
>https://lists.mozilla.org/listinfo/dev-accessibility

[David Bolter]

Hi Aaron,

When would this browser-side logic/heuristic execute? I'm guessing
anytime a page with a form control, or with an element with a wai role
loads? What about when the dom is manipulated? Or would it perhaps not
fire until an related accessible api call is made?

cheers,
David

[Aaron Leventhal]

Here's the bug for this, where I've been
discussing that:
https://bugzilla.mozilla.org/show_bug.cgi?id=376481

Right now my plan is to only calculate it when the
labelled-by relation is asked for, on a form
control that should have a label.
When the labelled-by is a guess I will expose some
kind of object attribute like:
"labelled-by-guess" = "true"

That way I don't have to deal with DOM mutations
and caching the answers. It also means we won't do
a lot of extra work for the label-for relation, on
every kind of element.

I think it should be good enough. It doesn't give
label-for for the element we're guessing, but
that's not nearly as important, AFAICT.

- Aaron

[David Bolter]

That sounds like a good plan: to do the guess when asked for a
labelled-by. A label-for could come later perhaps. I like that the
guess-ness is communicated as it means that run-time validation tools
can still report missing label-for relations. Note I'll probably post
future comments in the bug. Thanks for the link.

[Aaron Leventhal]

Yes I like when we expose that guesses are
guesses. That's what we're doing for layout vs.
data table.

Another item to do that for, which would be nice
to have, is a guess for missing ALT text. We had
some code that's been rotting, which was very good
at that. It used either the title or a heading
from a page linked to, or combination of the image
URL, the ancestor link URL (if that exists).

We could expose that as an object attribute
instead of just changing the image's text.
Something like "text-equiv-guess" = "[Mozilla's
best guess]"

[Willie Walker]

I don't know of a way in the AT-SPI to specifically request a labeled-by
relation. Instead, you ask an accessible for its relation set and then
iterate through the relations until you find the one(s) you're looking
for. I'm not sure if this will have an impact on the planned
implementation in FF or not.

Will

PS - Thanks for looking into it! This is great.

David Bolter wrote:
> That sounds like a good plan: to do the guess when asked for a
> labelled-by. A label-for could come later perhaps. I like that the
> guess-ness is communicated as it means that run-time validation tools
> can still report missing label-for relations. Note I'll probably post
> future comments in the bug. Thanks for the link.
>

[Aaron Leventhal]

We might need to look at how performance is with Firefox relations at
some point.

Sometimes it needs to do a fair amount of work for "inverse" relations
-- we have to search a significant part of the DOM tree, just to find
out that something *doesn't* have something pointing to it.

There is probably a more performant way of caching stuff, but it would
be very complex and not something we should tackle for Firefox 3.

- Aaron

Willie Walker wrote:
> I don't know of a way in the AT-SPI to specifically request a
> labeled-by relation. Instead, you ask an accessible for its relation
> set and then iterate through the relations until you find the one(s)
> you're looking for. I'm not sure if this will have an impact on the
> planned implementation in FF or not.
>
> Will
>
> PS - Thanks for looking into it! This is great.
>
> David Bolter wrote:
>> That sounds like a good plan: to do the guess when asked for a
>> labelled-by. A label-for could come later perhaps. I like that the
>> guess-ness is communicated as it means that run-time validation tools
>> can still report missing label-for relations. Note I'll probably post
>> future comments in the bug. Thanks for the link.
>>

[Jesse Pelton]

Is the idea that ALT-text guessing would respect empty ALT text? I'd
make the case that if the page author has specified alt="", no guessing
should occur; the intent is plain. Eye-candy images that reinforce
adjacent text are fairly common, and such images often have no
appropriate replacement text that wouldn't duplicate the adjacent text
in the page.

[Aaron Leventhal]

No, you're right. It's not for alt="", because
that means there was no ALT text on purpose (a
decorative image). It's only when ALT was not
specified.

- Aaron