Patch: checkable object on the web with firefox

[Peter Vágner]

Hello,
I've finally came up with a little dirty patch attempting at reporting
checkable objects found within the web content while using Mozilla
Firefox. This is the feature I have always wished to have in spiel so
occasionally I was reading on scala learning basic stuff, reading logcat
output identifying various events and how spiel is interpreting them etc.
So far my progres is rather non existant however anyway I'm attaching
this few lines patch to start some discussion hopefully.
This is reporting checked or unchecked state for checkboxes and radio
buttons on the web in firefox. However it does not update the state when
that changes. Double tapping that spot changes the state at least
talkback handles it however I can find no event I should hook into apart
of touch events. No of those touch events do have source attached so I
can grab checkable and checked states.
Nolan can you please give me some hints?
At the end I will most likelly end up factoring this into a seperate
presenter for clarity however now it's really basic and I am missing a
lot of functionality.

Thanks and greetings

Peter

[Nolan Darilek]

Thanks, at first glance looks great! I'll take a closer look later today.

One possible minor issue is that you may be speaking "checked" and "not
checked" before rather than after the content of the text. You may want
to flip R.string.notChecked and utterances in your conses (I'm leaving
out getString for simplicity.) utterances is a List, which Scala expects
to be Nil-terminated. So you'll probably need to replace "... ::
utterances" with:

utterances++getString(R.string.checked)

++ appends a single item to a list, whereas :: concatenates a series of
items but expects the list to end in Nil. Since utterances is already
Nil-terminated, you can't do:

utterances :: getString(...)

Because utterances already looks something like:

"Ok" :: "button" :: Nil

But you could do something like (and I'm making all these up for
simplicity):

"OK" :: "Button" :: "not checked" :: Nil

But since there's already a Nil in this list of utterances, it won't
work because you'd get:

"OK" :: "Button" :: Nil :: "not checked"

which isn't correct. Make sense?

Either way, I'll definitely merge it in a day or two, will fix anything
that needs fixing, and will let you know what I changed. Thanks a bunch
for looking into this!

Can you recommend a page with checkable content that would make for an
easy test?

[Peter Vágner]

Hello,
I am testing with the following page
http://www.tizag.com/htmlT/htmlcheckboxes.php
However I suspect checkboxes on the web is really verry basic thing
which should not break hopefully.

I have tested this and it works into a point where reporting of
checkable and checked state matters. As you've correctly pointed out in
a conversation on twitter event has checkable as well as checked set to
failse and we have to grab these from the source.
What is worying and I am unable to figure it out how do I implement
state changes. Apart from gestures related events I can see no window
change, focus change, clicked or any other events in the logcat output
of spiel when double tapping in sucha view. This is where I do need to
better understand what's going on and I would welcome any kind of hints,
pointers etc.

I am also looking into some other things but still being a total noob
this is all nothing fancy. I will try to post more questions as I
progress slowly.

Greetings

Peter

[Nolan Darilek]

Hmm. Not looking at the code, but off the top of my head, I'd guess that
I have a click event handler on something like android.view.Checkable,
android.widget.Checkable, etc. I'd guess that this presenter handles
speaking state changes for checkable items. It may be that this code has
to migrate to a more generic view, and the isCheckable check should be
moved to the source since the events no longer accurately reflect that.
As I stated on Twitter, whether something is checkable or not used to be
encapsulated in the event itself. Now it seems to be on the source, but
despite events still having an isCheckable method, this often doesn't
match what source.isCheckable reports. So which is right?
event.isCheckable or event.getSource.isCheckable? Nobody knows. :)

The big pain point with Android accessibility is that it evolved like
this over time, and there's no documentation. The Firefox folks did
their implementation by reading TalkBack's source, something I really
don't have the time or desire to do. So it's very much a trial and error
process. Make a little change, find that it works where you wanted but
discover hours later that it broke something unrelated, add a check to
fix the unrelated break, find that broke something else. Repeat until
you've added enough special cases and defensive programming that gets
things behaving like you want.

If they aren't emitting a click event, I'm not sure how to handle state
changes. Maybe ask Marco for pointers?

[Peter Vágner]

Hello,
hmm. I will examine it again to iron out my noobness
But still can you correct this?
I assume I should still be seeing the event in the logcat output
eventhough it might be consumed by the event handler in one of the
generic presenters or is it possible event is consumed too early to be
readable in the logcat output?

Greetings

Peter

[Nolan Darilek]

Events are first displayed, then processed. Whether an event is handled
doesn't determine whether it appears in the logs, it should always.

[Nolan Darilek]

Thanks, you did a good job with this patch. Sorry I used git apply
rather than git am, so your initial commit wasn't preserved, but I gave
you credit in the commit message and will use git am in the future. I
did some minor restructuring:

} else if(utterances != Nil) {
// (only in email, not in code) You're going to have to speak the
utterances either way, instead of placing them on the list, just speak
them here first then do the state check.
speak(utterances)
if(source.isCheckable)
// We've spoken the utterances, now we only need to speak " checked" or
"not checked". Here we speak that but we don't flush speech (the false)
so the utterances spoken above will continue speaking.
if(source.isChecked)
speak(getString(R.string.checked), false)
else
speak(getString(R.string.notChecked), false)
// We return true here to indicate that the event was consumed. The
speak function already does this as a convenience, but since it's in a
conditional it isn't guaranteed to run.
true
} else

Thanks!

[Nolan Darilek]

Also, following up based on a Twitter discussion:

I can confirm that double-tapping checkbuttons on the link Peter
provided does not emit TYPE_VIEW_CLICKED events. This is inconsistent
with other platform checkboxes, and is why their state changes aren't
being reported.

This would be a great way for a non-coder to help. File a bug at
https://bugzilla.mozilla.org, product core, component disability access
APIs, OS Android. Feel free to cut-and-paste the above paragraph if you
like, include the link from previously in this thread as a test, and
follow up here/there if anyone needs to exchange information. I'm about
to shift gears away from patch-wrangling, so it would be a great help
for someone to file this and send the bug link here for reference.

Thanks!

[Peter Vágner]

Hello,
Yeah how nice afternoon we got. I'm really happy.
I have been struggling two evenings until I was able to make such a
simple patch but following that Nolan has turned it into something more
usefull, we had a nice twitter conversation with Marco from Mozilla I do
really feel good at the moment.
Now it's really a perfect opportunity for someone to jump in report that
thingy with tapping on checkboxes within firefox web content not firing
clicked accessibility event and enjoy this moment with us.
Let's pump up some more life into this community.

Greetings

Peter

[Hareth alhamdany]

Hi Peter,
Congragelations for the good stard, and keep up the good work,
thanks abunch,
and of cource big thanks to nolan
hope spiel gets a real boost in the cumming days.
my best.

> --
> You received this message because you are subscribed to the Google Groups
> "Spiel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to spielproject...@googlegroups.com.
> To post to this group, send email to spielp...@googlegroups.com.
> Visit this group at http://groups.google.com/group/spielproject.
> For more options, visit https://groups.google.com/d/optout.
>

[Peter Vágner]

Hello,
Thanks Drew for filling the mozilla bug. It's located at
https://bugzilla.mozilla.org/show_bug.cgi?id=1122857

Greetings

Peter