Accessibility on Qubes OS

[Jorge Paez]

Hello everyone.
My name is Jorge.
I’m 22 and studying to be a developer.
I joined this group after hearing about Qubes from an interview done by Leo Laporte on the New Screen Savers.
Anyway, I really like the concept and I want to help with development.
And, one of the things I was thinking of was accessibility.
See, I’m totally blind, so in order to be able to use the operating system, I need screen reading software.
In the case of Windows that’s all 3rd party, but in the case of Mac OS and IOS Apple has actually built in a screenreader into the core of the operating system.
And I thought it would be awesome if we could develop something like that for Qubes OS.
I know it would be pretty complicated because of sandboxing, but it would be really good long term since this is a great open source and free OS that would be useful to a lot of the blind, specially internationally where there isn’t that much access to accessible technology in general, and where most people can’t afford high-end brands such as Apple. It would basically be the perfect combination of secure and accessible.
I’d also be happy to help out with other accessibility related stuff and core development in general, I’m just bringing up this idea of the screenreader since its the first thing that came to mind because its the one barrier for me that stops me from exploring the OS as of now.

Let me know if there is any way I can help, and what could be done in terms of building accessibility into the platform.



Thanks,




Jorge

[Jean-Philippe Ouellet]

On Fri, Dec 23, 2016 at 11:18 AM, Jorge Paez <jp07...@gmail.com> wrote:
> See, I’m totally blind, so in order to be able to use the operating system, I need screen reading software.

> And I thought it would be awesome if we could develop something like that for Qubes OS.

This raises the very interesting question of how one can indicate
which domain's contents are being read in a trustworthy manner.

This is done in a visual way by the colored and labeled window
decorations, combined with trusted dom0 window manager actions to
determine if something is truly a window or just a fake one drawn
within a less-trusted window.

The reason this is secure fundamentally relies on the fact that there
is something a legitimate window can do that a fake window can not:
namely appear as a distinct window in dom0 window management actions.
How this could translate to screen reading is an interesting question.
Would there have to be some "trusted sound" followed by reading the
name of the domain that the following read contents belong to? A sound
which somehow would need to be filtered out by a trusted filter such
that untrusted domains could not produce it? Such in-band signaling
seems fragile and doomed to failure.

The question, then, is what would be an appropriate out-of-band
accessible signaling mechanism to identify (in a trusted manner) which
domain you are interacting with?

[Jean-Philippe Ouellet]

On Fri, Dec 23, 2016 at 7:56 PM, Jean-Philippe Ouellet <j...@vt.edu> wrote:
> The question, then, is what would be an appropriate out-of-band
> accessible signaling mechanism to identify (in a trusted manner) which
> domain you are interacting with?

Or maybe it doesn't need to be entirely out-of-band.

Perhaps a keyboard shortcut which would say the name of the frontmost
VM while also pausing all audio coming from VMs? This would also
require having real working focus-stealing prevention in order to be
meaningful.

[Vít Šesták]

Jean, this is mostly what I wanted to suggest. I'll add few ideas.

1. Only the current VM could make sond output. While this essentially prevents using a background music, I don't think that people that need screenreaders can have background music at all. Well, maybe one could allow some VMs to produce some sound all the time (similar to fullscreen in today's Qubes), maybe with some volume restriction. But this is rather an egde case.
2. There should be some exception for notifications. That is, dom0 would be able to play some predefined sound(s) that inform the user that there is something new. Ideally, this would be a part of centralized notification system. Less ideally, this could be some extension of notification systems present in specific domUs. Discussed at https://github.com/QubesOS/qubes-issues/issues/889 .
3. There should be focus stealing prevention. Less ideally, at least some focus-stealing notification would also help. Focus-stealing is poartially discussed in seemingly unrelated issue https://github.com/QubesOS/qubes-issues/issues/1455 .

Regards,
Vít Šesták 'v6ak'

[Jean-Philippe Ouellet]

On Sat, Dec 24, 2016 at 8:58 AM, Vít Šesták
<groups-no-private-mail--con...@v6ak.com>
wrote:

> 2. There should be some exception for notifications. That is, dom0 would be able to play some predefined sound(s) that inform the user that there is something new.

But how could we prevent a domU from just playing the notification
sound and contents? I can think of two ways:

1) Filter out some "trusted sounds" from domUs.

2) Have some kind of input event following which sound may only come
from dom0. For example an "identify domain" shortcut, which would stop
all domU audio and say the name of the domain of the focused window.

#1 seems to me like a very fragile idea doomed to fail, and #2
requires being triggered by the user and not dom0. This raises a
problem that notifications are likely to be caused by events not
triggered by user input.

So perhaps some untrusted "event pending" sound should be played to
indicate that a user should press the trusted "tell me next event"
key?

[Jean-Philippe Ouellet]

Jorge,

Is there some software you currently recommend for screen reading on
linux in general?

[Vít Šesták]

If centralized notifications are implemented, then NotifyVM would trigger it. If NotifyVM == dom0, it can play the sound anytime. Else, it can call some RPC service in dom0. Maybe dom0 would be not play multiple notification sounds in parallel, in order to prevent mixing other sounds from this one.

This would be just some event pending notification, as you write. I believe this is also the best way from UX PoV, because it does not distract the user too much, unless the user wants to. However, I might be wrong there. Jorge, what's your opinion? (I'll also ask one blind guy. Opinion of two people is not much, but it might provide us at least a rough idea.)

If all VMs still have the notifications on their own (i.e. the current state), a sound could be made through RPC. Again, the domU would not still care what sound is played.

I agree that filtering the trusted sound is very fragile, especially if you don't want to add a latency. I'd say this is virtually no way.

Other ideas (beyond MVP):

If there are too much apps that handle notifications on their own, some additional notification (e.g., “MailVM has played a sound.”) could be added.

I also had one more idea, but I forgot it. 😔

Regards,
Vít Šesták 'v6ak'

[john.david.r.smith]

> I agree that filtering the trusted sound is very fragile, especially if you don't want to add a latency. I'd say this is virtually no way.

this problem maybe could be solved without filtering:
* on setup the user chooses a notification sound (maybe one for each
color)
* the sounds should be random generated (here i don't know how easy it
is to generate easy distinguishable randomized notification sounds)
* since most users will use the default selection, it should be
randomized.
* every time the focus changes, the notification sound is played (if
different sounds are chosen for each color, the user even knows the
color of the active dom)
* if the user presses a key-combination (intercepted by dom0), the
sound is played again (maybe followed by some tts component saying the
vmname).

since the attacker can't know the the sound (if it is possible to create
such random sounds well enough), it can't be faked (except brute force
is used, which could be detected by the user).

the user maybe could choose a sequence of sounds.

as already posted, all other sounds should be muted when the
notification sound is played.

maybe the best way would be to get the user to configure the
confirmation sound by recording some custom sound with a microphone.
this would be much harder to fake than random sounds.
especially if the user records her/himself saying the vmname (and i
guess this would be the most secure way).

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I'd not depend on VM-not-knowing-the-sound. While it could be achieved
initially, I think it will eventually leak into VM. For example when
user assign a microphone to a VM. But the idea of a key intercepted by
dom0 to play VM name (and/or some per-VM or per-label sound) is good. Of
course all VMs should be muted for this time.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYYjYPAAoJENuP0xzK19csheoIAJFTy2vBZ8lwXHGhGMzYcBoN
OF+1URUn/jiEVaoFKf5DBThDqXsGj+GVOWcjpWJzD+ZAOWQ7kHr35eHfpeeCVc4g
7yvG4DUhjfbN2K7sAw6c39oLwKPx1rANKHvS4BPxuSSWTUBz3Uvo0Z0HIXcathVC
1cUF3IApOW2BC6DNenxX/ZWKd4s4XXNxmBlHdDxziZeM325LyZ5XQnW/cyTLO5Aj
4udvtPIfhmZHXAbSHLC5KQer/Z8TeQRU/bixDwDth+p5PExseX+oeNTeb1TkRKLH
2bWNTNMyl9MWprBFGAzYVm6pPnQ7y48eezO1Tddp5c1b1SUk+PKUFXqmAQeNMQg=
=Z376
-----END PGP SIGNATURE-----

[john.david.r.smith]

> I'd not depend on VM-not-knowing-the-sound. While it could be achieved
> initially, I think it will eventually leak into VM. For example when
> user assign a microphone to a VM.

at least the issue of data leaking via microphone could be fixed by
disabling microphones during playback.

but i guess this could be impossible, since there is strange
USB-hardware and detecting a microphone could be very hard.

but anyways i think allowing the user to supply a custom sound or to
record a voice clip would:
* add security, since detection of a [few] second[s] signal should be
less error prone than detecting a signal shorter than a second.
* improve usability, since voice is easier to recognize than some beep
meaning "blue vm activated".

[Vít Šesták]

I asgree.

I've realized two potential issues of accessible Qubes:

First, VM could try to temporarily DoS dom0 sound output, e.g. by requesting much memory and opening many windows and issuing some expensive RPC calls. This could be an issue when dom0 has to notify something important, e.g. change of domain. Maybe integrating sound output woth VM could mitigate such attacks. I am not sure how much practical those attacks would be, though.

Second, there is a problem with definition of focus. For example, when pressing alt+tab, the original window has still focus, but user actually interacts with dom0.

Regards,
Vít Šesták 'v6ak'