LSR as an AT framework
Steve Lee
LSR features that might be of interest to us. It is clear that the LSR
designed has considered it becoming a general purpose Python framework
for assistive technology and attention has be given to abstracting
from the accessibility APIs and other platform dependencies. Thought
also went into the possibility of supporting alternative input.
Currently it only has code for for Linux, AT-SPI and GTK though other
platforms can be added in a modular way.
Peter's published a version of his introductory post on the LSR FAQ
http://live.gnome.org/LSR/FrequentlyAskedQuestions#extensible
--
Steve Lee
www.oatsoft.org
www.fullmeasure.co.uk
st...@fullmeasure.co.uk
1) What language are the Perks. Python in a restricted context? What's
the script context (API) like?
2) Could we expose scripts to 'users' to give customisation but limit
the functionality there
3) how does it compare to orca scripting, ie is it OO or is the nesting
something other than sub classing.
4) how could it blend with XUL/XBL application in a natural way
5) How flexible is it, how close to what we need for OSK/Alt input
6) How would split the fraemwork out of LSR and manage sharing it.
Would it be best to make it a new platform project?
7) As always with using platform what decisions will it force that
might not be so good.
I guess most will be answered by studying the code?
Bearing in mind I am thinking about a XULRunner based approach and I'm
trying to feel my way around the differences without any great
experience of either or quite where we are going, some further thoughts
are:
XUL is declarative web document standards based whereas LSR takes an
functional application framework approach. I prefer declarative where
possible with functional extensions.
LSR may well offer most of the 'plumbing' and structure needed.
XUL is lightweight which may make adaptation easy (for the reason HTML
became so popular).
XUL may easily offer hosted or server side solutions or indeed web
applications (web 2.0) features. There might be might be a thin / rich
application tradeoff. Thin may offer advantages or cause problems.
However it may well be possible to combine the best of both worlds
easily in a mixed LSR/XUL solution.
Do these help kick off a discussion?
Does anyone have any comments?
Steve
Chris Jones
to adding switching support in onboard.
--
Chris Jones
jabber - skating....@gmail.com
msn - skating_...@dsl.pipex.com
Peter Parente
> the script context (API) like?
The Perks are just Python classes. The API is formed from instance
methods. See
http://www.gnome.org/~parente/lsr/epydoc/public/Task.Tools.All-class.html
for the available methods. See
http://cvs.gnome.org/viewcvs/lsr/src/Perks/GTerminalPerk.py?rev=1.9&view=markup
for a simple example of a Perk.
> 2) Could we expose scripts to 'users' to give customisation but limit
> the functionality there
Scripts are exposed to users to add and remove components. See section
4.4 of our UI spec at
http://www.gnome.org/~parente/lsr/ui/dialogs.html#sec.perk-chooser for
one way a user can add/remove components at runtime. See section 4.1.4
at
http://www.gnome.org/~parente/lsr/ui/dialogs.html#sec.settings-chooser
for one way a user can set components to load each time LSR starts.
> 3) how does it compare to orca scripting, ie is it OO or is the nesting
> something other than sub classing.
It's OO with delegation rather than inheritance. Events propogate from
the top of the Perk stack down to the bottom. Perks can stop events
from propogating or call other Perks at an time.
> 4) how could it blend with XUL/XBL application in a natural way
LSR supports four kinds of extensions that it can dynamically add and
remove at runtime. I've been telling you about one kind of extension, a
Perk. Another kind is a "chooser." The most common kind of chooser is a
GUI dialog. For instance, the LSR settings dialog is just an extension
to LSR. It's not built-in in any way. A Perk actually loads it when the
user presses a key.
Since choosers are extensions, nothing about LSR dictates what GUI
toolkit they must use (or if they even must use a GUI toolkit). In your
case, you could easily have a chooser that displays an on-screen
keyboard using XUL.
> 5) How flexible is it, how close to what we need for OSK/Alt input
Pretty darn flexible. :) The problem is trying to explain just how
flexible without implementing lots of proof of concept demos.
Did you get a chance to see the LSR screen cast I played at the GNOME
summit? If not, it's online at
http://www.gnome.org/~parente/lsr/screencast/lsr-gsummit06.html. It
includes two proof-of-concept interfaces that re-use the LSR platform,
but have little to nothing to do with screen reading. They're just new
sets of extensions.
> 6) How would split the fraemwork out of LSR and manage sharing it.
> Would it be best to make it a new platform project?
I don't think you'd have to fork LSR in any way. If you needed new
functionality added to the platform (likely in the scripting API), we
can definitely work together to make it happen. But to work on the OSK,
I think you could keep LSR intact and simply develop extensions and
adapters for your particular platform.
> 7) As always with using platform what decisions will it force that
> might not be so good.
I'll admit one problem immediately. The LSR code is currently under the
Common Public License (CPL) which is not GPL compatible, though it is
an OSI approved license. This will cause you headaches working with GPL
code. However, we are currently seeking approval from the
powers-that-be at IBM to change to the Berkeley License which *is* GPL
compatible.
> I guess most will be answered by studying the code?
>
> Bearing in mind I am thinking about a XULRunner based approach and I'm
> trying to feel my way around the differences without any great
> experience of either or quite where we are going, some further thoughts
> are:
>
> XUL is declarative web document standards based whereas LSR takes an
> functional application framework approach. I prefer declarative where
> possible with functional extensions.
> LSR may well offer most of the 'plumbing' and structure needed.
> XUL is lightweight which may make adaptation easy (for the reason HTML
> became so popular).
> XUL may easily offer hosted or server side solutions or indeed web
> applications (web 2.0) features. There might be might be a thin / rich
> application tradeoff. Thin may offer advantages or cause problems.
>
> However it may well be possible to combine the best of both worlds
> easily in a mixed LSR/XUL solution.
I'm no XUL experts, so I'm not in a position to compare an LSR-based
approach with a XUL approach. However, I think the comparison is worth
the effort and a hybrid solution is worth considering.
What exactly are you thinking of using XUL/XBL for? Just the graphical
UI of the keyboard? If so, then I think that fits nicely into the LSR
framework as a chooser (the view) with Perks defining the logic (the
model) of the OSK globally, per application, etc.
Just thinking outloud...
Pete
Steve Lee
Even without detailed understanding it appears to exceed my
expectations. In particular the input side seems to have the
abstractions/patterns that be suitable for our purposes of handling
alternative input (I was thinking we want to fold input into common
virtual events with some specifics and your gestures look like a part
of the solution). In addition the code quality and 'ornamentation'
plus the users documentation are good.
I'd say a big chunk of the required work is done but it looks like
'fair old' learning curve lies ahead.
Your hunch that we'd be working in the extension/adapter space seems
reasonable to me though I'm sure there would be some core changes
required.
The one important question that I didn't ask is: How easy do you think
it will be to port to Windows (and Mac). What ae the points of contact
with the OS/Desktop platform? My hunch is you've pushed much down into
choosers and events in Perks.
Out of interest how long has it been under dev.
How many of you working on it now? I saw Brett (Hawking toolbar)
worked on the sample Perk.
Thanks for coming clean on the licence issue. I'd say this is
persuasive enough to want to resolve that. Perhaps by the time of
release we will be in a better position but we'd have to be careful
about any interim published code.
I'm sold and will spend time looking into how I can use it.
Aaron: thanks for hooking us all up on this.
Steve
Steve Lee
> UI of the keyboard? If so, then I think that fits nicely into the LSR
> framework as a chooser (the view) with Perks defining the logic (the
> model) of the OSK globally, per application, etc.
Well Mozilla is a complete platform for web style applications and
offers many technologies of interest. Chief here are the XUL/XBL GUI
(plus SVG for cool graphics). I'm a fan of component architectures and
xpCOM would provide basic architectural support as well as forcing
good design with interfaces. Splitting up into re-usable components is
attractive and the components would be available from XUL pages via
javascript which may open up the possibilities. Perhaps key parts of
LSR could be wrapped in xpCOM components.
2 other big advantages are the supported standards and the light
weight declarative + script nature of the web documents used lend
themselves to RAD and also encourage take up by others (look at how
fast HTML was a hit). It's not completely clear how that might work in
this space but a way of allowing easy 'end user' customisation and web
deployment is interesting.
The many xpCOM components are of less obvious utility right now but
then who knows at this point when those services such as connectivity
would be a useful resource.
Using XUL runner allows python scripting via xpPython as Python is not
a native scripting language (though I believe might be at Firefox 3.0)
I'm now interested in how Mozilla (XUL etc) and LSR will fit. Both are
platforms that handle events and expect to be 'in charge'. Either may
want to be 'top dog' and you naturally have a LSR-centric view.
However I'm sure it can be done to create a fantatic platform for
Assitive technology applications. This is going to be fun.
Steve
Peter Parente
I'm going to play both LSR-promoter and devil's advocate in my
responses. I think it's my responsibility to be both.
> Gosh Peter that looks like a very mature and effective architecture.
> Even without detailed understanding it appears to exceed my
> expectations.
Thanks. Be sure to keep in mind that we have not yet attempt to port
LSR to another platform. Though we designed LSR to hide all the AT-SPI
details from scripts and even the scripting API, there may be some
issues that we have not yet considered.
> In particular the input side seems to have the
> abstractions/patterns that be suitable for our purposes of handling
> alternative input (I was thinking we want to fold input into common
> virtual events with some specifics and your gestures look like a part
> of the solution). In addition the code quality and 'ornamentation'
> plus the users documentation are good.
The gestures are intended to be device agnostic and could be extended
as needed.
> Your hunch that we'd be working in the extension/adapter space seems
> reasonable to me though I'm sure there would be some core changes
> required.
Certainly. We'll be happy to make simple changes as needed and larger
changes after reviewing how they impact our already functioning screen
reader UI.
> The one important question that I didn't ask is: How easy do you think
> it will be to port to Windows (and Mac). What ae the points of contact
> with the OS/Desktop platform? My hunch is you've pushed much down into
> choosers and events in Perks.
There is no AT-SPI specific code in Perks. There is no AT-SPI specific
code in the scripting API either. The components that have AT-SPI
specific code are:
1) Adapters/ATSPI, which adapts AT-SPI events and objects to our
internal interfaces (see LSRInterfaces.py). The package is designed
such that you could create Adapters/<your toolkit here>, import it, and
have any scripts continue to function as-is. The scripting API queries
an object provided to it to see if an adapter exists to map it to one
of our internal interface. Look at some of the methods in
Task/Tools/View.py to see how we use the name of an interface to
automatically select the correct adapter for an object. Look at
AccAdapt.py and the code in Adapters/ATSPI to see how this works.
2) pyLinAcc/, which is our connection to the desktop accessibility
infrastructure on a given platform. You would have to provide an
equivalent
3) AccessEngine, which runs the platform-specific message pump for LSR.
This could be abstracted to another module or protected with some
try/excepts.
What this implementation abstraction does not account for is
differences in the design of each accessibility toolkit. In other
words, our internal interfaces provide methods that are necessary and
sufficient for full access to AT-SPI. The interfaces may have methods
that are not supported by other toolkits (e.g. getting relations in
MSAA) or lack methods that are supported in other toolkits (e.g. object
subroles in Apple's UA). I believe the exception handlers in the
scripting API would properly handle the case of an unsupported feature
(e.g. adapter raises NotImplementedError, scripting API method exits
gracefully returning a reasonable value if needed). Again, this is
untested because we've never tried porting.
Choosers do have platform specific code. At present, all of our GUI
dialogs are written to use gtk. If you wanted to have a XUL equivalent
of our SettingsChooser, for instance, you'd have to implement a
XULSettingsChooser from scratch. Then you'd load the XULSettingsChooser
as an extension instead of our current SettingsChooser.
This is an area where another abstraction layer could be added for
GUIs. We didn't concentrate on abstracting the GUI toolkit because we
have been focusing on building a screen reader, first and foremost. But
that's not to say it's impossible.
> Out of interest how long has it been under dev.
> How many of you working on it now? I saw Brett (Hawking toolbar)
> worked on the sample Perk.
Brett worked on the GaimPerk and parts of the infrastructure. Others at
IBM worked on parts of the core. LSR has been under development since
May, 2005 with its first release on gnome.org in May, 2006.
> Thanks for coming clean on the licence issue. I'd say this is
> persuasive enough to want to resolve that. Perhaps by the time of
> release we will be in a better position but we'd have to be careful
> about any interim published code.
I'll keep you posted on our progress.
> I'm sold and will spend time looking into how I can use it.
> Aaron: thanks for hooking us all up on this.
>
> Steve
>
> On 11/5/06, Peter Parente <pare...@gmail.com> wrote:
>
>
>
> > > 1) What language are the Perks. Python in a restricted context? What's
> > > the script context (API) like?
>
> > The Perks are just Python classes. The API is formed from instance
> > methods. See
> >http://www.gnome.org/~parente/lsr/epydoc/public/Task.Tools.All-class....
> > for the available methods. See
> >http://cvs.gnome.org/viewcvs/lsr/src/Perks/GTerminalPerk.py?rev=1.9&v...
Steve Lee
In addition to finding AT-SPI features not handled by other a11y APIs we also have the opposite of wanting to use features in the others that are not in SPI. I guess you have 3 main approaches
1) create a least common denominator API and forget the extras
2) expand the abstraction to include the new features but simulate in other versions
3) provide extension mechanisms.
The last leads back to unportable code or pushes the portability logic up the stack so is to be avoided.
This must be a fairly standard 'pattern' and we should find good examples somewhere.
Steve
that are not supported by other toolkits ( e.g. getting relations in
Peter Parente
New Berkeley Software Distribution (BSD) license. This change makes the
LSR code GPL-compatible without introducing a copyleft restriction.
Enjoy!
Pete
Steve Lee
I'll be looking at LSR for device input next year and now we're free
to use it if that looks like being the right way to go.
--
Steve Lee
www.oatsoft.org
www.schoolforge.org.uk
www.fullmeasure.co.uk