Request for feedback: CSS-based widget layout

[Russell Keith-Magee]

Hi all,

I've just pushed an exploratory branch that I've been working on for the last couple of days, and I'm interested in feedback.

The problem I've been trying to solve is the complexity associated with the constraint-based layout algorithm that Toga currently uses. Constraint-based layout has three problems, IMHO:

 * A really simple layout requires a lot of code - most widgets will require at least 4 constraints, which translates into a lot of lines of fairly dense code.

 * When a constraint layout fails, it's very difficult to debug - bugs manifest somewhere between incomprehensible layout behaviour and system crashes.

 * The Cassowary algorithm that underpins constraint-based layout is complex to implement, and can be computationally intensive to evaluate. 

 * OS/X and iOS provide their own implementations of Cassowary, but the specifics of that implementation aren't documented, so there are some interesting inconsistencies in Toga behaviour between OS/X and GTK/Windows/Android as a result.

 * Expressing constraints in code requires either extremely verbose syntax (see the OS/X and iOS native APIs), or unintuitive "expression as argument" syntax (Toga's API). 

Earlier this year, Facebook provided a possible alternative - using CSS, and in particular FlexBox, as a layout mechanism. They open sourced a partial implementation of flexbox [1], and demonstrated how it can be used through their React Native project.

[1] http://github.com/facebook/css-layout

This approach has a number of benefits, IMHO:

 * Flexbox fixes many of the problems with CSS layout on the web, and is relatively easy to understand and debug. 

 * The implementation of Flexbox is also relatively easy to understand, and layouts can be computed relatively quickly.

 * Flexbox layout is "native nowhere", so all Toga platforms would have implementation parity.

 * Looking forward - an API to expose Toga widgets in a browser would now be possible, because CSS *is* native in the browser.

So - what would CSS-based layout look like in practice? Well, I've done an exploratory port of Facebook's CSS-layout code to Python:

https://github.com/pybee/colosseum

and I've created a 'colosseum' branch in both the toga-cocoa and toga-core repositories:

https://github.com/pybee/toga/tree/colosseum

https://github.com/pybee/toga-cocoa/tree/colosseum

If you put the code from these branches, plus colosseum, in your PYTHONPATH, you can use CSS-based layout on OS/X. 

The following example code shows the build method for Toga Tutorial 1 - the F to C calculator. The major differences are the introduction of a container for each of the "rows" in the layout (c_container and f_container), and the use of the .style() method to set layout style.

==== 

    def build(app):

        c_container = toga.Container()

        f_container = toga.Container()

        container = toga.Container()

        c_input = toga.TextInput(readonly=True)

        f_input = toga.TextInput()

        c_label = toga.Label('Celcius', alignment=toga.LEFT_ALIGNED)

        f_label = toga.Label('Fahrenheit', alignment=toga.LEFT_ALIGNED)

        join_label = toga.Label('is equivalent to', alignment=toga.RIGHT_ALIGNED)

        def calculate(widget):

            try:

                c_input.value = (float(f_input.value)- 32.0) * 5.0 / 9.0

            except Exception:

                c_input.value = '???'

        button = toga.Button('Calculate', on_press=calculate)

        f_container.add(f_input)

        f_container.add(f_label)

        c_container.add(join_label)

        c_container.add(c_input)

        c_container.add(c_label)

        container.add(f_container)

        container.add(c_container)

        container.add(button)

        container.style(flexDirection='column', paddingTop=10)

        f_container.style(flexDirection='row', margin=5)

        c_container.style(flexDirection='row', margin=5)

        c_input.style(flex=1)

        f_input.style(flex=1, marginLeft=160)

        c_label.style(width=100, marginLeft=10)

        f_label.style(width=100, marginLeft=10)

        join_label.style(width=150, marginRight=10)

        button.style(margin=5)

====

At this point, I'm reasonably convinced this change would be a net win, but I'm interested in whether others agree.

The weaknesses I can see in this approach are mostly tied to the limitations of colosseum, rather than the approach itself - in particular, support for min/max width height on widgets, and percentage based width/height specifications would be nice to have. These are well defined in the CSS spec, but not in the algorithm provided by Facebook, so there's no reason they couldn't be implemented AFAICT.

Questions? Comments?

Yours,

Russ Magee %-)

[Tzu-ping Chung]

Hi Russ,

I think this is a great idea. All the problems you outlined for constraint-based layout are spot-on; the second and fourth points are, in my experience, particularly annoying implementing things with Toga. There are multiple times I got stuck in a constraint problem that’s just impossible to debug, and had to nuke everything and start all over again. And since Autolayout in OS X and iOS projects have the same problem, I don’t think there’s anyway to get around it, as long as Cassowary is used. So this seems like a good way out.

As for the interface, however, the example below is still pretty verbose to me. I would wish for a more declarative way to specify styling, instead of separate .style calls. I made a Gist to demonstrate what I mean:[*]

https://gist.github.com/uranusjr/02ccb82ff02e5531ef6a

This is inspired partly by django-crispy-forms’s FormHelper, which I think provides a very nice API for UI layouts in Python code.

[*]: I didn’t really implement the API and test the code, so it probably won’t work. But I hope it shows the idea.

Yours,

TP

--
Tzu-ping Chung (@uranusjr)
uran...@gmail.com
https://uranusjr.com

--
You received this message because you are subscribed to the Google Groups "Beeware Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beeware-develop...@googlegroups.com.
To post to this group, send email to beeware-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/beeware-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/beeware-developers/CAJxq84_D0tGUx-yQkL_AYRaAnaGvkYTcGN3bGJBikK_-gwwnEw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Austin Hicks]

Well, this is certainly interesting.
The big thing I see here, and the reason I'm replying is that, if you still care about accessibility, it might prove problematic.  The emphasis here is on might.  The specific issue I'm seeing is that label creation and tab order can matter, and this might encourage people to make, say, a bunch of labels.  And then move them to the right places with CSS.  And then make the controls.
I need to find out how screen readers on Windows associate labels with controls.  There's people I can ask, but I believe it's tab order.  I'm not sure what happens for other OSes, but I'm interested in Toga because it holds promise to be an accessible GUI toolkit, something which we kind of don't have (as in blind programmers, the ones who care the most, have to jump through hoops for anything more advanced than the very basics and figure out workarounds for each platform.  My friend had to write an abstraction over WX because regular listboxes don't work on mac, for example).
Assuming it doesn't, I'm not exactly opposed.  I find it easier to think in constraint layout, but this is probably a shortcoming of me and not the approach under discussion.

[Gabriel Csapo]

On Sunday, March 29, 2015 at 4:26:42 AM UTC-4, Tzu-ping Chung wrote:

Hi Russ,

I think this is a great idea. All the problems you outlined for constraint-based layout are spot-on; the second and fourth points are, in my experience, particularly annoying implementing things with Toga. There are multiple times I got stuck in a constraint problem that’s just impossible to debug, and had to nuke everything and start all over again. And since Autolayout in OS X and iOS projects have the same problem, I don’t think there’s anyway to get around it, as long as Cassowary is used. So this seems like a good way out.

As for the interface, however, the example below is still pretty verbose to me. I would wish for a more declarative way to specify styling, instead of separate .style calls. I made a Gist to demonstrate what I mean:[*]

https://gist.github.com/uranusjr/02ccb82ff02e5531ef6a

This is a really verbose way of explaining a particular look and feel. Would it not be easier to make a gui editor?  

[Russell Keith-Magee]

Hi Tzu-ping

On Sun, Mar 29, 2015 at 4:26 PM, Tzu-ping Chung <uran...@gmail.com> wrote:

Hi Russ,

I think this is a great idea. All the problems you outlined for constraint-based layout are spot-on; the second and fourth points are, in my experience, particularly annoying implementing things with Toga. There are multiple times I got stuck in a constraint problem that’s just impossible to debug, and had to nuke everything and start all over again. And since Autolayout in OS X and iOS projects have the same problem, I don’t think there’s anyway to get around it, as long as Cassowary is used. So this seems like a good way out.

As for the interface, however, the example below is still pretty verbose to me. I would wish for a more declarative way to specify styling, instead of separate .style calls. I made a Gist to demonstrate what I mean:[*]

https://gist.github.com/uranusjr/02ccb82ff02e5531ef6a

This is inspired partly by django-crispy-forms’s FormHelper, which I think provides a very nice API for UI layouts in Python code.

[*]: I didn’t really implement the API and test the code, so it probably won’t work. But I hope it shows the idea.

What you're describing here is should be possible as long as you can:

 * pass the list of children to the constructor of a Container

 * pass the style arguments in the constructor of a Widget

If you meet these two basic requirements, the rest of your DSL syntax gets a lot simpler - you don't need toga.Style(), and the tuple syntax isn't needed either AFACIT:

Container(

    Container(

        TextInput(flex=1, marginLeft=10),

        Label(text='Fahrenheit'), 

        flexDirection='row',

        margin=5,

    ),

    Container(

        Label(text='is equal to'), 

        TextInput(flex=1, marginLeft=10),

        Label(text='Celcius'), 

        flexDirection='row',

        margin=5,

    ),

    Button(label='Calculate', on_press=calculate, margin=5),

    paddingTop=10,

    flexDirection='column'

)

Internally, this would potentially simplify a few things. Widgets currently have a colosseum "CSSNode" as an attribute that is used to manage layout; this syntax would effectively make a base Toga Widget a subclass of CSSNode. This would mean every widget would end up with a lot more arguments (all the style parameters); but then, they're all fundamental descriptors of the widget, so maybe that isn't a bad thing.

Alternatively, you could push all these arguments down a layer:

    Container(

        children=[

            TextInput(flex=1, marginLeft=10),

            Label(text='Fahrenheit'), 

        ],

        style={

            'flexDirection': 'row',

            'margin': 5,

        }

    )

    ...

This is more verbose, and adds some explicitness; but I'm not sure if that explicitness is valuable.

Personally, I'm not completely sold on the idea of a DSL-like syntax approaches. I know some people are enamoured with them; personally, I generally prefer the linear, define entities, define structure, define style approach. However, that is a stylistic choice, and in some situations (e.g., layout of simple dialogs), I can see how a DSL approach could be beneficial; if it doesn't massively complicate the API for widgets, then we might as well support this approach IMHO.

Yours,

Russ Magee %-)

 

[Russell Keith-Magee]

Hi Austin,

On Sun, Mar 29, 2015 at 8:48 PM, Austin Hicks <caml...@gmail.com> wrote:

Well, this is certainly interesting.
The big thing I see here, and the reason I'm replying is that, if you still care about accessibility, it might prove problematic.  The emphasis here is on might. 

I definitely still care about accessibility. If accessibility presents a reason why the CSS-based approach isn't viable, that would be a deal-breaker for me.

 

The specific issue I'm seeing is that label creation and tab order can matter, and this might encourage people to make, say, a bunch of labels.  And then move them to the right places with CSS.  And then make the controls.
I need to find out how screen readers on Windows associate labels with controls.  There's people I can ask, but I believe it's tab order. 

I haven't done anything explicit to support accessibility or tab order, and I'll certainly defer to your experience in this area.

However, based on my limited knowledge, I would have thought that the CSS approach would have *improved* accessibility by default. The constraint-based approach means you can insert children into a container in any order you want; the constraint satisfaction algorithm then puts those widgets on the page. You *could* impose an "insertion order is screen reader order" semantic (and I don't know if GUI toolkits do this by default), but unless you're explicitly testing *for* a screen reader, that wouldn't manifest in any tangible way - you could just insert all the labels, then insert all the inputs, then all the buttons, and the constraints would put them in the right place on the screen.

The CSS-based approach means that the order of insertion really does matter, and that order will be right-to-left, top-to-bottom, so that the boxes lay out in a helpful manner (substituting R-to-L with "natural text direction" for i18n purposes). R-to-L, T-to-B will generally match the preferred screen reader order (as browsers demonstrate). For the cases where it doesn't, style attributes like "tabIndex=" and "label for=" to provide a way to set an optimised screen reader path (or we might be able to come up with a better mechanism based on top level containers tracking screen reader ordering). I haven't implemented any of these extra attributes/approaches so far, but that's an omission due to time constraints, not intent.

 

I'm not sure what happens for other OSes, but I'm interested in Toga because it holds promise to be an accessible GUI toolkit, something which we kind of don't have (as in blind programmers, the ones who care the most, have to jump through hoops for anything more advanced than the very basics and figure out workarounds for each platform.  My friend had to write an abstraction over WX because regular listboxes don't work on mac, for example).

As I said - I'm definitely interested in supporting accessibility. My knowledge of the subject is somewhat limited - mostly a light dusting of knowledge from web design space. My understanding of the application of that knowledge to native GUI design is definitely limited. If there are specific APIs or techniques that we need to keep in mind, please let me know.

 

Yours,

Russ Magee %-)

--

You received this message because you are subscribed to the Google Groups "Beeware Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beeware-develop...@googlegroups.com.
To post to this group, send email to beeware-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/beeware-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/beeware-developers/8ace5f51-94e7-4ac8-b941-b450370b72cc%40googlegroups.com.

[Russell Keith-Magee]

On Mon, Mar 30, 2015 at 12:22 AM, Gabriel Csapo <gabe...@gmail.com> wrote:

On Sunday, March 29, 2015 at 4:26:42 AM UTC-4, Tzu-ping Chung wrote:

Hi Russ,

I think this is a great idea. All the problems you outlined for constraint-based layout are spot-on; the second and fourth points are, in my experience, particularly annoying implementing things with Toga. There are multiple times I got stuck in a constraint problem that’s just impossible to debug, and had to nuke everything and start all over again. And since Autolayout in OS X and iOS projects have the same problem, I don’t think there’s anyway to get around it, as long as Cassowary is used. So this seems like a good way out.

As for the interface, however, the example below is still pretty verbose to me. I would wish for a more declarative way to specify styling, instead of separate .style calls. I made a Gist to demonstrate what I mean:[*]

https://gist.github.com/uranusjr/02ccb82ff02e5531ef6a

This is a really verbose way of explaining a particular look and feel. Would it not be easier to make a gui editor?  

... it's easier to build a *GUI editor*? Ok... you first :-)

Seriously - as I said in my earlier response to Tzu-ping, there's a lot in his syntax that is superfluous; it can be simplified quite a bit. The real question is whether a DSL-like language for GUI layout is desirable; that question is mostly orthogonal to the CSS layout question.

Yours,

Russ Magee %-)

[Gabriel Csapo]

I was being a little sarcastic. I think the problem with layout is not the syntax it is the way some people visualize that syntax. Without having a visual aid, it is hard to understand a constrained layout and sometimes it is a pedantic process trying to get a certain layout, but for the most part, building forms and or simple layouts is perfectly understandable with a constrained layout. Also constrained layouts are not just an cocoa realization, they exist in all realms of ui. Even android doesn't know how to fix this and they use xml....

The GUI editor solves a lot of headache for people because it takes the guess work out. I for one don't think that changing the syntax for layout elements will help solve the bigger issue of visualizing that desired look.

Yours,

Russ Magee %-)

[Russell Keith-Magee]

On Mon, Mar 30, 2015 at 9:51 AM, Gabriel Csapo <gabe...@gmail.com> wrote:

On Sunday, March 29, 2015 at 9:29:11 PM UTC-4, Russell Keith-Magee wrote:

On Mon, Mar 30, 2015 at 12:22 AM, Gabriel Csapo <gabe...@gmail.com> wrote:

On Sunday, March 29, 2015 at 4:26:42 AM UTC-4, Tzu-ping Chung wrote:

Hi Russ,

I think this is a great idea. All the problems you outlined for constraint-based layout are spot-on; the second and fourth points are, in my experience, particularly annoying implementing things with Toga. There are multiple times I got stuck in a constraint problem that’s just impossible to debug, and had to nuke everything and start all over again. And since Autolayout in OS X and iOS projects have the same problem, I don’t think there’s anyway to get around it, as long as Cassowary is used. So this seems like a good way out.

As for the interface, however, the example below is still pretty verbose to me. I would wish for a more declarative way to specify styling, instead of separate .style calls. I made a Gist to demonstrate what I mean:[*]

https://gist.github.com/uranusjr/02ccb82ff02e5531ef6a

This is a really verbose way of explaining a particular look and feel. Would it not be easier to make a gui editor?  

... it's easier to build a *GUI editor*? Ok... you first :-)

Seriously - as I said in my earlier response to Tzu-ping, there's a lot in his syntax that is superfluous; it can be simplified quite a bit. The real question is whether a DSL-like language for GUI layout is desirable; that question is mostly orthogonal to the CSS layout question

I was being a little sarcastic. I think the problem with layout is not the syntax it is the way some people visualize that syntax. Without having a visual aid, it is hard to understand a constrained layout and sometimes it is a pedantic process trying to get a certain layout, but for the most part, building forms and or simple layouts is perfectly understandable with a constrained layout. Also constrained layouts are not just an cocoa realization, they exist in all realms of ui. Even android doesn't know how to fix this and they use xml....

XML... well there's the first problem... :-)

Again, seriously - I know about Android's layout schemes; they're quite different to iOS constraints. I haven't seen any platform that uses the Cassowary algorithm in the same way that iOS/OSX does. And regardless of the layout scheme, I agree that a GUI tool to visualise that layout is valuable. However, it shouldn't be *required*. 

The advantage that I see in the CSS-based approach is that the "no style" layout is, to the first approximation, useful - widgets are all displayed, in some semblance of "the right place", and styling is then used to push them somewhere more appropriate. There's such a thing as a *bad* CSS layout, but there's no such thing as an *impossible* layout. Compare and contrast with Cassowary, where there are actually mathematically underspecified, overspecified, and impossible layouts, and there isn't a clear way to identify, diagnose, or resolve those problems. GUI tools certainly help, but even Xcode's tools make designing a constraint-based layout a complex process. The CSS-based equivalent would allow for essentially the same UI as browser dev-tools - a live view of a layout, allowing you to tweak style attributes until you have what you want.

The GUI editor solves a lot of headache for people because it takes the guess work out. I for one don't think that changing the syntax for layout elements will help solve the bigger issue of visualizing that desired look.

 

For sure. And, to be fair, a GUI layout tool is on my list of tools I would like to see in the Toga ecosystem one day. I'm just not expecting to see it any day soon :-)

Yours,

Russ Magee %-)

 

[Gabriel Csapo]

I am not discounting the fact that this is a large issue, but would you consider delaying this implementation till all toga gui libraries have enough features to facilitate the need for a new layout strategy?

I can devote a lot of my time to helping with the gtk and osx libraries, but implementing this will cause a set back in progress and debugging issues that might set this back a couple of months.

Again, I really like the idea of this and the reason I want to contribute to toga is because it makes building native applications a understandable and non-comprehensive process, but really would like to see all the widgets having events and implementation documentation before a new layout strategy is considered.  

[Russell Keith-Magee]

Hi Gabriel,

On Tue, Mar 31, 2015 at 2:47 AM, Gabriel Csapo <gabe...@gmail.com> wrote:

I am not discounting the fact that this is a large issue, but would you consider delaying this implementation till all toga gui libraries have enough features to facilitate the need for a new layout strategy?

I can devote a lot of my time to helping with the gtk and osx libraries, but implementing this will cause a set back in progress and debugging issues that might set this back a couple of months.

I doubt this is the case, for two reasons.

Firstly, the layout change is orthogonal to most of the code adding features to a widget. Most of the work to support Colosseum on OS/X is at the base Widget level, and is mostly done (I might tweak some bits, like turning the base Widget into a subclass of CSSNode). Widgets that haven't been ported generally only need 1 extra method in their Cocoa-side Impl, and a one line change in the startup() method. As proof of the point - most widgets don't have any code directly relating to the current layout scheme.

Secondly, if we're going to make a fundamental change to the framework, I'd like to do that *before* we have a large base of code that needs to be ported and updated.

Lastly, in terms of debugging issues - I don't know if you've taken a look at the GTK+ port, but one of the major reasons I looked into CSS layout was because I was hitting a brick wall debugging some behaviours in constraint-based layout - mostly because striving for parity with OS/X is almost impossible without actually having visibility on the OS/X implementation. If anything, this change will make debugging easier, because the layouts *will* be consistent between platforms.

Again, I really like the idea of this and the reason I want to contribute to toga is because it makes building native applications a understandable and non-comprehensive process, but really would like to see all the widgets having events and implementation documentation before a new layout strategy is considered.  

Again - I'm of the opposite opinion here - I'd like to get the groundwork sorted before we've got a bunch of documentation that needs to be updated.

Yours,

Russ Magee %-)

[Austin Hicks]

I got information for windows specifically directly from james Teh, one of the developers on NVDA.  This looks like it's not a problem on that platform: he says that there's a way to annotate textboxes and such with MSAA without having to do a full implementation and that it's only 10 lines or so in C/C++.  I should be able to work out this code fairly quickly, should it be necessary.

According to him, however, it's creation order that matters.  I can't find official docs on this because screen reader development is the darkest black magic you can imagine, but he is telling me that the default implementation of MSAA labels textboxes automatically if a static text is created just before them.  The beauty of constraint-based layout dsls is that you often call constructors, and you often call constructors for labels immediately before the things they label.  I think this entire issue can be simply worked around by providing a labelled textbox of some sort and, in addition, that's a pretty common thing to use anyway.

On Mac, the accessibility hierarchy is exposed directly with no intermediate interpretation.  This is why most blind people dislike it so much, but it also means that this question has less importance there.  From my time on the platform, I am almost certain that Voiceover actually sorts controls left to right, top to bottom, but I'd need to actually do nontrivial experiments to determine if this statement is true.

So I think accessibility is probably a nonissue in regard to this topic, especially if someone writes a best practices guide.  But you're overestimating my expertise: I don't have a huge deal of first-hand experience save that I'm a pretty competent programmer who happens to be blind and, in that situation, you can't help but dig into it some at some point.

As for my personal preference, though: I love Cassowary.  Finding a GUI library that would give me constraint-based layout in a way which is not painful and tied to the pixel made me very happy.  This is primarily because of my visual impairment, consequently giving me zero motivation to actually learn CSS and what we can call a rather unique perspective on the issue.  But I am a minority.  Would it be possible to have constraint-based layout on top of CSS at some point?

And does CSS solve the problem of everything from a 5-inch to a 50-inch screen better than Cassowary?  Because given the platforms Toga aims to support, I think that might be the real problem and that encouraging people to go for a specific look on one device will not help.

To view this discussion on the web visit https://groups.google.com/d/msgid/beeware-developers/CAJxq84-zCBpmeBUT6T1aRz3pH5fr1WdVLsuJP6FCdCLZ55pRwQ%40mail.gmail.com.

[Russell Keith-Magee]

Hi Austin,

On Fri, Apr 3, 2015 at 7:47 AM, Austin Hicks <caml...@gmail.com> wrote:

I got information for windows specifically directly from james Teh, one of the developers on NVDA.  This looks like it's not a problem on that platform: he says that there's a way to annotate textboxes and such with MSAA without having to do a full implementation and that it's only 10 lines or so in C/C++.  I should be able to work out this code fairly quickly, should it be necessary.

According to him, however, it's creation order that matters.  I can't find official docs on this because screen reader development is the darkest black magic you can imagine, but he is telling me that the default implementation of MSAA labels textboxes automatically if a static text is created just before them.  The beauty of constraint-based layout dsls is that you often call constructors, and you often call constructors for labels immediately before the things they label.  I think this entire issue can be simply worked around by providing a labelled textbox of some sort and, in addition, that's a pretty common thing to use anyway.

Thanks for looking into this.

By creation order - do you mean the order in which the widgets are instantiated, or the order in which they're added to the page? If it's instantiation order, then there's no difference between Cassowary and CSS - you can create the widget objects in any order you want. A problem only arises if it's the order in which they're added to the page, because unlike Cassowary, CSS *is* document order dependent. 

That said - the CSS flexbox has a flex-order attribute that can be used if the order of definition is different to the document order. Colosseum (the flexbox implementation I'm using) doesn't implement this at present, but it could be added if there was sufficient need.

 

On Mac, the accessibility hierarchy is exposed directly with no intermediate interpretation.  This is why most blind people dislike it so much, but it also means that this question has less importance there.  From my time on the platform, I am almost certain that Voiceover actually sorts controls left to right, top to bottom, but I'd need to actually do nontrivial experiments to determine if this statement is true.

So I think accessibility is probably a nonissue in regard to this topic, especially if someone writes a best practices guide.  But you're overestimating my expertise: I don't have a huge deal of first-hand experience save that I'm a pretty competent programmer who happens to be blind and, in that situation, you can't help but dig into it some at some point.

No worries - your expertise in this area is definitely a lot more that my own, so I'm appreciative of any wisdom you can share, and thankful that we've got someone who has this as matter of personal concern that it won't be forgotten.

As for my personal preference, though: I love Cassowary.  Finding a GUI library that would give me constraint-based layout in a way which is not painful and tied to the pixel made me very happy.  This is primarily because of my visual impairment, consequently giving me zero motivation to actually learn CSS and what we can call a rather unique perspective on the issue.  But I am a minority.  Would it be possible to have constraint-based layout on top of CSS at some point?

It's not something that is trivial to do right now, but it's certainly in the realm of the possible. The creators of Cassowary (Greg Badros and Alex Russell) have produced "Grid Style Sheets":

http://gridstylesheets.org

This is a polyfill for browsers to use constraint-based layout definitions. This proves that it's possible to layer constraint-based layout over the top of CSS. Worst case, you can fall back to the CSS option of marking everything "position:absolute; top: (y); left (x); width: (w); height: (h)". 

The only implementation concern is that we have to make sure that there are appropriate triggers to make sure that any layout change (window size changes, etc) trigger a cassowary recalculation before a CSS relayout. But if we know this is something people want to do, we can easily add those hooks.

 

And does CSS solve the problem of everything from a 5-inch to a 50-inch screen better than Cassowary?  Because given the platforms Toga aims to support, I think that might be the real problem and that encouraging people to go for a specific look on one device will not help.

 

If anything, CSS solves the problem *better* than Cassowary. 

Cassowary will let you say "use 30% of the available width for this widget", but won't re-flow widgets on the page based on how much content is available. It also doesn't have any inherent concept of different layouts for different page sizes/styles. You could certainly build such a setup, but it would be based on deleting all the previous constraints, and then adding new ones as the screen size changed.

On the other hand, CSS clearly does this - it's the basis for responsive layout on the web. The Flexbox algorithm (which, to be clear, is part of the CSS3 spec, and is a significant improvement over the old CSS2.1 box model) will allow you to reflow boxes based on available space. On top of that, CSS has the idea of applying different style sheets based on media size. As a result, a single CSS codebase is able to render mobile, tablet, notebook and 30" screen, adapting appropriately as the screen size changes.

Yours,

Russ Magee %-)

[Tzu-ping Chung]

On 03/4, 2015, at 07:47, Austin Hicks <caml...@gmail.com> wrote:

>
> I got information for windows specifically directly from james Teh, one of the developers on NVDA. This looks like it's not a problem on that platform: he says that there's a way to annotate textboxes and such with MSAA without having to do a full implementation and that it's only 10 lines or so in C/C++. I should be able to work out this code fairly quickly, should it be necessary.
>
> According to him, however, it's creation order that matters. I can't find official docs on this because screen reader development is the darkest black magic you can imagine, but he is telling me that the default implementation of MSAA labels textboxes automatically if a static text is created just before them. The beauty of constraint-based layout dsls is that you often call constructors, and you often call constructors for labels immediately before the things they label. I think this entire issue can be simply worked around by providing a labelled textbox of some sort and, in addition, that's a pretty common thing to use anyway.
>
> On Mac, the accessibility hierarchy is exposed directly with no intermediate interpretation. This is why most blind people dislike it so much, but it also means that this question has less importance there. From my time on the platform, I am almost certain that Voiceover actually sorts controls left to right, top to bottom, but I'd need to actually do nontrivial experiments to determine if this statement is true.

I can confirm this for both OS X and iOS, based on my own experience. VoiceOver read elements from left to right, and then top to bottom. All grouping has to be done manually by the developer with methods and properties in the NSAccessibility and NSAccessibilityElement protocols (or their UIKit counterparts). To override reading direction, you will need to put views inside a common ancestor NSView or UIView, or manage the accessibility hierarchy yourself. So as long as there is a best-practice guide to sort things out, especially for container classes, things should work out fine (at least no worse than native frameworks) on these platforms.

[Austin Hicks]

To clarify on the MSAA point: the default is based on the order in which the handles are created, as far as I understand.  But with annotation we can change it to whatever we want, whenever we want.  If textboxes incorporate their labels in some manner, then the windows textbox need only annotate accessible name to something appropriate, probably the label text.  The important thing is that Toga know what the label is.

I have a tendency to be wordy and to provide way more information than is required, so I'm not surprised this was unclear.

--
You received this message because you are subscribed to the Google Groups "Beeware Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beeware-develop...@googlegroups.com.
To post to this group, send email to beeware-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/beeware-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/beeware-developers/CAJxq849WfQbUrDj76pBucndLwzMsjOuKAJdApTsuooF2qZt__A%40mail.gmail.com.

[Russell Keith-Magee]

On Mon, Apr 13, 2015 at 7:39 AM, Austin Hicks <caml...@gmail.com> wrote:

To clarify on the MSAA point: the default is based on the order in which the handles are created, as far as I understand.  But with annotation we can change it to whatever we want, whenever we want.  If textboxes incorporate their labels in some manner, then the windows textbox need only annotate accessible name to something appropriate, probably the label text.  The important thing is that Toga know what the label is.

 

That doesn't sound like a problem at all. I sounds like we can do *exactly* what CSS does - that is, in the absence of specific information, just use creation order; but also provide "id=" tag on an input element, and "for=" tags on a label to associate a label with an input element.

I have a tendency to be wordy and to provide way more information than is required, so I'm not surprised this was unclear.

No worries - I'm prone to the same thing :-)

Russ %-)