Shelly extensions - discussion

[Ondrej Hanslik]

I have worked with Frank for a while and I noticed there are many things I can't do and they would be nice to have so I would like to open a discussion about possible Shelley extensions.

I will be glad for your thoughts and opinions. Do you think it is a good idea? How should the grammar look like?

Accessibility elements

Shelley can currently inspect only views. However, some of the user interaction can be done only through accessibility elements. One example is the iOS native keyboard which uses elements with typeUIAccessibilityTraitKeyboardKey and UIAccessibilityTraitButton.

It would be nice to have accessibility selectors, e.g. uiaelement:button marked:'Hide Keyboard'. This could be easily combined with the view hierarchy, e.g. view:'MyCustomView' uiaelement:button

This would also require to add some additional interaction selectors (e.g. touch) on NSObject. There is already a similar functionality for Mac.

Custom predicates

Currently there are predefined predicates, e.g. marked:, markedExactly:, isAnimating etc. The all have to return a BOOL. There is no possibility to use logical operations (NOT, OR). It would be nice to allow custom predicates without the need to add custom UIView methods.

Proposal example: view filter:"tag = 10"

This could be easily implemented using NSPredicate, automatically enabling comparisons, regular expressions and logical operations.

Inspecting views returning from methods

Some of the views are very difficult to describe only using marked. It would be nice to have the possibility to find a view and then get next views using selectors on the first view.
Cons: The grammar could get complicated

E.g. view:'UITableView' {"tableHeaderView"} or view:'UITableView' {"headerViewForSection:", 0}

Sorting

The index predicate usually doesn't work properly because the view ordering is random. Unfortunately, it is needed for some situations, e.g. testing if table view cells are sorted correctly.

It would be nice to have a possibilty to sort the returned selectors:

Proposal: view:'UITableViewCell' sort:position

[Taylor, Martin]

In general I think this is a great idea! See my specific responses and comments embedded below:

 

Cheers,

Martin

 

From: frank-...@googlegroups.com [mailto:frank-...@googlegroups.com] On Behalf Of Ondrej Hanslik
Sent: Monday, May 13, 2013 1:13 PM
To: frank-...@googlegroups.com
Subject: [Frank] Shelly extensions - discussion

 

I have worked with Frank for a while and I noticed there are many things I can't do and they would be nice to have so I would like to open a discussion about possible Shelley extensions.

I will be glad for your thoughts and opinions. Do you think it is a good idea? How should the grammar look like?

Accessibility elements

Shelley can currently inspect only views. However, some of the user interaction can be done only through accessibility elements. One example is the iOS native keyboard which uses elements with typeUIAccessibilityTraitKeyboardKey and UIAccessibilityTraitButton.

It would be nice to have accessibility selectors, e.g. uiaelement:button marked:'Hide Keyboard'. This could be easily combined with the view hierarchy, e.g. view:'MyCustomView' uiaelement:button

[CMT: ] I would love to be able to tap the “Hide Keyboard” button on the standard keypad!

This would also require to add some additional interaction selectors (e.g. touch) on NSObject. There is already a similar functionality for Mac.

Custom predicates

Currently there are predefined predicates, e.g. marked:, markedExactly:, isAnimating etc. The all have to return a BOOL. There is no possibility to use logical operations (NOT, OR). It would be nice to allow custom predicates without the need to add custom UIView methods.

Proposal example: view filter:"tag = 10"

This could be easily implemented using NSPredicate, automatically enabling comparisons, regular expressions and logical operations.

[CMT: ] Sounds useful, but I don’t have a specific use-case for this at present.

Inspecting views returning from methods

Some of the views are very difficult to describe only using marked. It would be nice to have the possibility to find a view and then get next views using selectors on the first view.
Cons: The grammar could get complicated

E.g. view:'UITableView' {"tableHeaderView"} or view:'UITableView' {"headerViewForSection:", 0}

[CMT: ] I thought Frank could already do stuff like this.  For example:
view:'_UIPopoverView' view:'UILabel' index:0
view:'_UIPopoverView' view:'UITableViewCell' index:3
view:'UITableViewCell' view:'UILabel' marked:’MyFontName' parent view:'UITableViewCell'

Sorting

The index predicate usually doesn't work properly because the view ordering is random. Unfortunately, it is needed for some situations, e.g. testing if table view cells are sorted correctly.

It would be nice to have a possibilty to sort the returned selectors:

Proposal: view:'UITableViewCell' sort:position

[CMT: ] This would be really useful, but its more complicated than this for scrolled tables.  iOS re-allocates index positions on-the-fly as you scroll a table, and keeps more than just the visible UITableViewCell items in memory.  It may even have some duplicated entries that are actually invisible. I’ve had to write some really complicated code that sorts by the Y-address (taking rotation into account) and then filtering by visibility, in order to get a list of UITableViewCells that are in true positional order AND tappable by their index.  It would be nice to have this feature embedded deeper in Frank, and I may be able to contribute my code, if someone would like to re-work my Python into Ruby or Objective-C.

--
You received this message because you are subscribed to the Google Groups "Frank" group.
To unsubscribe from this group and stop receiving emails from it, send an email to frank-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Ondrej Hanslik]

Custom predicates

[CMT: ] Sounds useful, but I don’t have a specific use-case for this at present.

[OH:] Neither do I. However, using NSPredicate also means having regular expressions and case&diacritics insensitive comparisons. I think this will be useful for me in the future.

Inspecting views returning from methods 

[CMT: ] I thought Frank could already do stuff like this.  For example:

view:'_UIPopoverView' view:'UILabel' index:0
view:'_UIPopoverView' view:'UITableViewCell' index:3
view:'UITableViewCell' view:'UILabel' marked:’MyFontName' parent view:'UITableViewCell'

[OH:] The use cases were listed only as an example. I am not sure about this myself - it seems too complicated to use.

 

Sorting

[CMT: ] This would be really useful, but its more complicated than this for scrolled tables.  iOS re-allocates index positions on-the-fly as you scroll a table, and keeps more than just the visible UITableViewCell items in memory.  It may even have some duplicated entries that are actually invisible. I’ve had to write some really complicated code that sorts by the Y-address (taking rotation into account) and then filtering by visibility, in order to get a list of UITableViewCells that are in true positional order AND tappable by their index.  It would be nice to have this feature embedded deeper in Frank, and I may be able to contribute my code, if someone would like to re-work my Python into Ruby or Objective-C.

[OH:] I have probably the very same code.

Obj-C implementation would enable smarter sorting by screen coordinates of the view. Checking if a view is tappable was one of the reasons why I started implementing the modifications for  "touch". Now I can select all the tappable cells simply by "tableViewCell FEX_canTouch"

[Dale Emery]

> Custom predicates
>
> [CMT: ] Sounds useful, but I don’t have a specific use-case for this at present.

We use predicates rarely (via the Igor* selector engine), but they are helpful in a few cases. Sometimes, for example:

accessibilityLabel BEGINSWITH 'Monkeyshines'
accessibilityLabel ENDSWITH 'J. Fred Muggs'

And sometimes we want to test two or more properties of a view:

property1 == 'value1' AND property2 == 'value2'

Our most common use is to find a view that matches some property and is also animating (or not animating). I don't remember whether Shelley can do that or not.

I think predicates would make a terrific extension for Shelley.

Dale

*Igor: https://github.com/dhemery/igor
--
Dale Emery
Consultant to software teams and leaders
http://dhemery.com

[Martin Taylor]

I've just come up with a need to select "view marked:'DLG' OR marked:'POP'" (i.e. a view with an accessibilityLabel that begins with the string DLG or the string POP.  Can Shelley do this some way now, or would this fall under one of the proposed extensions?

Thanks,

Martin

[Ondrej Hanslik]

I am pretty sure this can't be done by Shelley now. There is no support for logical operations. Currently you have to do it on the scripting side (find both selectors and check which one exists). With the proposed extensions it could be done by a regular expression.