How to use ObjC structs within frankly_map?

[Alexey Denisov]

UIscrollView setContentOffset:animated:

How can I use it within frankly_map? And how can I send CGPoint
struct?

[Stewart Gleadow]

I've not tried this before, so can't say if there is an in-built way to do structs out of the box. If there isn't, depending on what's picking up the 'map' command on the native side, you can easily encode/decode using provided functions like CGRectFromString and NSStringFromCGRect from UIGeometry.h- it's not generic though.

- Stew

[Dale Emery]

A client and I added a method to UIScrollView like this:

@interface UIScrollView (MYFX)
- (void)MYFX_setContentx:(NSUInteger)x y:(NSUInteger)y animated:(BOOL)animated;
@end

For now, we send x and y separately, rather than trying to send a point. I can't send you the implementation (for now), but you just create a CGPoint from x and y, then call the UIScrollView method.

Soon we'll need to send more complex stuff like CGRect and NSIndexPath, so we'll figure out how to wrap and unwrap those. My guess is that it will work like this:

Objective-C code:
- Write an Objective-C wrapper method (attached via a category to the appropriate view class) that takes a string.
- The string is a JSON encoding of the struct.
- The wrapper method decodes the JSON into an NSDictionary, then uses the contents of the dictionary to initialize the struct.
- The wrapper method invokes the real view method by sending the appropriate view message to self, passing it the struct.
- (If the return value's type is other than string, number, or bool) The wrapper method encodes the return value as JSON, and returns it as a string.

Somewhere in the test code (I use Java. You'll likely use Ruby):
- Write a class that represents the struct.
- Before sending the message, test code encodes the struct as JSON.
- Test code calls frankly_map, giving the name of the wrapper method, with the encoded string as an argument.
- (If necessary) test code decodes the JSON-encoded return value into an appropriate class.

Apropos of nothing: Because my client and I are writing wrappers around Frank, we are almost irresistibly tempted to call our wrapper library "corn dog." (As far as I can tell, your only appropriate response here is to roll your eyes groan.)

Dale

--
Dale Emery
Consultant to software teams and leaders
http://dhemery.com

[kra Larivain]

Just a thought, but there might be a simpler option on the objc that wouldn't involve pulling in a JSON library in a view category.
Most, if not all, of these CF types have string conversion functions, so you could just send up the Cocoa string representation of these structs. By the end of the day, NSIndexPath is merely an array of NSUIntegers (cf -indexPathWithIndexes:length:).

For instance, your objc side would be something like:
- (void) setFrankContentOffset: (NSString *) aContentOffset animated: (BOOL) animated{
CGPoint point = CGPointFromNSString(aContentOffset);
[self setContentOffset: point animated: animated];
}

Your client side would have to encode the structs in the right format, but I guess there's no working around that problem.

a Java implementation of CGPoint would then be something like:
public class CGPoint {
private int x;
private int y;

public String frankRepresentation {
return "{"+x+","+y+"}";
}
}

and map it like:
frankly_map(selector, "setFrankContentOffset:animated:", point.frankRepresentation(), true);

Would be cool if frank could do some automagic type conversion like that, but I don't believe it's possible to pass CF types to performSelector:, Frank would have to drop down to objc_msgSend to do something like that, might not be worth the effort.

/kra

[Dale Emery]

Hi kra,

> Just a thought, but there might be a simpler option on the objc that wouldn't involve pulling in a JSON library in a view category.
> Most, if not all, of these CF types have string conversion functions, so you could just send up the Cocoa string representation of these structs. By the end of the day, NSIndexPath is merely an array of NSUIntegers (cf -indexPathWithIndexes:length:).
>
> For instance, your objc side would be something like:
> - (void) setFrankContentOffset: (NSString *) aContentOffset animated: (BOOL) animated{
> CGPoint point = CGPointFromNSString(aContentOffset);
> [self setContentOffset: point animated: animated];
> }
>
> Your client side would have to encode the structs in the right format, but I guess there's no working around that problem.

I can explore the CF string representations. For now I don't mind pulling in a JSON library, because I can use the one that's already linked into Frank... at least until you talk Pete into separating that into its own library. ;-)

> Would be cool if frank could do some automagic type conversion like that, but I don't believe it's possible to pass CF types to performSelector:, Frank would have to drop down to objc_msgSend to do something like that, might not be worth the effort.

I wonder if it would be worthwhile (or useful) to have Frank delegate the type conversion to a set of pluggable libraries, the way it does with selector engines. In the same way that the Frankly format for queries has a 'selector_engine' field, the format for methods could have a 'type_codec' field. Then folks who are in the mood could add codecs to implement whatever conversions they need.

I can think of two ways to manage the type information, neither of which seem any damned fun:

1. The codec has a dictionary from Objective-C method signatures to parameter types. When asked to convert, the codec looks up the signature, decodes the Frankly argument strings into the types specified by the signature, and hands the values to Frank to make the call.

2. The codec has a dictionary from type names to Objective-C types. Test code sends the type name through frankly along with each argument. The codec looks up the type name, translates the Frankly argument string to the appropriate value, and hands the value to Frank to make the call.

Similarly, each codec could encode the return types.

With the first option, the codec would have to have a dictionary entry for every method you want to call. Yuck.

With the second option, the codec would have a dictionary entry for each class. That seems not so bad to me. More unfortunate is that you would need to encode the type information into the Frankly messages. I don't know whether that's horrendous or not.

Some of this is easier for the CG types, because (if I understand correctly) they're already encoded into method signatures. Still you'd need a map from CG type to CGMake...() function and NSStringFrom...() function.

Bleah.

Dale

[Peter Hodgson]

It sounds like you're heading towards creating an RPC framework for iOS. I'm not sure how to feel about that, to be honest. 

My sense is that methods you're exposing for automation shouldn't need complex types. Really I think the standard JSON primitives plus points, rects and colors should be all that's needed. If we're feeling a need for a more sophisticated RPC mechanism then I'd say we're not exposing the right automation methods.

[Dale Emery]

Hi Pete,

> It sounds like you're heading towards creating an RPC framework for iOS.

I hope not "heading towards." Only "noting in case the need arises."

> My sense is that methods you're exposing for automation shouldn't need complex types. Really I think the standard JSON primitives plus points, rects and colors should be all that's needed.

I hope that's true.

> If we're feeling a need for a more sophisticated RPC mechanism then I'd say we're not exposing the right automation methods.

I unfortunately bumped into a limitation right away. The third thing I wanted to automate (after tapping a button and determining whether the app displayed the right page) was to enter some text into a text field. I quickly discovered that "setText:" was not enough. It bypasses some of the text field's normal behavior.

When the user types keys or uses the popup menu to paste or delete text, the text field asks its delegate to approve the change. If the delegate approves, the text field sets its text property and notifies other observers of the change.

When asked to approve a change, the delegate may take actions in addition to approving or denying. One common behavior is to offer completion suggestions.

As far as I can tell, Frank currently offers no direct way to provoke a text field's full normal behavior, and therefore no direct way to exercise the delegate's responsibilities.

So I think there are some additional common automation methods to implement.

Beyond that... I'm coming to see the wisdom of your caution. I've been wanting to use Frank not only to simulate user actions and observations, but also to sneak around the user interface to do other things (for example, to establish test conditions more directly and quickly, rather than simulating all of the user's actions).

As I write this, I realize: Frank's job is to allow me to interact through the UI, not to help me sneak around it. I think you're wise to keep Frank focused on that. Thanks for reeling me in.

[Taylor, Martin]

Hi Pete & Dale,

I'd like to add my $0.02 worth to this discussion, in particular my comments on Dale's final comment below:

" As I write this, I realize: Frank's job is to allow me to interact through the UI, not to help me sneak around it. I think you're wise to keep Frank focused on that. Thanks for reeling me in."

Frank inherently provides a "sneak around the UI" method in "app_exec". I plan to use this to call a whole set of "Testability APIs" that have been built into our app to allow ANY test automation framework to query the state of the application.

I would LOVE to be able to use Frank to interact exclusively through the UI, and use "app_exec" to check the resulting app state, but Frank is not sufficiently powerful (yet?) to let me do this. The big problem here is Frank's inability to see (via Symbiote) and interact with (e.g. touch) iOS keypads. Our app defines MANY custom keypads and so far I haven't been able to interact with ANY of these via Frank. My fallback position is to use "app_exec" to call the underlying method in our app that gets called when a keypad key is pressed. I'm also doing all kinds of experimentation and research to find out if there's ANY WAY that Frank could be made to recognize and interact with Keypads. If anyone knows a definitive answer on this (possible or not) I'd appreciate hearing it.

This blog post (http://www.andrashatvani.com/2011/05/uiakeyboard-keys-and-buttons.html) shows how keypad keys can be tapped using JavaScript and Apple's Instruments approach to test automation:
UIATarget.localTarget().frontMostApp().keyboard().keys()["e"].tap();
I've been trying to find something similar in Xcode, but [UIApplication sharedApplication] doesn't seem to have any access method for "keyboard". Is there some way with Frank to execute a fragment of JavaScript on the target app? Then maybe we could use JavaScript like the above directly if it can't be done from Objective-C.

I'll continue experimenting with Frank this week and next while the rest of my team does similar experiments with MonkeyTalk (http://www.gorillalogic.com/testing-tools/monkeytalk). After 2 weeks, we'll determine which tool is more suited to our needs.

Thanks,
Martin

[Pete Hodgson]

Frank uses KIF as the low level interaction simulator, and KIF does allow you to interact with the iOS keyboard I believe. If you want to see how tricky this is, check out their code. It's pretty gnarly.

I'm guessing it wouldnt be too much work to expose this stuff in Frank, but I haven't looked into it much. It could probably be hooked up via app_exec or alternatively I'd be happy to accept a patch which adds an extra API endpoint for driving the keyboard.

Cheers,
Pete
--
Typed on a little bitty keyboard

[Brian King]

There is a function in KIFTestStep enterCharacter: which would be
great to get bundled into frank. It will shift through keyboards and
such until it finds what it needs. So one could type in a long
string without thinking about hitting the shift keyboard or numeric
keyboard. I used to use a custom Frank build where I pulled out KIF,
and included KIF by hand and used this functionality.

It sounds like this isn't Martin's issue though, most of your
keyboards are custom? They should be tappable through Frank. I
forget if the UISpec stuff defaults to keyWindow or uses all windows.
if you query the keyWindow, you very often will not catch the
keyboard window if you use the UIResponder inputView form, it will be
in a separate window, often named 'UITextEffectsWindow'.


Brian