Automation test against a physical device

[cansaner]

Pete Hodgson just told me "Yes, it is possible to run Frank tests
against a physical device." on his reply to my comment on his blog.
Is there anybody that actually tried to test any application against a
physical device before that can help me try this? For now I even
couldn't understand howto set the environment variable for bundle path
to show the application working on IPAD :( And I can't find any
documentation about howto set Frank to test an application on a
physical device. Any help provided is greatly appreciated! :)

[Stewart Gleadow]

I have spiked out running Frank on a physical device, and had some success, although I've never got it to a point where it was stable enough to run as part of a CI build.

I hardcoded the IP address on my iPhone on a local network, although Nathan De Vries tried to get this dynamic using using bonjour discovery in a quick fork. https://github.com/atnan/Frank

Last time I looked, the frank helper inside the frank-cucumber gem talked to localhost so I was running directly from the ruby source of the gem in the Frank repo... although recent tweets from Pete indicate maybe there's been a change?

Also, if you want to use the accessibility labels, you'll need to enable VoiceOver on the phone, which you can do in the Settings app but it makes the phone a bit hard to use manually I found. You can enable voice over on the phone through iTunes when your device is plugged in.

- Stew

[Can Saner]

I really didn't expect such a quick response. I will just try out do it using Nathan's approach now. Thank you very much for this very quick orientation :)

CS

[Pete Hodgson]

This is a creepy coincidence. I was just working on getting Frank to work with a physical device last night too. And, turns out I took pretty much the exact same approach as Nathan! My changes are on a branch, but I will probably merge them into master soon since I don't think they have any impact on testing with the simulator. Here's the branch: https://github.com/moredip/Frank/tree/bonjour

[Stewart Gleadow]

Pete,

I couldn't get it connecting every time... there was some magical combination of enabling VoiceOver (through iTunes), cleaning, building, running up the Frankified target (through Xcode) and checked in Symbiote... but it wasn't every time. Sometimes, I just couldn't get it to connect. Once it worked, I got a good hour or so when it would connect pretty reliable.

It's awesome watching a phone sitting on your desk moving, and green cucumber tests in your terminal.

Any solutions for automatically booting the app after an xcodebuild CI compile or are we back to Applescripting Xcode?

- Stew

[Chris Stevenson]

Pete - you guys rock.

Stewart - Pete and I spiked this a while back and I did a bit of research. I could not find a way of starting the app without using Xcode. And as far as applescripting it the AppleScript apis are not complete in Xcode 4 so you have to use 3.X. As I'm on Lion I couldn't install that safely so that's where I stopped.

Pete - did you solve that or did you have to use 3.X?

Chris

[Markus]

Ah, the joy of automating an automation system! I'll chime in with my experience with trying to automate an instruments setup, and I apologize for the hijack!

With Xcode 3.2.5 I had an applescript that issued an xcodebuild command, and when complete it would open the project, this would select the correct target and then I could use the "debug" option to run the pre-compiled app, which would result in it being copied to the simulator. I then had the script modifiy a pre-existing instruments trace file to change the path to the newly installed app, otherwise instruments would be talking to the wrong target. This wasn't something I could try with a device, as I could not access the location of the app on the device, well, not unless I had jailbroken the app.

With Xcode 4.0.x, as mentioned previously, the applescript API's aren't complete, so I had to re-code things. What I now do  is to use xcodebuild to build the file, then I clear out the Applications folder in the appropriate version of the iPhone Simulator folder (~/Library/Application Support/iPhone Simulator/<version>/Applications/) and create a new folder in there and then copy the dot app file into that folder. Again I then modify an instruments trace file to update the path to the app. Both of these steps cannot be done against a device unless I jailbreak it, and to be honest, the device side of things is actually an assumption as I've not tried it, but I figure I'd need somehow access the file system on the device to copy it over, probably have to install ssh and and scp and then scp the file over.

When I get round to having a proper look at Frank, it will possibly make things slightly easier as I would not be messing around with having to modifiy instrument templates, I'd just need to somehow tell it where the app is located, which I think is simply a case of setting an environment variable, but obviously I'd still need to get the location to the file.

Sorry for waffling on, and hopefully this info might be helpful/informative.

[Pete Hodgson]

OK, so I've been mulling over various approaches to getting this app-restart-on-a-physical-device nut cracked. Here's the best (and wackiest) idea I've come up with so far. I'd love to hear from you all as to whether this sounds feasible or not.

We create a separate 'relauncher' app which just sits in background mode waiting for a command (presumably over http). That app is launched manually on the device one time, and then just put into the background and left running forever (if that's possible?).

Then, for every Frank test scenario:

  - find any existing instance of the frankified app via bonjour, and ask it to kill itself

  - find the relauncher app via bonjour, and ask it to launch the frankified app (using a custom URL schema I guess? Or maybe a private API?)

  - wait for the frankified app to announce itself on bonjour

  - run yer tests

  - rinse and repeat

I /think/ that this approach might actually work, but at the same time I've done no work with background apps or custom URL schemas, so I'm not really sure. I would love to hear opinions from those that have experience in either.

Cheers,

Pete

[Stewart Gleadow]

Your approach seems like it could possibly work, although I'm not sure of the reliability of the background app receiving the messages you send it?

I'm interested in getting this working mainly because it's a challenge. It's still a battle just to get full test coverage via the simulator, and most (not all) bugs that can be caught by the simulator... so in work hours, I'm not looking at the device at all.

I want to hit up the Apple engineers in the labs at WWDC and try and find out how they drive the app on the device over USB. I hear it's pretty low level and not allowed by us though!

- Stew

[Chris Stevenson]

Hey Stewart,

I'll be there too and I'd love to be a part of that discussion. It would be great to get some testing contacts inside Apple.

Chris

[Stewart Gleadow]

Oh yeah, you're going to have to show me around you realise, haven't been to SF for 17 years.

Everyone recommends having a sample application ready to play with when you go to the labs... maybe a small app with some Frank tests so we can bug them on how to run it... and also some sample UIAutomation tests to see if they can be run without Instruments in any way!

- Stew

[Pete Hodgson]

Stew, I'm exactly the same - I don't really see testing on the device as being *that* valuable, but it is a fun challenge and a cool way to demonstrate Frank!

And yes, absolutely agree that it would be great to have some contact with folks inside Apple. Regarding a sample app, I'm working at the moment on getting a really simple demo test set up using the CountItOut example app which I recently added to the repo. It will be able to run on a physical device. It will also be ready for WWDC - I am getting it ready for a demo at a ThoughtWorks event the week before. Hopefully we can meet up before WWDC and conspire!

[Rodney Degracia]

I need to test the camera aspect of my app, so I am totally fine with launching the App on the device manually from Xcode's Run button and running just a single feature test.

Is there a way to find out the tags or accessibility labels that are assigned by Apple to the controls within the Camera's UIImagePickerController interface so that I can use Frank to take a picture and choose the picture?   

Rodney Degracia

rdeg...@gmail.com

http://www.elitefrontier.org

[zhc...@gmail.com]

You should be able to use Symbiote to find out the control hierarchy. Just point it to the bonjour IP address of the device.

chris

[Can Saner]

Hello again,

I just checked out bonjour brach and tried to run CountItOut example as it is tried to be run against real device. When I try to run test cases it always gives me "uninitialized constant Frank::USING_PHYSICAL_DEVICE (NameError)" and I don't understand why. As far as I understand, USING_PHYSICAL_DEVICE constant should be set false when there is not any environment variable set or any variable USING_PHYSICAL_DEVICE is defined from this code snippet:

module Frank

USING_PHYSICAL_DEVICE = !!ENV['USING_PHYSICAL_DEVICE'] ||

( defined?(::USING_PHYSICAL_DEVICE) && !!::USING_PHYSICAL_DEVICE ) ||

false

But still It says that USING_PHYSICAL_DEVICE constant is uninitialized and I really can't find the reason. I set USING_PHYSICAL_DEVICE environment variable or tried to write it in env.rb to define it as ruby variable but I can't run test cases against neither physical device nor simulator.
Can you please help me if you see anything that I miss?

Detailed error trace is this:
*****************************************************************************************************************************
Yelo-Tests-MacBook-Pro:CountItOut yelotest$ pwd
/Users/yelotest/Desktop/frank/example/CountItOut
Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber
Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5
      uninitialized constant Frank::USING_PHYSICAL_DEVICE (NameError)
      ./features/step_definitions/launch_steps.rb:2:in `running_on_device'
      ./features/step_definitions/launch_steps.rb:7:in `/^the app is launched$/'
      features/main.feature:4:in `Given the app is launched'

  Scenario: Zeroing out counts  # features/main.feature:6
    Given I touch "Zero Counts" # frank-cucumber-0.6.1/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.1/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1
    And I touch "Bites"         # frank-cucumber-0.6.1/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.1/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1
    And I touch "Bites"         # frank-cucumber-0.6.1/lib/frank-cucumber/core_frank_steps.rb:178

Failing Scenarios:
cucumber features/main.feature:6 # Scenario: Zeroing out counts

1 scenario (1 failed)
8 steps (1 failed, 7 skipped)
0m0.004s
Yelo-Tests-MacBook-Pro:CountItOut yelotest$
*****************************************************************************************************************************

Thanks in advance,
CS

[Can Saner]

Hello again,

I passed USING_PHYSICAL_DEVICE error, i initialized it in features/support/env.rb and than finally found the application that works on iPAD no matter I open VoiceOver or not but only with running up the Frankified Count It Out target through XCode.
Than I think I come to the same position that Stewart Gleadow told here, I just can't get it connecting every time...

When runned it first time, I get:
Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber features/main.feature
finding Frank server via Bonjour...
...found Frank via Bonjour: 10.138.48.79

Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5

  Scenario: Zeroing out counts  # features/main.feature:6

    Given I touch "Zero Counts" # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

1 scenario (1 passed)
8 steps (8 passed)
0m24.701s

And most importantly, although I touched 3 "Bites", I just had 1 touch for "Bites" on application running on device, and than it just stopped working and neither "Zero Counts" nor "Bites" touches didn't happen on the application working on iPAD but test steps passed on the terminal that means Frank server returned results...

But than when I stop running and restart it, I get:
Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber features/main.feature
finding Frank server via Bonjour...
...found Frank via Bonjour: 10.138.48.79

Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5

  Scenario: Zeroing out counts  # features/main.feature:6

    Given I touch "Zero Counts" # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
      Could not touch [Zero Counts], it does not exist. (RuntimeError)
      features/main.feature:7:in `Given I touch "Zero Counts"'
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

Failing Scenarios:
cucumber features/main.feature:6 # Scenario: Zeroing out counts

1 scenario (1 failed)

8 steps (1 failed, 6 skipped, 1 passed)
0m11.717s

and XCode logs like:
[Switching to process 11779 thread 0x0]
sharedlibrary apply-load-rules all
2011-09-15 13:29:33.570 CountItOut[171:707] Creating the server: <HTTPServer: 0x12e340>
2011-09-15 13:29:33.580 CountItOut[171:707] Started HTTP server on port 37265
2011-09-15 13:29:34.297 CountItOut[171:707] Bonjour Service Published: domain(local.) type(_http._tcp.) name(Frank UISpec server)
2011-09-15 13:29:51.543 CountItOut[171:707] UISPEC WARNING: UIView doesn't exist at index 0
2011-09-15 13:29:51.548 CountItOut[171:707] returning:
{"results":[],"outcome":"SUCCESS"}

Than when I run test case again without restarting application, test steps passes again:
Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber features/main.feature
finding Frank server via Bonjour...
...found Frank via Bonjour: 10.138.48.79

Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5

  Scenario: Zeroing out counts  # features/main.feature:6

    Given I touch "Zero Counts" # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

1 scenario (1 passed)
8 steps (8 passed)
0m20.872s

XCode logs:
2011-09-15 13:30:02.704 CountItOut[171:707] returning:
{"results":[null],"outcome":"SUCCESS"}
2011-09-15 13:30:05.285 CountItOut[171:707] returning:
{"results":["<COMPLEX TYPE>"],"outcome":"SUCCESS"}
2011-09-15 13:30:06.846 CountItOut[171:707] returning:
{"results":[null],"outcome":"SUCCESS"}
2011-09-15 13:30:07.838 CountItOut[171:707] returning:
{"results":["<COMPLEX TYPE>"],"outcome":"SUCCESS"}
2011-09-15 13:30:10.710 CountItOut[171:707] returning:
{"results":[null],"outcome":"SUCCESS"}
2011-09-15 13:30:11.775 CountItOut[171:707] returning:
{"results":["<COMPLEX TYPE>"],"outcome":"SUCCESS"}
2011-09-15 13:30:15.339 CountItOut[171:707] returning:
{"results":[null],"outcome":"SUCCESS"}
2011-09-15 13:30:16.389 CountItOut[171:707] returning:
{"results":["<COMPLEX TYPE>"],"outcome":"SUCCESS"}
2011-09-15 13:30:20.439 CountItOut[171:707] returning:
{"results":[null],"outcome":"SUCCESS"}
2011-09-15 13:30:21.451 CountItOut[171:707] returning:
{"results":["<COMPLEX TYPE>"],"outcome":"SUCCESS"}

I just don't understand why sometimes test steps fails and than they pass on the same running application. And I really don't understand why I don't see any touch event happening actually on iPAD although i get responses from Frank server and test steps pass.
 
And I wonder if there is any improvement on the matter of testing application against a real device? Is that simple demo test set up using CountITOut example app the one that resides in /example/CountItOut within bonjour branch or is there any other test set up that I miss in repository? I just know that Pete prepares a set up for WWDC but probably I miss it in repository. Is there any other improvement on this matter?

Thanks again
CS

Regarding a sample app, I'm working at the moment on
> > getting a really simple demo test set up using the CountItOut example app
> > which I recently added to the repo. It will be able to run on a physical
> > device. It will also be ready for WWDC - I am getting it ready for a demo at
> > a ThoughtWorks event the week before. Hopefully we can meet up before WWDC
> > and conspire!

On Thu, Aug 18, 2011 at 2:51 AM, Abhishek Banerjee <abhi...@gmail.com> wrote:

On Jun 8, 5:01 am, Can Saner <cansa...@gmail.com> wrote:
> Hello again,
>
> I just checked out bonjour brach and tried to run CountItOut example as it
> is tried to be run against real device. When I try to run test cases it
> always gives me "uninitialized constant Frank::USING_PHYSICAL_DEVICE
> (NameError)" and I don't understand why. As far as I understand,
> USING_PHYSICAL_DEVICE constant should be set false when there is not any
> environment variable set or any variable USING_PHYSICAL_DEVICE is defined
> from this code snippet:
>
> module Frank
>
>     USING_PHYSICAL_DEVICE = !!ENV['USING_PHYSICAL_DEVICE'] ||
>
>       ( defined?(::USING_PHYSICAL_DEVICE) && !!::USING_PHYSICAL_DEVICE ) ||
>
>       false
>
> But still It says that USING_PHYSICAL_DEVICE constant is uninitialized and I
> really can't find the reason. I set USING_PHYSICAL_DEVICE environment
> variable or tried to write it in env.rb to define it as ruby variable but I
> can't run test cases against neither physical device nor simulator.
> Can you please help me if you see anything that I miss?
>
> Detailed error trace is this:

> ***************************************************************************­**************************************************

> ***************************************************************************­**************************************************
>
> Thanks in advance,
> CS
>

> On Fri, May 20, 2011 at 6:59 AM, Pete Hodgson <phodg...@thoughtworks.com>wrote:

>
>
>
> > Stew, I'm exactly the same - I don't really see testing on the device as
> > being *that* valuable, but it is a fun challenge and a cool way to
> > demonstrate Frank!
>
> > And yes, absolutely agree that it would be great to have some contact with
> > folks inside Apple. Regarding a sample app, I'm working at the moment on
> > getting a really simple demo test set up using the CountItOut example app
> > which I recently added to the repo. It will be able to run on a physical
> > device. It will also be ready for WWDC - I am getting it ready for a demo at
> > a ThoughtWorks event the week before. Hopefully we can meet up before WWDC
> > and conspire!
>

> > On Thu, May 19, 2011 at 4:16 PM, Stewart Gleadow <sglea...@gmail.com>wrote:

>
> >> Oh yeah, you're going to have to show me around you realise, haven't been
> >> to SF for 17 years.
>
> >> Everyone recommends having a sample application ready to play with when
> >> you go to the labs... maybe a small app with some Frank tests so we can bug
> >> them on how to run it... and also some sample UIAutomation tests to see if
> >> they can be run without Instruments in any way!
>
> >> - Stew
>
> >> On 20/05/2011, at 9:08 AM, Chris Stevenson wrote:
>
> >> Hey Stewart,
>
> >> I'll be there too and I'd love to be a part of that discussion. It would
> >> be great to get some testing contacts inside Apple.
>
> >> Chris
>

> >>>> On May 18, 2011, at 5:18, Stewart Gleadow < <sglea...@gmail.com>

> >>>> sglea...@gmail.com> wrote:
>
> >>>> Pete,
>
> >>>> I couldn't get it connecting every time... there was some magical
> >>>> combination of enabling VoiceOver (through iTunes), cleaning, building,
> >>>> running up the Frankified target (through Xcode) and checked in Symbiote...
> >>>> but it wasn't every time. Sometimes, I just couldn't get it to connect. Once
> >>>> it worked, I got a good hour or so when it would connect pretty reliable.
>
> >>>> It's awesome watching a phone sitting on your desk moving, and green
> >>>> cucumber tests in your terminal.
>
> >>>> Any solutions for automatically booting the app after an xcodebuild CI
> >>>> compile or are we back to Applescripting Xcode?
>
> >>>> - Stew
>
> >>>> On 18/05/2011, at 9:39 PM, Pete Hodgson wrote:
>
> >>>> This is a creepy coincidence. I was just working on getting Frank to
>

> ...
>
> read more »- Hide quoted text -
>
> - Show quoted text -

[Gazi]

Hi,

It will be really good if you can attach your demo video to the Frank
home page. Also if you can put some documentation like step by step
methods to run the tests on the device about inner app purchases etc..


thanks,
Gazi

On Sep 15, 11:59 am, Can Saner <cansa...@gmail.com> wrote:
> Hello again,
>

[Pete Hodgson]

Hi Can,

Thanks for the detailed report. I'm bummed that you're seeing these
intermittent issues trying to run tests against a physical device. I
saw similar behavior when I very briefly experimented with this stuff
a few months back. I never got a chance to investigate further - there
just isn't that much need for automating a physical device on the iOS
project we're working on.

That said, I would love to get more info on what exactly is going on
with these intermittent failures. Has anyone else on the list done any
work with getting Frank to work with a physical device?

Cheers,
Pete

On Sep 15, 3:59 am, Can Saner <cansa...@gmail.com> wrote:
> Hello again,
>

> ...
>
> read more »

[Can Saner]

Hello Pete,

I continue to investigate the problem but I have some problems here. Maybe you know the solutions for them or you can show me a better way to investigate.

Situation:
I am trying to make black box testing for an application that is actually integrated with Frank and the only communication with it is Frank right now.
I don't have source code of the application.
I have a simulator version and a device version(ad-hoc) of the application. I can browse the application DOM with simulator version
So the only thing that I have is the bonjour service published.
Problem:
For the device:
1- I can't browse DOM with device version, dunno why i can't i am trying to understand why http://<device_ip>/37265 doesn't work although I can communicate application with cucumber. Can you guess anything about the problem?
2- I can't reach NSLogs of the application when it runs on device NOT through XCode. (Because i don't have source code and I only have ad-hoc distribution of the application). Do you know any mechanism to get these logs when application runs?

Thanks,
CS

[Can Saner]

Because I am idiot I just typed http://<device_ip>/37265 instead of http://<device_ip>:37265 to the browser and that's why it didn't gave me response.
After I added debug lines to the frank_helper.rb file to simulate NSLogs of application on the host computer, I just saw that I made this mistake!!! :)
Now for the problems:
1- I can browse DOM with device version with Chrome and Safari now using http://<device_ip>:37265 but sometimes it doesn't reply to the http request like the requests of Cucumber. When you kill application and restart it, you can browse it but it is veeeeeeeeeeeeeerrryyy slow and I don't know why but I can't see Accessible Elements through browser when I open application DOM on Symbiote web interface. Also Flash function of Symbiote doesn't work when application runs on device. You can only see hierarchy and properties.
2- I just added some debug lines to the frank_helper.rb like this:
  def debug string
    puts string if $DEBUG
  end

  #taken from Ian Dee's Encumber
  def post_to_uispec_server( verb, command_hash )
    url = frank_url_for( verb )
    req = Net::HTTP::Post.new url.path
    req.body = command_hash.to_json

    debug url
    debug req
    debug req.body
    make_http_request( url, req )
  end

  def make_http_request( url, req )
    http = Net::HTTP.new(url.host, url.port)

    res = http.start do |sess|
      sess.request req
    end

    debug res
    res.body
  end

And got same results when I try to run tests on device but now I have some information about requests and responses:

Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber features/main.feature
finding Frank server via Bonjour...

Exception `LoadError' at /Users/yelotest/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:54 - no such file to load -- dnssd
got a reply
got a reply
#<DNSSD::Reply::Browse:0x80d041f4 "Frank\\032UISpec\\032server._http._tcp.local." interface: en0 flags: #<DNSSD::Flags add>>
Frank UISpec server on Users-iPad.local.:37265
first address for Users-iPad.local. is 10.138.48.79
OK WE HAVE AN ADDRESS: 10.138.48.79

...found Frank via Bonjour: 10.138.48.79

Exception `NameError' at /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cucumber/constantize.rb:12 - uninitialized constant Cucumber::Formatter::Pretty

Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5

  Scenario: Zeroing out counts  # features/main.feature:6

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b928f8>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Zero Counts' first"}
#<Net::HTTPOK:0x101b90300>
{"results":[],"outcome":"SUCCESS"}
Exception `RuntimeError' at /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:183 - Could not touch [Zero Counts], it does not exist.
Exception `RuntimeError' at /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cucumber/core_ext/instance_exec.rb:73 - Could not touch [Zero Counts], it does not exist.

    Given I touch "Zero Counts" # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
      Could not touch [Zero Counts], it does not exist. (RuntimeError)
      features/main.feature:7:in `Given I touch "Zero Counts"'
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

Failing Scenarios:
cucumber features/main.feature:6 # Scenario: Zeroing out counts

1 scenario (1 failed)
8 steps (1 failed, 6 skipped, 1 passed)

0m16.685s

Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber features/main.feature
finding Frank server via Bonjour...

Exception `LoadError' at /Users/yelotest/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:54 - no such file to load -- dnssd
got a reply
got a reply
#<DNSSD::Reply::Browse:0x80d041f4 "Frank\\032UISpec\\032server._http._tcp.local." interface: en0 flags: #<DNSSD::Flags add>>
Frank UISpec server on Users-iPad.local.:37265
first address for Users-iPad.local. is 10.138.48.79
OK WE HAVE AN ADDRESS: 10.138.48.79

...found Frank via Bonjour: 10.138.48.79

Exception `NameError' at /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cucumber/constantize.rb:12 - uninitialized constant Cucumber::Formatter::Pretty

Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5

  Scenario: Zeroing out counts  # features/main.feature:6

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b928f8>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Zero Counts' first"}
#<Net::HTTPOK:0x101b90058>

{"results":[null],"outcome":"SUCCESS"}

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b8fc70>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Zero Counts' first"}
#<Net::HTTPOK:0x101b8ae78>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}

    Given I touch "Zero Counts" # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b78660>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b6f0d8>

{"results":[null],"outcome":"SUCCESS"}

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b6e368>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b6bb90>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b632d8>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b5f840>

{"results":[null],"outcome":"SUCCESS"}

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b5f480>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b5cc80>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b56830>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b50688>

{"results":[null],"outcome":"SUCCESS"}

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b4faa8>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b4a5d0>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b31990>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b2db10>

{"results":[null],"outcome":"SUCCESS"}

http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b2d750>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b2b220>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}

    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

1 scenario (1 passed)
8 steps (8 passed)

1m13.757s

Again for the first test case, it fails on the Zero Counts step and when another more test case run is done, test steps passes
Still I don't see any touch event on the device application, and I don't know why?

Is it because of network is very slow or lack of RAM or something i don't see any touch event on application on device?

CS

[Can Saner]

Hello Pete,

I don't know why but when I connect to Frank web interface for the application running on device, It doesn't give any accessibility label for any view of the application. All the accessibilityLabel properties of views are null when I check Symbiote. Can it be the reason for instability of the test cases for the application running on device? I can't use flash function of the symbiote as well.

CS

[Pete Hodgson]

Do you have VoiceOver turned on? You need to do that in order to have
iOS on the device generate accessibility information which Frank uses
to automate the app.

> > /Users/yelotest/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/site_ruby/1.8/rubygems /custom_require.rb:54

> > - no such file to load -- dnssd
> > got a reply
> > got a reply
> > #<DNSSD::Reply::Browse:0x80d041f4
> > "Frank\\032UISpec\\032server._http._tcp.local." interface: en0 flags:
> > #<DNSSD::Flags add>>
> > Frank UISpec server on Users-iPad.local.:37265
> > first address for Users-iPad.local. is 10.138.48.79
> > OK WE HAVE AN ADDRESS: 10.138.48.79
>
> > ...found Frank via Bonjour: 10.138.48.79
> > Exception `NameError' at

> > /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cu cumber/constantize.rb:12

> > - uninitialized constant Cucumber::Formatter::Pretty
>
> > Feature: Demonstrating using Frank to test the CountItOut example app
>
> >   Background:                 # features/main.feature:3
> >     Given the app is launched # features/step_definitions/launch_steps.rb:5
>
> >   Scenario: Zeroing out counts  # features/main.feature:6
> >http://10.138.48.79:37265/map
> > #<Net::HTTP::Post:0x101b928f8>
> > {"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"v iew
> > marked:'Zero Counts' first"}
> > #<Net::HTTPOK:0x101b90300>
> > {"results":[],"outcome":"SUCCESS"}
> > Exception `RuntimeError' at

> > /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/frank-cucumber-0.6.2/lib/fra nk-cucumber/core_frank_steps.rb:183

> > - Could not touch [Zero Counts], it does not exist.
> > Exception `RuntimeError' at

> > /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cu cumber/core_ext/instance_exec.rb:73

> > /Users/yelotest/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/site_ruby/1.8/rubygems /custom_require.rb:54

> > - no such file to load -- dnssd
> > got a reply
> > got a reply
> > #<DNSSD::Reply::Browse:0x80d041f4
> > "Frank\\032UISpec\\032server._http._tcp.local." interface: en0 flags:
> > #<DNSSD::Flags add>>
> > Frank UISpec server on Users-iPad.local.:37265
> > first address for Users-iPad.local. is 10.138.48.79
> > OK WE HAVE AN ADDRESS: 10.138.48.79
>
> > ...found Frank via Bonjour: 10.138.48.79
> > Exception `NameError' at

> > /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cu cumber/constantize.rb:12

> > - uninitialized constant Cucumber::Formatter::Pretty
>
> > Feature: Demonstrating using Frank to test the CountItOut example app
>
> >   Background:                 # features/main.feature:3
> >     Given the app is launched # features/step_definitions/launch_steps.rb:5
>
> >   Scenario: Zeroing out counts  # features/main.feature:6
> >http://10.138.48.79:37265/map
> > #<Net::HTTP::Post:0x101b928f8>
> > {"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"v iew

> > On Fri, Sep 16, 2011 at 2:39 PM, Can Saner <cansa...@gmail.com> wrote:
>
> >> Hello Pete,
>
> >> I continue to investigate the problem but I have some problems here. Maybe
> >> you know the solutions for them or you can show me a better way to
> >> investigate.
>

> >> *Situation:*

> >> I am trying to make black box testing for an application that is actually
> >> integrated with Frank and the only communication with it is Frank right now.
> >> I don't have source code of the application.
> >> I have a simulator version and a device version(ad-hoc) of the
> >> application. I can browse the application DOM with simulator
>

> ...
>
> read more »

[Can Saner]

Wooooooooooohoooooooooooooooooooooooooo!
When I turn VoiceOver on everything just worked!!! :))))
All accessibilityLabels comes to the web interface, Flash feature is working and test cases run without problem...
Damn it why cases were passing on the second run when VoiceOver was turned off?? That's the reason I couldn't guess it...

Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber features/main.feature
finding Frank server via Bonjour...

Exception `LoadError' at /Users/yelotest/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:54 - no such file to load -- dnssd

got a reply
got a reply
#<DNSSD::Reply::Browse:0x80d041f4 "Frank\\032UISpec\\032server._http._tcp.local." interface: en0 flags: #<DNSSD::Flags add>>
Frank UISpec server on Users-iPad.local.:37265
first address for Users-iPad.local. is 10.138.48.79
OK WE HAVE AN ADDRESS: 10.138.48.79
...found Frank via Bonjour: 10.138.48.79

Exception `NameError' at /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cucumber/constantize.rb:12 - uninitialized constant Cucumber::Formatter::Pretty

Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5

  Scenario: Zeroing out counts  # features/main.feature:6
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b928f8>

{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Zero Counts' first"}

#<Net::HTTPOK:0x101b90058>
{"results":[null],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b8fc70>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Zero Counts' first"}
#<Net::HTTPOK:0x101b8ae78>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    Given I touch "Zero Counts" # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b78660>

{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b6f0d8>
{"results":[null],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b6e368>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b6bb90>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b632d8>

{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b5f840>
{"results":[null],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b5f480>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b5cc80>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b56830>

{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b50688>
{"results":[null],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b4faa8>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b4a5d0>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b31990>

{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b2db10>
{"results":[null],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b2d750>
{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b2b220>
{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

1 scenario (1 passed)
8 steps (8 passed)
1m13.757s

Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber features/main.feature
finding Frank server via Bonjour...

Exception `LoadError' at /Users/yelotest/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:54 - no such file to load -- dnssd
got a reply
#<DNSSD::Reply::Browse:0x80d04618 "Frank\\032UISpec\\032server._http._tcp.local." interface: en0 flags: #<DNSSD::Flags add>>

Frank UISpec server on Users-iPad.local.:37265
first address for Users-iPad.local. is 10.138.48.79
OK WE HAVE AN ADDRESS: 10.138.48.79
...found Frank via Bonjour: 10.138.48.79

Exception `NameError' at /Users/yelotest/.rvm/gems/ruby-1.8.7-p352/gems/cucumber-0.9.2/bin/../lib/cucumber/constantize.rb:12 - uninitialized constant Cucumber::Formatter::Pretty

Feature: Demonstrating using Frank to test the CountItOut example app

  Background:                 # features/main.feature:3
    Given the app is launched # features/step_definitions/launch_steps.rb:5

  Scenario: Zeroing out counts  # features/main.feature:6
http://10.138.48.79:37265/map

#<Net::HTTP::Post:0x101b92a10>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Zero Counts' first"}
#<Net::HTTPOK:0x101b90418>
{"results":["Zero Counts"],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b90008>

{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Zero Counts' first"}

#<Net::HTTPOK:0x101b8d8d0>

{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    Given I touch "Zero Counts" # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
http://10.138.48.79:37265/map

#<Net::HTTP::Post:0x101b79da8>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b70528>
{"results":["Bites"],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b6f808>

{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b6c1a8>

{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 1 seconds        # features/step_definitions/frank_steps.rb:1
http://10.138.48.79:37265/map

#<Net::HTTP::Post:0x101b638f0>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b5fe58>
{"results":["Bites"],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b5fa70>

{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b5d270>

{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
http://10.138.48.79:37265/map

#<Net::HTTP::Post:0x101b56ec0>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b50e58>
{"results":["Bites"],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map
#<Net::HTTP::Post:0x101b50548>

{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b4adc8>

{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178
    And I wait 2 seconds        # features/step_definitions/frank_steps.rb:1
http://10.138.48.79:37265/map

#<Net::HTTP::Post:0x101b32188>
{"operation":{"method_name":"accessibilityLabel","arguments":[]},"query":"view marked:'Bites' first"}
#<Net::HTTPOK:0x101b2de30>

{"results":[null],"outcome":"SUCCESS"}
http://10.138.48.79:37265/map

#<Net::HTTP::Post:0x101b2da70>

{"operation":{"method_name":"touch","arguments":[]},"query":"view marked:'Bites' first"}

#<Net::HTTPOK:0x101b2b298>

{"results":["<UIRedoer>"],"outcome":"SUCCESS"}
    And I touch "Bites"         # frank-cucumber-0.6.2/lib/frank-cucumber/core_frank_steps.rb:178

1 scenario (1 passed)
8 steps (8 passed)

0m33.920s

Thanks Pete...
Cases against real device are running but god damn VoiceOver sucks, I can't interact with IPAD :)

For a last request, do you recommend me anything for the logging of the application running on device? Can i get logs that are written by application in any way from IPAD or can i access to the logs written by NSLog?

CS

[Stewart Gleadow]

Having VoiceOver on is a real pain for anything other than testing, at least when trying to interact with the device manually...

If your device is plugged in, you can switch VoiceOver on and off in iTunes, which is a lot easier than navigating all those menus in the settings. That way you can toggle it on and off as needed.

- Stew

[sqsuica]

Hi Can,

I am having the same issue as you had before. Can you please help me
on what I should pass in the env.rb?

require 'frank-cucumber'
APP_BUNDLE_PATH = USING_PHYSICAL_DEVICE

and I am getting "uninitialized constant Object::USING_PHYSICAL_DEVICE
(NameError)"

Thanks!

On Sep 15, 8:59 pm, Can Saner <cansa...@gmail.com> wrote:
> Hello again,
>

> ...
>
> read more »

[sqsuica]

Hi,

I have finally figured out that I need to put USING_PHYSICAL_DEVICE =
true before the line and then install the dnssd. Anyway, I am having
an intermittent issue where it only worked on the first steps and
having this connection error:

Scenario: #
my_first.feature:15

Given the app is launched #

step_definitions/launch_steps.rb:5
When I wait for 5 seconds # /Users/
administrator/Downloads/moredip-Frank-075/gem/lib/frank-cucumber/
core_frank_steps.rb:282
When I touch the button marked "Hello" # /Users/
administrator/Downloads/moredip-Frank-075/gem/lib/frank-cucumber/
core_frank_steps.rb:218
Connection refused - connect(2) (Errno::ECONNREFUSED)
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
net/http.rb:644:in `initialize'
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
net/http.rb:644:in `open'
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
net/http.rb:644:in `block in connect'
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
timeout.rb:44:in `timeout'
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
timeout.rb:89:in `timeout'
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
net/http.rb:644:in `connect'
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
net/http.rb:637:in `do_start'
/Users/administrator/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/
net/http.rb:626:in `start'
/Users/administrator/Downloads/moredip-Frank-075/gem/lib/frank-
cucumber/frank_helper.rb:162:in `make_http_request'
/Users/administrator/Downloads/moredip-Frank-075/gem/lib/frank-
cucumber/frank_helper.rb:144:in `post_to_uispec_server'
/Users/administrator/Downloads/moredip-Frank-075/gem/lib/frank-
cucumber/frank_helper.rb:69:in `frankly_map'
/Users/administrator/Downloads/moredip-Frank-075/gem/lib/frank-
cucumber/frank_helper.rb:9:in `touch'
/Users/administrator/Downloads/moredip-Frank-075/gem/lib/frank-
cucumber/core_frank_steps.rb:219:in `/^I touch the button marked "([^
\"]*)"$/'
my_first.feature:18:in `When I touch the button marked "Hello"'
Then I should see "Hello, World!" # /Users/
administrator/Downloads/moredip-Frank-075/gem/lib/frank-cucumber/
core_frank_steps.rb:47

Anyone has any ideas?

Thanks!

> > > > can't run test cases against neitherphysicaldevice nor simulator.

> > > > Can you please help me if you see anything that I miss?
>
> > > > Detailed error trace is this:
>
> > > *************************************************************************** ­**************************************************
> > > > Yelo-Tests-MacBook-Pro:CountItOut yelotest$ pwd
> > > > /Users/yelotest/Desktop/frank/example/CountItOut
> > > > Yelo-Tests-MacBook-Pro:CountItOut yelotest$ cucumber
> > > > Feature: Demonstrating using Frank to test the CountItOut example app
>
> > > >   Background:                 # features/main.feature:3
> > > >     Given the app is launched #
>

> ...
>
> read more »

[Can Saner]

Your env.rb file should be something like below:


-------------------------

Yelo-Tests-MacBook-Pro:automated_tests yelotest$ cat features/support/env.rb 

require 'frank-cucumber'

require 'rspec/expectations'

APP_BUNDLE_PATH = "/Users/yelotest/Library/Application Support/iPhone Simulator/4.2/Applications/50CB7112-424A-4DCF-9D6A-3C7D7DAF8342/Yelo.app"

USING_PHYSICAL_DEVICE = true

-------------------------

That error means that you have to define USING_PHYSICAL_DEVICE variable.

CS

[LongRed]

Is there a more complete step by step guide for testing against a physical device available?

 

Which fork/branch?

What additional code?

Additional launch steps?

Env variables?

 

It's still not working for me.

 

Thanks and Regards,

[LongRed]

How to compile the bonjour branch? There is no rakefile and the
xcodeproj isn't working with the latest xcode.

Any help would be appreciated.

Regards,

[BrFrank]

Has this step by step guide to run test on the device been posted anywhere?

Thanks

[Peter Hodgson]

Inspired by Pat's email I took a bit of time to add basic support for running Frank on a physical device. The changes are here: https://github.com/moredip/Frank/commit/8ec2d2c92e05880c31428d01491d4685199b54ba

I haven't written this up (yet), but if you take a look at the changes I made to the Controls example app you should get an idea of what's involved. Basically just set an environment variable when you run the tests, then have your cuke support scripts check for that env var. If it's there then ask Frank to search for servers via Bonjour, and use the first one it finds - https://github.com/moredip/Frank/blob/8ec2d2c92e05880c31428d01491d4685199b54ba/example/Controls/features/support/env.rb.

HTH,

Pete

[BrFrank]

Thanks Pete. It looks like I have done everything that is listed here, but it is running the tests on the iPhone simulator instead of the iPad device.
The app is running fine on the iPad, I can see this log info:
2012-06-05 23:43:12.949 xxxxFrank[1010:707] Bonjour Service Published: domain(local.) type(_http._tcp.) name(Frank UISpec server)

So for some reason it is not finding the device. Am I missing anything here?
Thanks a lot!! any help is really appreciated since I am running out of ideas.

Marcio

[David]

So I managed to get my tests running on device, but it restarts my simulator with every test. I don't know why it even does this when the tests are running on the device. 

Any help would be appreciated. 

Thanks,

David

[David]

Argh nevermind. I stupidly modified the wrong launch step. Running on device and simulator no longer launching with every test :) 

[Franker]

Hi David,

Could you please post step by step procedure for connecting physical device. Appreciate your help.

.franker

[Andrey Kozlov]

Hi,

Yeah David,

If you could share exact steps of how you was managed to run test on the Device it would be much appreciated. 

Thanks,

Andrey.

суббота, 20 октября 2012 г., 7:35:26 UTC+1 пользователь Franker написал:

[Franker]

Pete Hodgson

Could you please post step by step procedure for connecting physical device. Appreciate your help.

 .franker

On Wednesday, May 18, 2011 1:25:02 AM UTC-7, cansaner wrote:

Pete Hodgson just told me "Yes, it is possible to run Frank tests

[Tom]

Kind of an old issue, but were the steps for building an app and getting it on a device documented anywhere? I keep running in to problems actually installing the Frankified app on my device itself. :(

[ted....@shopnastygal.com]

Same, here been research for a whole day and instruction could really help. 

[Peter Hodgson]

I've personally never done this (installing/launching an app on a device), but I believe you have two possible options. First you can use fruitstrap, either the 'official' repo or one of the subsequent. Second Kam has said in the past on this mailing list that you can actually use the Instruments via the command line to do this. Hope that's of some help. If you get it figured out please write it up for others. I'd be happy to add instructions to the Frank website.

--
You received this message because you are subscribed to the Google Groups "Frank" group.
To unsubscribe from this group and stop receiving emails from it, send an email to frank-discus...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Pete Hodgson

Señor Consultant!

@ph1

thepete.net

cell: 707-206-8897
skype: thepete

[ted....@shopnastygal.com]

I've got it running on a device.  The instructions below worked for my case, your mileage may very:

1.  You've got to build an frankified app for the device.  You can use xcodebuild on the command line to do this.  If you have multiple targets, you have to specify the target to build (or schemes if you have a workspace).   The default frank build command builds only for the simulator.

cd <the_dir_your_xcode_project_is_in>

xcodebuild \

-target <your_target_(optional_if_you_have_only_one)> \

-xcconfig ./Frank/frankify.xcconfig \

-arch armv7 \

-configuration Debug \

-sdk iphoneos \

DEPLOYMENT_LOCATION=YES \

DSTROOT="<full_system_path_to_the_dir_your_xcode_project_is_in>/Frank/frankified_build" \

FRANK_LIBRARY_SEARCH_PATHS="full system path to the dir your xcode project is in>/Frank" \

clean build

Hopefully, you can build it without errors.

2. You have to install the app to the device.  As Pete mentioned, you can use the fruitstrap tool.  Make sure your provisioning includes your device or it will fail on the verification step; and that you have proper code-signing certs in your Xcode.

git clone git://github.com/ghughes/fruitstrap.git

cd fruitstrap

make fruitstrap

... still in your fruitstrap directory

./fruitstrap -b <full_system_path_to_the_app_you_just_built>

This will start the process of loading it to your device.  Now, pray that it finishes loading.  Delete your previous build first to verify that it actually loaded.

3. Next, in your env.rb you have to tell it to use the physical device.  Look in the Controls example app in the Frank Repo for how to do that. 

... cd to your Frank directory

cucumber

hopefully, If everything worked, you will see...

finding Frank server via Bonjour...

...found Frank via Bonjour: 192.168.1.116

THATS IT! 

Oh yeah, I had to manually start the app on the device or else Bonjour won't kick in.  And it won't work if the app is in the background.  I'm still looking for a way to launch the app remotely after installation (I think that should be possible).  I think you can certainly write a shell script to do the build, install, and launch all in one go.

Hope this is helpful for those looking to use frank-cucumber on the device.  If you encounter other things I've missed, please share it here.

ted

[Ankur Agarwal]

Nice tutorial!

I've not got my app working with frank on the device, but i did find a way to load the app on the device from the terminal. I found this command line from another post (https://groups.google.com/forum/#!searchin/frank-discuss/test_on_physical_device_via_bonjour/frank-discuss/5me-BWI2ZZw/AjI7D7QCpskJ):

instruments -w <Your_Device_UDID> -t "/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/Resources/templates/Activity Monitor.tracetemplate" -D /tmp/ignoredtracedata.trace <App_Name>.app

Just one question about building a frankified app for the device, usually it builds two version of the app, one named blah_debug.app and another called frankified.app. When building for armv7 using the command line, it only builds a blah_debug.app. Is this the frankified version of my app? If so then is there any other way of checking that the app is broadcasting itself to bonjour?

Thanks!

[Ted Liao]

Ankur,  yes the blah_debug.app is your frankified app.  And I did figure out how to use fruitstrap to load and RUN the app.  I also wrote a shell scrip to build, install, and run the frankified app all in one command.

my updated instructions:

//----- STEP 1: build frankified app from command line -----//

xcodebuild \

-target <app_target (optional, if you have only one)> \

-xcconfig ./Frank/frankify.xcconfig \

-arch armv7 \

-configuration Debug \

-sdk iphoneos \

DEPLOYMENT_LOCATION=YES \

DSTROOT="<full_path_to_your_xcode_project_directory>/Frank/frankified_build" \

FRANK_LIBRARY_SEARCH_PATHS="<full_path_to_your_xcode_project_directory>/Frank" \

clean build

//----- STEP 2: build and run fruitstrap -----//

Preparation:

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/6.1[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/6.1 (10B141)/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/6.0[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/6.0/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/5.1[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/5.1/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/5.0[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/5.0/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/4.3[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/4.3/DeveloperDiskImage.dmg

1. Get my fork of fruitstrap ( I had to patch the default path for gdb)

git clone git://github.com/ekbiker/fruitstrap.git

2. Build and run fruitstrap

cd fruitstrap

make fruitstrap

./fruitstrap -d -b <full_path_to_your_xcode_project_directory>/Frank/frankified_build/<app_name>.app

Or run the shell script, I named it odo:

#!/bin/bash

# Exit on errors

set -e

# Set paths

xcode_proj_path= <xcode_proj_path>

app_name=<app_name>

build_target=<build_target>

# Build frankified app

xcodebuild \

-target ${build_target} \

-xcconfig ./Frank/frankify.xcconfig \

-arch armv7 \

-configuration Debug \

-sdk iphoneos \

DEPLOYMENT_LOCATION=YES \

DSTROOT="${xcode_proj_path}/Frank/frankified_build" \

FRANK_LIBRARY_SEARCH_PATHS="${xcode_proj_path}/Frank" \

clean build

# Use fruitstrap as a means to install and run the app

./fruitstrap -d -b ${xcode_proj_path}/Frank/frankified_build/${app_name}.app

cheers,

ted

[Ted Liao]

Ankur, when the app start, check console messages for "Bonjour"...  you should see some mentions of Bonjour and the port that it stared on.

ted

On Monday, March 18, 2013 2:27:06 AM UTC-7, Ankur Agarwal wrote:

[Tom]

Ted, you rock! Following your instructions worked like a charm for me!

[Ankur Agarwal]

So i've finally got it working... kind of. I found that the port 37265 was being blocked by the firewall, so i had to change the port in FrankServer.h and also in bonjour.rb. There is still one issue that remains, which is that the bonjour server on the device doesn't seem to start (no reference to bonjour in the app logs), so for now i've had to hard code the ip address in bonjour.rb file. I'm going to keep trying and get back, if i find a solution to this issue.

[Peter Hodgson]

Ted, thanks so much for the detailed writeup. Really great! It will be very helpful for the future. 

Would you be OK with this write up being added to the testingwithfrank.com website? Do you feel motivated to add it yourself to the website repo and send in a pull request? Or does anyone else want to help out with this?

Cheers,

Pete

[Ted Liao]

Pete, thank you for all the work put into Frank.  Of course you can put my instruction on the website.

Here's the text:

To get Frank running on a physical device.

//----- STEP 1: build frankified app from command line -----//

You can use xcodebuild on the command line to do this.  If you have multiple targets, you have to specify the target to build (or schemes if you have a workspace).   The default frank build command builds only for the simulator.  This build command will leave you with a binary <app_name>.app in your frankified_build directory; and it is the frankified app.

xcodebuild \

-target <app_target (optional, if you have only one)> \

-xcconfig ./Frank/frankify.xcconfig \

-arch armv7 \

-configuration Debug \

-sdk iphoneos \

DEPLOYMENT_LOCATION=YES \

DSTROOT="<full_path_to_your_xcode_project_directory>/Frank/frankified_build" \

FRANK_LIBRARY_SEARCH_PATHS="<full_path_to_your_xcode_project_directory>/Frank" \

clean build

//----- STEP 2: build and run fruitstrap -----//

Fruitstrap will install the app to the device and launch it using gdb. Make sure your provisioning includes your device or it will fail on the verification step; and that you have proper code-signing certs in your Xcode.  Delete your previous build first to verify that it actually loaded.

Preparation:

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/6.1[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/6.1 (10B141)/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/6.0[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/6.0/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/5.1[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/5.1/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/5.0[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/5.0/DeveloperDiskImage.dmg

symlink: /Users/<user_dir_name>/Library/Developer/Xcode/iOS DeviceSupport/4.3[.x] (xxx)/DeveloperDiskImage.dmg from 

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/4.3/DeveloperDiskImage.dmg

1. Get my fork of fruitstrap ( I had to patch the default path for gdb)

git clone git://github.com/ekbiker/fruitstrap.git

2. Build and run fruitstrap

cd fruitstrap

make fruitstrap

./fruitstrap -d -b <full_path_to_your_xcode_project_directory>/Frank/frankified_build/<app_name>.app

//----- STEP 3: run cucumber -----//

In your env.rb you have to tell it to use the physical device.  Look in the Controls example app in the Frank Repo for how to do that. 

... cd to your Frank directory

cucumber

hopefully, If everything worked, you will see...

finding Frank server via Bonjour...

...found Frank via Bonjour: 192.168.1.116

Or, in your console logs you will see the bonjour port the Frank Sever is now bound to...

THATS IT! 

//----- Here's a shell script that will build and run in one step -----//

#!/bin/bash

# Exit on errors

set -e

# Set paths

xcode_proj_path= <xcode_proj_path>

app_name=<app_name>

build_target=<build_target>

# Build frankified app

xcodebuild \

-target ${build_target} \

-xcconfig ./Frank/frankify.xcconfig \

-arch armv7 \

-configuration Debug \

-sdk iphoneos \

DEPLOYMENT_LOCATION=YES \

DSTROOT="${xcode_proj_path}/Frank/frankified_build" \

FRANK_LIBRARY_SEARCH_PATHS="${xcode_proj_path}/Frank" \

clean build

# Use fruitstrap as a means to install and run the app

./fruitstrap -d -b ${xcode_proj_path}/Frank/frankified_build/${app_name}.app

[Peter Hodgson]

Awesome, thanks Ted. Here's the page: http://testingwithfrank.com/device.html Let me know if you see any issues.

[Tony ZH]

Hi Pete,

I have been following the steps. The app is installed on the physical device but the cucumber testing is still running on simulator. I am also able to see the message 'finding Frank server via Bonjour...\n...found Frank via Bonjour: 192.168.1.151'. Do you know why this happens and what I can do to get it running on actual device? 

Thank you so much.

Best,

Tony

[Pete Hodgson]

Sounds like you have two Frank servers running - one on your local simulator and one on a device. The server on the simulator is also discoverable over bonjour. I think that server is getting discovered before the server on the device. Any reason you can just close the Simulator when trying to test on a device?

Cheers,

Pete

--

Typed on a little bitty keyboard

[Kyle Kendall]

Hi all,

I have gone through the instructions provided by Ted.  When I put in the following, with my path to the .app:

  

  ./fruitstrap -d -b <full_path_to_your_xcode_project_directory>/Frank/frankified_build/<app_name>.app

It looks like everything works, it displays that it's copying files to the device, and it gets to 49%, but then I get this:

[ 52%] CreatingStagingDirectory

[ 57%] ExtractingPackage

[ 60%] InspectingPackage

AMDeviceInstallApplication failed: -402653103

I searched online, and I found this forum and it's saying that the certificates or provisioning profiles are wrong.  I have a provisioning profile and I can run the app on the device not through fruitstrap, so I don't believe that something is up with my provisioning profiles.  Does anyone have a solution to this problem?

Thank you all

[Kyle Kendall]

I had the wrong .app specified, I changed that to the correct .app, and now I get this:

[ 52%] CreatingStagingDirectory

[ 57%] ExtractingPackage

[ 60%] InspectingPackage

[ 60%] TakingInstallLock

[ 65%] PreflightingApplication

[ 70%] VerifyingApplication

AMDeviceInstallApplication failed: -402620388

[Kyle Kendall]

In step 2 of the testing on device tutorial it just has a list of symlinks.  I only put the first one in.  Do I need to do the other 4?

[Kyle Kendall]

I am stuck on getting a Frankified.app running on a device.  When I do step 1 on the online tutorial:

xcodebuild -target Frank-Fruitstrap -xcconfig ./Frank/frankify.xcconfig -arch arm7 -configuration Debug -sdk iphoneos DEPLOYMENT_LOCATION=YES DSTROOT="~/Documents/Workspace/Frank-Fruitstrap/Frank/frankified_build/" FRANK_LIBRARY_SEARCH_PATHS="~/Documents/Workspace/Frank-Fruitstrap/Frank" clean build CODE_SIGN_IDENTITY='iPhone Developer: <Firstname Lastname>'

and I get this:

** CLEAN SUCCEEDED **

=== BUILD NATIVE TARGET Frank-Fruitstrap OF PROJECT Frank-Fruitstrap WITH CONFIGURATION Debug ===

Check dependencies

No architectures to compile for (ONLY_ACTIVE_ARCH=YES, active arch=arm7, VALID_ARCHS=armv7 armv7s).

** BUILD FAILED **

The following build commands failed:

Check dependencies

(1 failure)

The codesign works for other apps when I run them from command line, just not a Frankified app.

Am I the only one who has problems with this?

[Peter Hodgson]

This is a fairly common project setting issue I think. You should be able to resolve it by modifying the build settings for your Frank-Fruitstrap target. Either change "Build Active Architecture Only" to "No", or add "arm7" to the "Valid Architectures" list. 

--

[Kyle Kendall]

Awesome, thanks a ton Pete.  That solved one problem.  I think I have just one more thing and I am good to go.  It doesn't recognize my codesign.  I get this:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

** CLEAN SUCCEEDED **

=== BUILD NATIVE TARGET Frank-Fruitstrap OF PROJECT Frank-Fruitstrap WITH CONFIGURATION Debug ===

Check dependencies

warning: no rule to process file '$(PROJECT_DIR)/Frank-Fruitstrap/main.m' of type sourcecode.c.objc for architecture arm7

warning: no rule to process file '$(PROJECT_DIR)/Frank-Fruitstrap/DAppDelegate.m' of type sourcecode.c.objc for architecture arm7

warning: no rule to process file '$(PROJECT_DIR)/Frank-Fruitstrap/DViewController.m' of type sourcecode.c.objc for architecture arm7

Code Sign error: The identity 'Kyle Kendall' doesn't match any valid, non-expired certificate/private key pair in your keychains

** BUILD FAILED **

The following build commands failed:

Check dependencies

(1 failure)

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I can run all my apps on my device.  I'm not sure why it doesn't recognize my codesign.  In Keychain Access in "My Certificates" I see this "iPhone Developer: <FirstName> <LastName> (XXXXXXXXX)"

I tried including the numbers in the () but it gives me the same message.  Does someone have an idea of what I might try?

[Kyle Kendall]

Thanks for everyone's help on this.  I am SOOOO close!  I hope at least...  I was able to build my target successfully.  The problem I have now is when I do this:

./fruitstrap -b ~<path to my app>/<MyAppName>/Frank/frankified_build/Frankified.app/

I get this still:

[ 70%] VerifyingApplication

AMDeviceInstallApplication failed: -402620388

I thought that if I was able to build the target successfully that this wouldn't be a problem, but I thought wrong.  Has anyone ever had this problem?

[Kyle Kendall]

Here is my stack overflow question with a 50 point bounty for another 6 days (til May 22nd 2013): stackoverflow run frank tests on device

I appreciate the help on this.  If I can get Frank up and running, our company will likely use this to do all our iOS tests, so hopefully we will be able to contribute to back to Frank.

[Kyle Kendall]

Hey everyone, I was able to get the tests running on my device, but none of the touch events are working.  I can type into textfields and verify that a view is on the screen.  I can run the same tests in the simulator and the touch events work, but on the device I get a log that it passed, but the button isn't touched.  I've tried it with VoiceOver turned on, but that doesn't change anything.  Do I need to have VoiceOver turned on?

[Stewart Gleadow]

You shouldn't need to turn voiceover on. I think the hacky code that enables accessibility automatically when Frank starts should also work on the device (at least, it did when written), but it uses private APIs so may have broken with subsequent iOS updates.

On 30/05/2013, at 7:56 AM, Kyle Kendall <kkend...@gmail.com> wrote:

Hey everyone, I was able to get the tests running on my device, but none of the touch events are working.  I can type into textfields and verify that a view is on the screen.  I can run the same tests in the simulator and the touch events work, but on the device I get a log that it passed, but the button isn't touched.  I've tried it with VoiceOver turned on, but that doesn't change anything.  Do I need to have VoiceOver turned on?

[Kyle Kendall]

When I do <devicesIPAddress>/37265/dump with the app running on a device I get a lot of this:

{"uid":520756288,"subviews":[{"uid":520782736,"subviews":[{"uid":520765856,"subviews":[{"uid":520778448,"subviews":[{"uid":508792272,"subviews":[{"uid":521120592,"subviews":[{"uid":520536368,"subviews":[],"class":"UIImageView"}, ....

and when I do http://localhost:37265/dump

I get stuff like this:

{"frame":{"size":{"width":320,"height":480},"origin":{"x":0,"y":0}},"uid":199387296,"backgroundColor":{"green":0,"red":0,"alpha":1,"blue":0},"isHidden":0,"subviews":[{"frame":{"size":{"width":320,"height":460},"origin":{"x":0,"y":20}},"uid":164311920,"backgroundColor":{"green":0.1960784,"red":0.1254902,"alpha":1,"blue":0.1607843},"isHidden":0,"subviews":[{"frame":{"size":{"width":320,"height":460},"origin":{"x":0,"y":0}},"uid":164253904,"backgroundColor":"<NON-RGB COLOR>","isHidden":0,"subviews":[{"frame":{"size":{"width":320,"height":460},"origin":{"x":0,"y":0}},"uid":164092976,"isHidden":0,"subviews":[{"frame":{"size":{"width":320,"height":460},"origin":{"x":0,"y":0}},"uid":164407120,"isHidden":0,"subviews":[{"zoomScale":1,"alwaysBounceHorizontal":0,"frame":{"size":{"width":320,"height":460},"origin":{"x":0,"y":0}},"alpha":1,"contentInset":"<UIEdgeInsets>","class":"UIScrollView","showsHorizontalScrollIndicator":1,"subviews":[{"frame":{"size":{"width":100,"height":100},"origin":{"x":110,"y":52}},"uid":164074080,"isHidden":0,"accessibilityLabel":"logo_100","subviews":[],"tag":0,"accessibilityFrame":{"size":{"width":100,"height":100},"origin":{"x":110,"y":72}},"alpha":1,"autoresizingMask":37,"class":"UIImageView"},{"frame":{"size":{"width":320,"height":142},"origin":

with the accessibility labels and a lot more information.  Is that hacky code broken then?  Shouldn't it not work for both of them if the accessibiliy code is broken?

[Kyle Kendall]

Kinda weird, but I just added the touchxy method to my own step definitions, and now everything works just fine....   Good!  But I wish I knew why that worked!  I will now be working on being able to run these tests on multiple devices from the same machine, not at the same time.  For example I'll run the app on an iPhone 4S, then iPhone 5, then iPad mini, etc.  Has anyone already done something similar?

Thanks a bunch for all the help everyone!  Our company will most likely use Frank to do all of our iOS testing.  Hopefully we will be able to help make this awesome system even better!  Go team!

[Mallikarjun H N]

Hi can anyone tell me whether we need source code for automation in Frank.

[Stewart Gleadow]

Yes, you need to be able to rebuild the iOS app with the Frank library linked in order to use Frank for testing. If you don't have access to the source code, the only option I know if is UIAutomation. However, I think automated testing of this nature will not be effective or efficient without access to the source code. You'll likely want to improve the accessibility of the app in order to make it easier to test.

cheers,

Stew

[Santiago Rodriguez Sordo]

Hi all, i'm having troubles launching frankified app on physical devices. I'm building as: xcodebuild \

-target <app_target (optional, if you have only one)> \

-xcconfig ./Frank/frankify.xcconfig \

-arch armv7 \

-configuration Debug \

-sdk iphoneos \

DEPLOYMENT_LOCATION=YES \

DSTROOT="<full_path_to_your_xcode_project_directory>/Frank/frankified_build" \

FRANK_LIBRARY_SEARCH_PATHS="<full_path_to_your_xcode_project_directory>/Frank" \

clean build

but when the build is launched in device via ios-deploy or fruitstrap, I cant found events related to Bonjour, instead I can only see in console: 

SpringBoard server could not disconnect from frontmost app message service gracefully: (null)

my ipad is version 7.0.3

I would need some help over here...

thanks a lot, in advance.

[Tyler Norlund]

Hey Pete!

So I've been able to run fruitstrap and can't seem to run cucumber on device.

I took the env.rb from the Controls example and added some puts. Whenever I run cucumber, I can't get into the if statement

 if $USING_PHYSICAL_DEVICE

Frank::Cucumber::FrankHelper.test_on_physical_device_via_bonjour
After do

step 'I ask the app to reset to home'

I've symlinked the DeveloperDiskImage(s) and used fruitstrap. My app is signed correctly and built to armv7. I've tried just about everything but can't get it to build to device. 

Please help!