How to integrate appium and run those unit test cases from Jenkins CI server.

[Dipti Ranjan Lenka]

How to integrate appium and run those unit test cases from Jenkins CI server.
please help me in this regards.

i want to run those appium testes after the build has finished in my local jenkins CI server.

[Brent LaVelle]

I've been doing this for a few weeks now.  The way I work is to have Appium running all the time.  Since we have iOS and Android versions our jenkins nodes are OSX running on Mac Minis but it should work like this on other platforms as well.

I do only have 1 iOS device and soon 1 Android device on each machine but I plan on figuring out how to address multiple virtual and actual devices soon.

The problem I ran into with iOS was finding my .app file.  I solved it by copying the build job that was making the .app so I could move the directory to a known place that my rspec tests could pick it up from a known location.

To keep the Android and iOS tests running in parallel I have a job for each of them and they are triggered by the build I cloned passing.

If you have any problems I'd love to help you with you CI work.

[Dan Cuellar]

This is very similar to the setup we have. Mac Minis FTW

[Joel Miele]

Is running headlessly possible?

[Ling-Yi Kung]

Yes - we have Mac minis set up to run headlessly as Jenkins slaves.

I also copy either the .app or .apk from a build job ('copy artifact' in Jenkins) to run tests on. Sometimes I have to compile the .app in my test job rather than copying a build artifact (to compile with a simulator target). I launch an instance of Appium for every Jenkins test, then kill it. Haven't figured out how to parallelize it on a single node yet, but that's the goal.

On Wednesday, October 9, 2013 10:36:06 AM UTC-7, Joel Miele wrote:

Is running headlessly possible?

[Jonathan Lipps]

No, unless by headlessly you mean "without a monitor" :-)

On Oct 9, 2013, at 10:36 AM, Joel Miele <jmi...@snap-interactive.com> wrote:

Is running headlessly possible?

--
http://appium.io
---
You received this message because you are subscribed to the Google Groups "Appium-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to appium-discus...@googlegroups.com.
Visit this group at http://groups.google.com/group/appium-discuss.
For more options, visit https://groups.google.com/groups/opt_out.

[Joel Miele]

We have a bunch of virtual jenkins slaves with no graphical output capability.  As far as I know this falls under both "headless" and "without a monitor"?

What is the major difference between headless and without a monitor?

[Jonathan Lipps]

I thought you meant headless in the way that phantomjs is a headless web browser--i.e., doesn't involve chrome.

Yes, you don't need a monitor connected.

On Oct 9, 2013, at 11:17 AM, Joel Miele <jmi...@snap-interactive.com> wrote:

We have a bunch of virtual jenkins slaves with no graphical output capability.  As far as I know this falls under both "headless" and "without a monitor"?

What is the major difference between headless and without a monitor?

[Brent LaVelle]

Without graphics the emulators will fail.  I tried running on a mini as a different user than the one logged in on the console and that didn't work. You can probably pull the monitor but I haven't tried that.

[Joel Miele]

So Appium automation can run with the emulator in -no-window mode?

[Jonathan Lipps]

Ok, let me back off from answering these questions. Based on my knowledge of computers, unplugging the monitor shouldn't stop things from working. But beyond that, I don't know how emulators/appium will behave :-)

I have a proposition: try what you're thinking of and let us know :-)

On Oct 9, 2013, at 1:04 PM, Joel Miele <jmi...@snap-interactive.com> wrote:

So Appium automation can run with the emulator in -no-window mode?

[Anil Kumar]

If xcode instruments and UIA Automation can work with no-window emulator, appium will work with it too. As simple as that it is.

[dan....@lookout.com]

What's -no-window mode??

On Oct 9, 2013, at 1:04 PM, Joel Miele <jmi...@snap-interactive.com> wrote:

So Appium automation can run with the emulator in -no-window mode?

[Joel Miele]

Well I have tried launching an emulator in -no-window mode and automation promptly failed.  The Appium server continually displayed "waiting for app activity...".  This leads me to believe functional tests cannot be run in -no-window mode.  However, I'm no expert and there could be a way to make it work... which is what I'm getting at, is there?

[Patrick L]

Is there a way to not have Appium running all the time and only start it as needed before tests are run?

This would also help resolve issues where builds completely fail if Appium crashes and builds are started not knowing they cant connect to the appium server.

[Jonathan Lipps]

Yes, you can start appium from the command line…

[Patrick L]

When I run Appium command line it works, but in the Jenkins execution shell it doesn't work:

+ /usr/local/bin/appium

env: node: No such file or directory

[Jonathan Lipps]

Is node.js installed in that environment?

appium is run with:

/usr/bin/env node

[Melvin Laguren]

Is the path set up properly in Jenkins?  I resolved that issue for me by going to the Jenkins configuration and creating a Jenkins PATH environment variable that looked like this:  /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

[Patrick L]

Thanks for the heads up on that Melvin. I now have all my Jenkins environment variables set correctly and made some more progress with that. Now I am seeing issues when appium starts up and not connecting the test correctly. Here is my output:

+ appium
 [36minfo [39m: Welcome to Appium v0.15.0 (REV a7b47d73a27074cc928cc5b325e5d4de1b6e5594)
 [36minfo [39m: Appium REST http interface listener started on 0.0.0.0:4723
    [36minfo  - [39m socket.io started
 [36minfo [39m: Spawning instruments force-quitting watcher process
 [36minfo [39m: [FQInstruments STDERR] Force quit unresponsive instruments v0.0.1
 [36minfo [39m: [FQInstruments STDERR] _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.

Then my test tries to run and further down I get:

 [36minfo [39m: Instruments is at: /Applications/Xcode.app/Contents/Developer/usr/bin/instruments
 [36minfo [39m:  [90m[INSTSERVER] Instruments socket server started at /tmp/instruments_sock [39m
 [36minfo [39m: Spawning instruments with command: /Applications/Xcode.app/Contents/Developer/usr/bin/instruments -t /Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate /Users/Shared/Jenkins/Home/jobs/One-Test/workspace/./app/MyApp.app -e UIASCRIPT /usr/local/lib/node_modules/appium/lib/devices/ios/uiauto/bootstrap.js -e UIARESULTSPATH /tmp/appium-instruments/
 [36minfo [39m: And extra without-delay env: {"DYLD_INSERT_LIBRARIES":"/usr/local/lib/node_modules/appium/build/iwd/InstrumentsShim.dylib","LIB_PATH":"/usr/local/lib/node_modules/appium/build/iwd"}
 [36minfo [39m: And launch timeout: 90000ms
 [36minfo [39m:  [33m[INST STDERR] _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
 [39m
 [36minfo [39m:  [33m[INST STDERR] 2014-02-20 12:59:50.284 instruments[1243:507] Recording cancelled : At least one target failed to launch; aborting run
Instruments Trace Error : Error Domain=com.apple.instruments Code=1 "Error Starting Recording" UserInfo=0x7fec04af8da0 {NSLocalizedDescription=Error Starting Recording, NSLocalizedRecoverySuggestion=At least one target failed to launch; aborting run}
Instruments Trace Error : Failed to start trace.
 [39m
 [36minfo [39m:  [90m[INSTSERVER] Instruments exited with code 253 [39m

Retry's, fails, and then:

[31merror [39m: Instruments crashed on startup
 [36minfo [39m: Killall instruments
 [36minfo [39m: Stopping iOS log capture
 [31merror [39m: uncaughtException: Cannot call method 'kill' of null date=Thu Feb 20 2014 12:59:50 GMT-0500 (EST), pid=1194, uid=236, gid=1, cwd=/Users/Shared/Jenkins/Home/jobs/One-Test/workspace, execPath=/usr/local/bin/node, version=v0.10.23, argv=[node, /usr/local/bin/appium], rss=47648768, heapTotal=34235136, heapUsed=18380088, loadavg=[0.38427734375, 0.5771484375, 0.6640625], uptime=7103, trace=[column=13, file=/usr/local/lib/node_modules/appium/lib/devices/ios/ios-log.js, function=IosLog.stopCapture, line=108, method=stopCapture, native=false, column=26, file=/usr/local/lib/node_modules/appium/lib/devices/ios/ios.js, function=, line=235, method=null, native=false, column=10, file=/usr/local/lib/node_modules/appium/lib/devices/ios/instruments.js, function=Instruments.launchHandler, line=114, method=launchHandler, native=false, column=17, file=/usr/local/lib/node_modules/appium/lib/devices/ios/instruments.js, function=Instruments.onInstrumentsExit, line=309, method=onInstrumentsExit, native=false, column=12, file=/usr/local/lib/node_modules/appium/lib/devices/ios/instruments.js, function=, line=260, method=null, native=false, column=17, file=events.js, function=ChildProcess.EventEmitter.emit, line=98, method=EventEmitter.emit, native=false, column=12, file=child_process.js, function=Process.ChildProcess._handle.onexit, line=789, method=ChildProcess._handle.onexit, native=false], stack=[TypeError: Cannot call method 'kill' of null,     at IosLog.stopCapture (/usr/local/lib/node_modules/appium/lib/devices/ios/ios-log.js:108:13),     at null.<anonymous> (/usr/local/lib/node_modules/appium/lib/devices/ios/ios.js:235:26),     at Instruments.launchHandler (/usr/local/lib/node_modules/appium/lib/devices/ios/instruments.js:114:10),     at Instruments.onInstrumentsExit (/usr/local/lib/node_modules/appium/lib/devices/ios/instruments.js:309:17),     at null.<anonymous> (/usr/local/lib/node_modules/appium/lib/devices/ios/instruments.js:260:12),     at ChildProcess.EventEmitter.emit (events.js:98:17),     at Process.ChildProcess._handle.onexit (child_process.js:789:12)]

Not sure what this is about. I though it had something to do with Jenkins running headless since it fails to connect to the WindowServer but I'm still lost looking about. How do you have your appium setup to run with the tests so this doesnt happen?

[Patrick L]

I found out what my issue was!

If Jenkins is installed as a LaunchDaemon it cant run anything that connects to a GUI (also known as the window server). So when downloading Jenkins from their site and installing it this will happen since its considered headless and not attached to a user that is currently logged in, but rather a hidden user.

To install Jenkins as a LaunchAgent where it can interact with GUIs the method to install is via Homebrew

brew install jenkins

This will attach itself to the current user logged in, which should not be a problem if this user is the only one ever on the machine and is always used for CI. A main drawback is that the user must log in to the machine before jenkins will start up.

So in the end I am now able to start appium via jenkins, or even let appium get booted up within each test during my maven/junit builds. Still working out some kinks but looks like this will be a huge help with appium crashes during builds since a new instance will get booted up on the following test.

[bootstrap online]

I encountered the same agent vs daemon issue. I'm glad you figured it out.

> --
> http://appium.io
> ---
> You received this message because you are subscribed to the Google Groups
> "Appium-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to appium-discus...@googlegroups.com.
> Visit this group at http://groups.google.com/group/appium-discuss.

> For more options, visit https://groups.google.com/d/optout.

[Pooja Shah]

Hi Brent,

Can you please share the procedure to configure Appium test with Jenkins 

The Homework (with many doubts) I did till now is-

1. Running my appium android test on local machine using Appium.app + Genymotion(avd)

so can the same setup work on the remote headless machine on which jenkins work?

2. or I should add a launch appium script in jenkins job and (appium from source should be started) ?

3. There is one Jenkins plugin :https://saucelabs.com/jenkins/2   (is this what will launch Appium.app  or something like that)

How tests run in headless fashion  (for web testing we use XVFB for running browser tests in headless on Jenkins), so how it will work for automation on AVD/Device 

 

Have to find appropriate way to go ahead, (It would be grateful, if there is any step by step process to configure the same)

Please suggest...

Regards,

Pooja

[Mario]

The simplest way I have this working is write an ANT build.xml script and invoke it from the CI server. Appium server should be always running. 

[Vikas Chaudhary]

 

Hi Brent,

 

I am stuck with CI integration. My set up is like we have jenkins server and sepaerate git server. When I am running my test from loccal machine, I am uploading the .app file from local machine, and works fine. But I am not sure how will it work on the git server. I mean how will I copy the app file on the git server, so that it is acccessible to my scripts.  Currently I am providing the fileapth in a property file like this

     AppFilePath=/Users/vchau/Downloads/latestApp/Myapp.app

 and then reading the above property in a variable and providing the capaibilties like this.

    capabilities.setCapability("app",appFilePath);

On Tuesday, October 1, 2013 12:45:50 PM UTC-4, Brent LaVelle wrote:

[Pooja Shah]

Hi Brent,

Are you running the simulator/device on the jenkins machine?

Because I also want to execute my appium tests on jenkins, but the machine on which the tests are hosted, is without display.
Is there way to run tests on mobile (Android/IOS) headlessly? For web ui tests, XVFB servers well, but will it work for mobile simulators as well?

And if you are running on devices, how they are connected on OS(jenkins) physically or in network or cloud?
and for IOS, jenkins machine must be MAC ?
 
Will be great if you can share the configuration guidelines.

Thanks
pooja

On Tuesday, 1 October 2013 22:15:50 UTC+5:30, Brent LaVelle wrote:

[Pooja Shah]

Mario:
Please check my last query

Thanks,
Pooja

[Mario]

Hi Pooja,

Personally, I run on emulators at the moment as I don’t have access to devices on regular basis.

Running in headless mode; This should be possible on all your VM’s running XVFB as it behaves the same as a display if you’re running on devices. A quick google search told me that Android Emulator doesn’t run on XVFB.... But on the bright side you can run the Android Emulator in headless mode by supplying the --no-window option when starting the emulator. I’m not sure about running the iOS Simulator in headless mode. Maybe one of the Appium devs has more insight..

AFAIK, for all iOS testing you need to have a Mac as it runs off Instruments which are shipped with Xcode and Command Line tools.

Hope this helps,

Regards,

Mario

[Pooja Shah]

Thanks Mario for answering, i'm gonna try --no-window option,
also I'm curious to know that, When you run your Android/IOS tests on jenkins, how have you configured? (Just want to double check my plan to configure the same)
Are you having emulators always running on the machine--> invoke Appium  server from source instead of .app  (or using .app also can be done in jenkins process) -->  invoke test and kill the server @ end?

or (all commands written in test code to start emulator-->start appium server-->start test->on finish, shut down server) ?

Also how jenkins process automatically finds the display ? (for web->firefox/chrome it complains "No display" thats the reason have to use XVFB)

Thanks,
Pooja

[Mario]

Hi Pooja,

How you decide to do your config is at your discretion. I would let the project needs dictate the config. So whatever you think is best for your project. Personally, I need a quick turn around for my tests so I keep the emulators and appium server running in the background on the Jenkins machine. Let Appium do the app cleanup and install as required.

The Jenkins project, in my case, is run in tomcat and as such doesn’t need a display (I can access it from another machine that does have a display).. simples!! :) Even if you start Jenkins as a service/daemon you should be able to access it remotely.

Cheers,

Mario