End-to-end testing for iOS apps

[Engin Kurutepe]

Hey all,

Does anybody use any end-to-end testing tools for their iOS apps? Our web guys are using selenium and apparently it can drive the iOS simulator based tests using appium (http://appium.io).

Any experience?

Cheers,
Engin.

[Thomas Mellenthin]

Hi,

Engin Kurutepe wrote:
> Does anybody use any end-to-end testing tools for their iOS apps?
> Our web guys are using selenium and apparently it can drive the iOS
> simulator based tests using appium (http://appium.io).

We're using appium to run ui-tests on our app. My feelings about it are
mixed: when it works, it's great. But sometimes it breaks without any
indication what the problem is (often it's the communication between
Instruments.app and the appium server).

Appium itself is not yet stable. We're still working with 0.18, because
the next big release "1.0" changed a lot and broke functionality that is
mandatory for us [1]. So we have to wait for milestone 1.3 hoping that
"our" issue will be fixed...

You can write the tests in the language you prefer. So our QA-Guys can
write the tests too. And (in theory) you can share test code between
your iOS and Android tests (if the app shares the same UI concepts).

Of course there are problems that are specific to all frontend test
tools like latency. It sums up (animations, network, ..). Our Testsuite
takes 51 minutes for a complete run. You have sometimes do stupid things
like polling.

And the worst problem: every tiny bit can lead to a failing test.
Backend foo. Network hickups. A fly on the keyboard.
Our unittests are most of the time green. The frontend tests are most of
the time red :(

Some numbers: our app has 386853 LoC (*.m *.h *.xib), the test Suite
11183 LoC (*.java) which is roughly 3%.


[1] https://github.com/appium/appium/issues/2698


best regards,

// thomas

--
xmpp: me...@jabber.ccc.de passion: http://mellenthin.de
F489 2C4F E8C2 9A15 DBCB mission: http://freifunk-potsdam.de
127C 81B6 FDC3 7C1A FF85 pubkey: http://mellenthin.de/key.txt