Android -> web app integration tests

8 views

Skip to first unread message

John D. Hume

unread,

Jul 14, 2014, 8:36:11 AM7/14/14

to rapi...@googlegroups.com

I just noticed this https://github.com/rapidftr/tracker/issues/107 about stubbing the server in integration tests. This sounds like a major change, as once the server is stubbed, those tests cease to integrate nearly as much as they did previously. This may reduce their value to the point that the suite is not worth maintaining (depending on what other tests there are) and/or expose the system to integration risk that was previously guarded against and/or require more manual integration testing than was required previously.

There may be alternative solutions that would maintain the suite's coverage of the integrated system, though I don't know what constraints Travis imposes. Can we discuss that?

Apologies if there was a discussion on the mailing list that I missed.

Subhas Dandapani

unread,

Jul 14, 2014, 10:37:09 AM7/14/14

to rapi...@googlegroups.com

Hi John, Thanks for noticing this, and I guess we've put in quite a bit of context in the story itself. We've had a long standing problem of random failures in the Android test, because of the shared test.android.com, and the way data has to be seeded to it before the story, etc. We didn't really have much options except running the whole RapidFTR stack (including CouchDB, etc) everytime we wanted to test, but this was also difficult.

On running RapidFTR server as part of the Android tests on Travis - We tried that, We need Ruby 2.1.x to run RapidFTR. But Travis CI environment says that every VM will have at least one version of Ruby, but we have no way to control which version we get. Last time we tested, it had Ruby 2.0. And at some point in the future, it will end up having some other version of Ruby. So we can't rely on Travis to have the right requirements for running RapidFTR server.

We finally resorted to Martin Fowler's blog on Non-Determinism in Tests (which was a good read), especially the Remote Services part, and I was also part of his live talk on this. Seems this is a broad problem whenever we have external services to test against, and after much contemplating, his suggestion was to use Service Stubs as the most stable approach, coupled with have Contract tests on the stubs themselves. And having hit a wall many times on this problem, we decided to give this approach a try. We will be adding contract tests as well.

--
You received this message because you are subscribed to the Google Groups "rapidftr" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rapidftr+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.