Dependencies order and compile/provided

[Diego Costantini]

Hi,
I am struggling to have robolectric + robotium in a multiproject in AS (I am completely new to it and IDEA so forgive me if I am missing some history).
I think the biggest problem is that I put everything into instrumentTest, and this way I have to compile all dependencies altogether (which makes issues depend on each other and appear exponentially :) ).
As far as I understood, the module dependencies settings in AS is currently under construction, because I keep finding workarounds from the past to the problems I have that suggest to move the dependencies order (for robolectric - example: move junit and robolectric on top, add output folder at the end, ...) and change some of them to provided instead of compile (for robotium - java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation).

My question is: in 0.4.4 is there any way to set them?

And more medium term: is there really work in progress to (1) eventually restore the AS settings to manipulate module dependencies and (2) make unit tests a first class citizen (perhaps by making java and android plugins compatible)?

Thanks for the huge work!

[Jürgen Cruz]

I am also waiting for the day JUnit tests are available for android plugin even if you can't use the android library and require a third party (robolectric). there are various classes that are Java pure that shouldn't require instrumentation tests.

About modifying the dependencies in the modules settings, I would actually prefer to modify the gradle.build file since for the changes you make in the modules settings to be effective, they would have to change your build.gradle file.

[Diego Costantini]

About the order, I don't know if it is possible at all in Gradle to specify the order in the classpath for testrunners.
In that case it would be a Gradle feature to be added rather than AS. Of course AS should then affect the build.gradle in order to make order change effective.

For the compile vs. provided...I have no idea as well :p

[Xavier Ducrohet]

It seems like depending on dependency order is a signal that something is wrong.

Built-in support for junit test through robolectric will not happen, though we want to allow people to add support for it if they really want. This is partly there, though there are some small issues. I think Jake Wharton's plugin was almost there if it got updated to the new artifact api.

(next part missing is the artifact selection in studio but that'll come a bit later).

We won't allow manipulating dependencies in studio unless it's to edit build.gradle. There will not be a way to make Studio behave differently than when running Gradle only.

Make Java and Android plugin compatible is a work in progress and will happen. It will have no impact on helping run JUnit tests though.

--
You received this message because you are subscribed to the Google Groups "adt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adt-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Xavier Ducrohet
Android SDK Tech Lead
Google Inc.
http://developer.android.com | http://tools.android.com

Please do not send me questions directly. Thanks!

[Jürgen Cruz]

When you say "Built-in support for junit test through robolectric will not happen", what do yo mean?

If you mean Robolectric should be integrated with the android gradle plugin, then I agree it shouldn't. If you mean JUnit tests are never gonna be a part of android gradle plugin, then I really want to know why is that.

If the Android Gradle plugin allows for Built-in JUnit tests, it will allow testing for pure Java classes (since any dependency to the Android framework should run in an emulator) and in case someone wants to integrate Robolectric, then you would just need to add Robolectric to the classpath, annotate your tests and maybe configure some things here and there. You shouldn't need a whole new plugin just to add Robolectric, specially a plugin that depends on the Android Gradle plugin that can/will change. I think that is precisely the reason Jake Warthon discontinued his plugin and went full instrumentation tests. I agree with Jake that one should use Instrumentation tests in emulator since they are closer to reality than third party frameworks like Robolectric. But I still consider a lot of pure Java classes exist in an Android app that could benefit from JUnit tests.

In this Google I/O's conference: https://developers.google.com/events/io/sessions/331474237 Erik Kuefler mentions the advantages of separating the logic in a pure Java class (Presenter) to allow running JUnit tests instead of GWTTestCase. In case of Android it is an even bigger advantage since the emulator is a very slow process compared to JUnit.

Just my 2 cents.

[Diego Costantini]

Hi xavier, thanks for the clarification and the look ahead :)

I understand that if I want robolectric:

1 - I should absolutely not have it in instrumentTest with robotium or espresso (I expected that much)

2 - the android plugin will not provide any help, which may or may not be provided by some additional independent plugin instead. If this is what's coming (or not), I hope that Jake will reconsider discontinuing his deprecated plugin...

Perhaps I will try to go back experimenting a bit more with Jake's plugin.

About the dependency order, definitely something could be "less" wrong, but the problem must be somewhere.

During my efforts in the last 3 weeks I found out (at least) the following 4 issues:

1 - I had conflicting dependencies, and some of them (ASM 4.1 vs. 3.1) hit me only at runtime if not in a particular order. These and other dependencies have been tamed through group/module exclusion in gradle.

2 - I needed to move junit >3.7 at the top, because some other dependency has an older version. This is a reason good enough to allow dependency order for junit testrunner classpath. And if Android is the one with that old junit, well it is pretty hard to go without it :)

3 - move robolectric dependency before android, otherwise it will use real implementations rather than the shadows (or whatever we call them). Again, this is how it works, this starts to make me think there might be nothing so wrong about wanting dependency order (btw: didn't intellij have one? is it conflicting somehow within the android studio implementation?)

4 - add my output classpath at the end, so my test classes can be found.

In light of that, I can understand there might be other priorities for AS and gradle android plugin (and some of the issues might be solved by other plugins), but I believe these issues should be eventually addressed for a reasonable IDE.

[Jake Wharton]

On Tue, Feb 11, 2014 at 11:38 AM, Jürgen Cruz <supe...@gmail.com> wrote:

I think that is precisely the reason Jake Warthon discontinued his plugin and went full instrumentation tests. I agree with Jake that one should use Instrumentation tests in emulator since they are closer to reality than third party frameworks like Robolectric.

I still write unit tests that barely touch Android code (if at all) as unit tests should. They are just invoked by the instrumentation tests now because it's supported and I needed one less thing to maintain. People who rely on Robolectric to behave 100% like Android are writing their unit tests wrong.

I cannot describe how annoying it is to have to run tests on a device or Genymotion, though. It makes me not want to write tests. I run them once a week an tend to ignore failures for quite some time. What a stark contrast "modern" Android development has become to previously automatically failing pull requests unless all tests pass.

But I still consider a lot of pure Java classes exist in an Android app that could benefit from JUnit tests.

This.

Even code that barely touches Android (e.g., Bundle, Intent) should be unit testable via mocks. 

[Xavier Ducrohet]

It's impossible to run Android Junit tests on the host machine, in a way that's reliable. However you can run "junit" tests on a device, and this is what the Gradle plugin supports(*)

However, robolectric is a solution that could be considered "good enough" for some so we'll provide hooks for this. What those hooks are is a bit in flux right now.

Is it possible to reuse the same sourceSets that contains the "android" tests(*) and run them either on a device or robolectric? I don't think that's the case right now, but I think it would be a nice goal. If that were to happen, yes you would only maybe need to setup the right dependencies and make it run.

Right now though that won't work, and so you'll need to create a new sourceSet (which you can't right now because of the java/android plugins incompatibility) and run them separately. This is where the new plugin hooking up into the android one will facilitate this, dealing with variants and stuff.

(*) Let me be more clear about things. Unit tests are important, we support them. It's just you need to run them on a device. Running them on a different VM/Environment that is the actual target of the app is non trivial, and unless we have a good solution, we won't support it out of the box.

I think there's some confusion due to our sourcesets and dependency configurations naming (instrumentTest). We are planning to renaming these "androidTest" as they actually include Unit tests, instrument tests, and later uiautomator tests.

On Tue, Feb 11, 2014 at 11:38 AM, Jürgen Cruz <supe...@gmail.com> wrote:

--

You received this message because you are subscribed to the Google Groups "adt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adt-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Xavier Ducrohet]

On Tue, Feb 11, 2014 at 11:54 AM, Diego Costantini <diego.co...@gmail.com> wrote:

Hi xavier, thanks for the clarification and the look ahead :)

I understand that if I want robolectric:

1 - I should absolutely not have it in instrumentTest with robotium or espresso (I expected that much)

2 - the android plugin will not provide any help, which may or may not be provided by some additional independent plugin instead. If this is what's coming (or not), I hope that Jake will reconsider discontinuing his deprecated plugin...

Perhaps I will try to go back experimenting a bit more with Jake's plugin.

About the dependency order, definitely something could be "less" wrong, but the problem must be somewhere.

During my efforts in the last 3 weeks I found out (at least) the following 4 issues:

1 - I had conflicting dependencies, and some of them (ASM 4.1 vs. 3.1) hit me only at runtime if not in a particular order. These and other dependencies have been tamed through group/module exclusion in gradle.

The problem here is that ASM 3.x and 4.x are not compatible with each other. If you somehow depend on both in your dependency graph you are kind of stuck.

Was that a dependency from a plugin? I've seen this reported somewhere else? The problem is that the Gradle plugins aren't yet sandboxed and so they run with the same classpath and this triggers a conflict in that particular case.

 

2 - I needed to move junit >3.7 at the top, because some other dependency has an older version. This is a reason good enough to allow dependency order for junit testrunner classpath. And if Android is the one with that old junit, well it is pretty hard to go without it :)

3 - move robolectric dependency before android, otherwise it will use real implementations rather than the shadows (or whatever we call them). Again, this is how it works, this starts to make me think there might be nothing so wrong about wanting dependency order (btw: didn't intellij have one? is it conflicting somehow within the android studio implementation?)

4 - add my output classpath at the end, so my test classes can be found.

All of these ordering are due to having the same library on the classpath. as I said earlier, I think it would be fine to use the same sourcesets but compile them differently with different dependencies. One to generate a test apk, one to run on robolectric. 

 

In light of that, I can understand there might be other priorities for AS and gradle android plugin (and some of the issues might be solved by other plugins), but I believe these issues should be eventually addressed for a reasonable IDE.

We do really want to improve testing on Android. It's no simple task though and there are things happening, but it's going to take a couple of months.

 

On Thursday, February 6, 2014 4:46:16 PM UTC+1, Diego Costantini wrote:

Hi,
I am struggling to have robolectric + robotium in a multiproject in AS (I am completely new to it and IDEA so forgive me if I am missing some history).
I think the biggest problem is that I put everything into instrumentTest, and this way I have to compile all dependencies altogether (which makes issues depend on each other and appear exponentially :) ).
As far as I understood, the module dependencies settings in AS is currently under construction, because I keep finding workarounds from the past to the problems I have that suggest to move the dependencies order (for robolectric - example: move junit and robolectric on top, add output folder at the end, ...) and change some of them to provided instead of compile (for robotium - java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation).

My question is: in 0.4.4 is there any way to set them?

And more medium term: is there really work in progress to (1) eventually restore the AS settings to manipulate module dependencies and (2) make unit tests a first class citizen (perhaps by making java and android plugins compatible)?

Thanks for the huge work!

--

You received this message because you are subscribed to the Google Groups "adt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adt-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Diego Costantini]

Thanks guys, I see stuff is moving in the direction I hoped, I am sure we'll be happily testing soon enough. :)

Sent from my Nexus 4

You received this message because you are subscribed to a topic in the Google Groups "adt-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/adt-dev/Jj5nJsqR964/unsubscribe.
To unsubscribe from this group and all its topics, send an email to adt-dev+u...@googlegroups.com.

[Jürgen Cruz]

"It's impossible to run Android Junit tests on the host machine, in a way that's reliable."

I completely agree that it is impossible to run Android JUnit tests on the host machine. And it shouldn't be.

My idea would be more inclined to have this tasks:

Gradle Test Task: For Pure Java (And I mean PURE, if you insert a single Android reference It should blow)

Gradle Android Test Task : The current task that  handles the launching of the test app in the emulator and run instrumentation tests (And maybe some JUnit tests that touches Android classes)

check task: the task that would launch this other tasks like the current check task does.

now multiply the above task for the different combination of flavors, buildtype and variants and whatnot.

Now with the above strategy you could implement your tests in several flexible ways:

1.- write your code with wrappers that isolate android classes (MVP) and use only the Pure Java Test task and Hope the wrappers are fine.

2.- write your code with wrappers that isolate android classes (MVP) and use the Pure Java Test task for you Presenters and the Android Test task for your Views (The one I prefer since you could develop with TDD speed and still test your Views every once in a while).

3.- write your code extending the Android classes and use the both test tasks depending if the class touches the android framework.

4.- write your code extending the Android classes and use Robolectric in conjunction with the Pure Java Test task to test your app (Not recommended by Jake nor me).

5.- write your code however you want and use only the gradle android test task to test both instrumentationTests and JUnit tests (the current way).

Currently only the 5th use case is possible and the 1st and 2nd are only possible if you split your app in a pure java module (with Java plugin) and an android module (either Android library or android app plugins).

"Even code that barely touches Android (e.g., Bundle, Intent) should be unit testable via mocks. "

This would be awesome too, but one step at a time...