coverage support in Gradle 0.8+

[Carl-Gustaf Harroch]

I managed to get JaCoCo running within Gradle (preDex, postCompile) while running a manual "adb am instrument..." as Exec command. This suits my needs but was wondering what the current status is? I could provide a patched version of DeviceProviderInstrumentTestTask that would add instrumentation functionality as part of the testOptions similar to  testOptions { coverage: true }. This could be extended to take any argument as defined here. Is this a work in progress ATM?

[Xavier Ducrohet]

We haven't started work there. We would definitively accept contributions. thanks.

On Mon, Mar 17, 2014 at 3:19 PM, Carl-Gustaf Harroch <char...@gmail.com> wrote:

I managed to get JaCoCo running within Gradle (preDex, postCompile) while running a manual "adb am instrument..." as Exec command. This suits my needs but was wondering what the current status is? I could provide a patched version of DeviceProviderInstrumentTestTask that would add instrumentation functionality as part of the testOptions similar to  testOptions { coverage: true }. This could be extended to take any argument as defined here. Is this a work in progress ATM?

--
You received this message because you are subscribed to the Google Groups "adt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adt-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Xavier Ducrohet
Android SDK Tech Lead
Google Inc.
http://developer.android.com | http://tools.android.com

Please do not send me questions directly. Thanks!

[Carl-Gustaf Harroch]

Xavier, I have an initial Pull request available against latest idea33 to provide instrumentaiton options to the plugin. I pushed it to Github to give visibility and discussion:

https://github.com/novoda/android-tools-base/pull/2

It does touch on a couple of files and notions (especially the idea of DeviceAction which could be explored further). 

The actual code coverage is done separately here and is still being worked on:

https://github.com/novoda/gradle-android-jacoco-plugin

Let me know your thoughts. Happy to push it back into aosp if it meets criteria.

[Xavier Ducrohet]

I would rather see this been done directly in AOSP than on github and then moved to AOSP so that we can keep all the discussion on the same place.

I'm guessing the deviceaction is used by your jacoco plugin?

We're going to work on code coverage very soon, so this is timely, but be aware that we're thinking about focusing first on jacoco support (emma seems to be mostly abandoned at this point).

We may want to offer hooks for other third party solutions but we'll probably want jacoco to be built-in. What's your end goal with your current plugin?

[Carl-Gustaf Harroch]

Sure, I can push to AOSP. We generally use github for code review internally and have not pushed to AOSP.

The deviceAction is mainly used to grab the coverage file prior to the uninstallation of the APK. The file by default is held within /data/data/<package>/files/coverage.ec. Given the SimpleTestCallable uninstalls the app, we need a way to grab data prior to the uninstallation. Ideally I would have it somewhere generic on the task side. 

The JaCoCo plugin is fully functional and we tested it on a rather large project with instrumentation and espresso tests. The current JaCoCo plugin could be pushed into the AOSP code but I could see some reasoning on having it separated. If you rather have one PR for the JaCoCo instrumentation, I could do so.

PS: to get Android full support to JaCoCo there is a need to change the default InstrumentationTestRunner.java if one wants to remove deprecated EMMA coverage (http://grepcode.com/file/repository.grepcode.com/java/ext/com.google.android/android/4.4.2_r1/android/test/InstrumentationTestRunner.java/#InstrumentationTestRunner.generateCoverageReport%28%29). Currently JaCoCo wrap that call to get JaCoCo coverage without the need to change the runner (https://github.com/jacoco/jacoco/blob/master/org.jacoco.agent.rt/src/com/vladium/emma/rt/RT.java). Ideally one would change the runner but that is another task in its own right.

[Xavier Ducrohet]

Hi,

I started looking at offering extension points for your need, and realized that the only way to be sure was to have an actual use case of the extension points.

We also had a somewhat urgent need for this with an internal team moving to Gradle and nagging me about proper coverage support.

In the end, this it wasn't actually a lot of work after the original investigation, I ended up implementing code coverage (with Jacoco). It's built-in right now though I want to move it out later so that anyone can plugin their own solution if they want to use something else (yes, this would require changes to the test runner).

The Jacoco plugin/extension are currently separate (But automatically applied) but the rest isn't, though it shouldn't be too hard to do when we decide to add an API to open it up.

The code is here: https://android-review.googlesource.com/#/c/91712 and this will be part of the plugin version 0.10.

[Anders Aagaard]

Awesome. We've got sonar code quality for my entire company, and the Android apps look quite bad due to missing coverage metrics. I've been looking forward to this! :)

[ca...@novoda.com]

That s great news. I had quite a bit of code into my branch against AOSP but could not finalise it properly. I am taking a look at your PR and have couple of comments. Latest JaCoCo is 0.7-SNAPSHOT which seems to work better when it comes to instrumentation. I don't remember the specific of it but it would be beneficial to easily change the JaCoCo version. The agent obviously need to be mapped to the same version running on device.

I would also argue that having a map of key/value for instrumentation options is beneficial in may ways. Especially that is how one can pass information between the build tool/runner and the InstrumentationRunner on device. CoverageFile and others could be modified but also the likes of running single test/methods/classes could be pushed to that by the IDE.

You received this message because you are subscribed to a topic in the Google Groups "adt-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/adt-dev/xr7hnrHKYmI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to adt-dev+u...@googlegroups.com.

[Xavier Ducrohet]

It's definitively not finished. Adding optional instrumentation options would definitively be good.

Passing single test/method/classes is something we want to do for sure. I'd like to reuse the same command options that Gradle has (--tests). See here: http://www.gradle.org/docs/current/userguide/java_plugin.html#sec:java_test

[ca...@novoda.com]

interesting, I was looking at something similar when implementing the coverage support. Given time, I might try to build upon what is implemented. 

Regards, Carl-Gustaf Harroch | CTO @ Novoda.com
Tel: +44 (0) 7 725 721 005

Disclaimer: The information in this e-mail and any attachments is the property of Novoda Ltd and is confidential and may be legally privileged. This e-mail is intended solely for the person or organisation to which it is addressed. Any disclosure, copying or other use of the information by any person or organisation who is not the intended recipient is strictly prohibited and may be unlawful. If you have received this e-mail in error, please inform the sender immediately and delete/destroy this e-mail and any copies of it. Novoda Ltd has taken reasonable precautions to minimise the risk of any software viruses which may damage your systems, but we advise that you take the necessary steps to ensure that no virus contamination is suffered. Novoda Ltd does not accept any liability for any loss or damage caused by the transmission of any virus.

Novoda Ltd, Company No: 347444, Registered in Scotland Registered Office: C/O Alexander Sloan, 38 Cadogan Street, Glasgow, G2 7HF, Scotland.  VAT Registration Number  GB 984 2525 93    

[Sachin Goel]

Hi,

I also tried the similar approach for my library project. I used gradle 1.10+ and latest SDK tool and Android Studio.

testCoverageEnabled = true for the debug build version

When I am running the tests through the terminal, I am getting the report of the test results but the testCoverage report is not being generated. 

Kindly let me know if anything else needs to be done.

Thanks and Warm Regards,

sachin

[Nico Küchler]

Which task use you to run android tests?

Current only 'connectedCheck' generate coverage

[mister ligi]

as soon as I try to enable

 testCoverageEnabled = true

I get the following exception

:connectedAndroidTestNoMapsNoAnalyticsForFDroidDebug

Tests on 4.3_atom(AVD) - 4.3 failed: Instrumentation run failed due to 'java.lang.VerifyError'

02:49:23 E/Device: Error during Sync: Remote object doesn't exist!

null

java.io.IOException: com.android.ddmlib.SyncException: Remote object doesn't exist!

at com.android.builder.testing.ConnectedDevice.pullFile(ConnectedDevice.java:114)

at com.android.builder.internal.testing.SimpleTestCallable.call(SimpleTestCallable.java:158)

at com.android.builder.internal.testing.SimpleTestCallable.call(SimpleTestCallable.java:42)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

Caused by: com.android.ddmlib.SyncException: Remote object doesn't exist!

at com.android.ddmlib.SyncService.pullFile(SyncService.java:314)

at com.android.ddmlib.Device.pullFile(Device.java:849)

at com.android.builder.testing.ConnectedDevice.pullFile(ConnectedDevice.java:107)

... 8 more

:connectedAndroidTestNoMapsNoAnalyticsForFDroidDebug FAILED