Code coverage like BullseyeCoverage for NDK

[Adrian Vintu]

Hi all,

Is anyone using some kind of code coverage tools for the NDK C++
builds? We were looking for something like BullseyeCoverage but
unfortunately it does not support Android http://www.bullseye.com/platform.html

Has anyone experimented with this?

Thank you,
Best regards,
Adrian

[Nikolai Kondrashov]

On 07/07/2011 01:06 PM, Adrian Vintu wrote:
> Is anyone using some kind of code coverage tools for the NDK C++
> builds? We were looking for something like BullseyeCoverage but
> unfortunately it does not support Android http://www.bullseye.com/platform.html
>
> Has anyone experimented with this?

We're using gcov/lcov quite successfully. It should support C++, but
we're using it with C only at the moment.

There are a few hacks required, though.

Sincerely,
Nick

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------

[Nikolai Kondrashov]

On 07/08/2011 02:29 PM, Adrian Vintu wrote:
> Are the tests running on device?
>
> Can I please get a little bit more info on the hacking required? Maybe
> you have some links, articles?
Here is our version of lcov, supporting Android toolchain:
https://github.com/spbnick/lcov-android/ - use "android" branch.

[Nikolai Kondrashov]

Adrian,

let's share our discussion with the list.

BTW, I just remembered that you need the atexit trick only if you use
dlopen/dlclose.

I think I will push our version of lcov to github soon.

Sincerely,
Nick

-------- Original Message --------
Subject: Re: Code coverage like BullseyeCoverage for NDK
Date: Fri, 8 Jul 2011 13:42:02 +0200
From: Nikolai Kondrashov <nkond...@nvidia.com>
To: Adrian Vintu <adria...@gmail.com>

On 07/08/2011 02:29 PM, Adrian Vintu wrote:
> Are the tests running on device?

Sure.

> Can I please get a little bit more info on the hacking required? Maybe
> you have some links, articles?

There are many articles regarding general gcov and lcov usage. However, I
don't know of any Android-specific articles.

The problem is that Android uses its own toolchain and it has modifications
WRT code coverage instrumentation.

If you plan to get coverage data only for userspace code, then you will only
need to modify lcov a bit.

If you need kernel code coverage data, then you will need kernel patches as
well.

We haven't yet released neither of these, and I don't know if it's possible,
but I'll check. However, lcov fixes are pretty easy to do.

Then there are bugs in gcov instrumentation libraries in some toolchain
versions. And you will have trouble with getting coverage out of shared
libraries, because of bionic handling of __cxa_atexit and __dso_handle, but
this one is solvable by force-linking your own atexit implementation.

If you are still interested, I think I could provide you with at least some
details.

[Nikolai Kondrashov]

Hi Adrian,

Please, keep the discussion on the list. Use "Reply to all".

On 07/12/2011 11:36 AM, Adrian Vintu wrote:
> I think I need a rooted phone, in order to "sudo make install" the
> packages - is this correct?
I'm sorry, I don't know. I work on development boards - they don't need
rooting. And I'm not sure what you mean by "sudo make install".

> I mean, I am installing lcov to the phone, right?
You don't need it there, you don't need any tool there. You only need to
build your programs with -fprofile-arcs -ftest-coverage and link with -lgcc
-lgcov.

> Where does the compilation take place? On the device or off target? If
> it's off target, how do I hook the gcc flags into the Android toolchain?
You can modify your Android.mk's. For example, add this:
LOCAL_CFLAGS += -fprofile-arcs -ftest-coverage
LOCAL_LDFLAGS += -lgcc -lgcov

However, you will need to understand how it works in the end in order to
maintain it.

[Adrian Vintu]

Hi Nick,

Sorry about not replying to all, I think gmail changed something in their interface/default behavior.

Good news, I managed to get coverage and it looks splendid :)

One tip:
when setting GCOV_PREFIX_STRIP=1 I got no files.
setting GCOV_PREFIX_STRIP=4 writes the files to GCOV_PREFIX/trunk/projectA/app/obj/src/...xyz.gcda
setting GCOV_PREFIX_STRIP=123 writes the files to GCOV_PREFIX/xyz.gcda - i.e. flattens the directory hierarchy

I found the values 4 and 123 by luck...

I have one question for you: I am trying to deploy a shared lib in an apk and cover it. I do not get any coverage files - also, no exceptions when building/running.

You mentioned there is some trouble when covering a shared lib. Can you please detail a little bit?

Thank you for all your help,
Having coverage is just the cherry on top of the cake :)

Best regards,
Adrian

[Nikolai Kondrashov]

Hi Adrian,

On 07/18/2011 12:16 PM, Adrian Vintu wrote:
> Good news, I managed to get coverage and it looks splendid :)

Great :)!

> One tip:
> when setting GCOV_PREFIX_STRIP=1 I got no files.
> setting GCOV_PREFIX_STRIP=4 writes the files to
> GCOV_PREFIX/trunk/projectA/app/obj/src/...xyz.gcda
> setting GCOV_PREFIX_STRIP=123 writes the files to GCOV_PREFIX/xyz.gcda -
> i.e. flattens the directory hierarchy
>
> I found the values 4 and 123 by luck...

I'd say you should have just read the documentation instead of relying on
luck :)

> I have one question for you: I am trying to deploy a shared lib in an
> apk and cover it. I do not get any coverage files - also, no exceptions
> when building/running.

Check that the library source code gets compiled with the correct options.
Check that you have gcov symbols in your library by running
"nm library.so | grep gcov".

> You mentioned there is some trouble when covering a shared lib. Can you
> please detail a little bit?

Unless you use dlopen/dlclose to link it at runtime, while using atexit in
the library at the same time, there should be no problem.

> Thank you for all your help,
> Having coverage is just the cherry on top of the cake :)

You're welcome :)

[Adrian Vintu]

Hi Nick,

See my comments in blue below.

As always, many many tanks on your help :)

Best regards,
Adrian

On Tue, Jul 19, 2011 at 12:40 PM, Nikolai Kondrashov <nkond...@nvidia.com> wrote:

Hi Adrian,

On 07/18/2011 12:16 PM, Adrian Vintu wrote:

Good news, I managed to get coverage and it looks splendid :)

Great :)!

One tip:
when setting GCOV_PREFIX_STRIP=1 I got no files.
setting GCOV_PREFIX_STRIP=4 writes the files to
GCOV_PREFIX/trunk/projectA/app/obj/src/...xyz.gcda
setting GCOV_PREFIX_STRIP=123 writes the files to GCOV_PREFIX/xyz.gcda -
i.e. flattens the directory hierarchy

I found the values 4 and 123 by luck...

I'd say you should have just read the documentation instead of relying on

luck :)

well, the documentation suggests values 1 and 4. 1 did not work for me - I have no idea why. number 123 is just perfect - it's not in the documentation. could be that my google skills are failing me, if there is a good explanation of this flag I would love to have it.

I have one question for you: I am trying to deploy a shared lib in an
apk and cover it. I do not get any coverage files - also, no exceptions
when building/running.

Check that the library source code gets compiled with the correct options.
Check that you have gcov symbols in your library by running
"nm library.so | grep gcov".

I see what you mean. I will give it a go.
 

You mentioned there is some trouble when covering a shared lib. Can you
please detail a little bit?

Unless you use dlopen/dlclose to link it at runtime, while using atexit in
the library at the same time, there should be no problem.

One lib seems to use dlopen/dlclose and another lib atexit - should not be a problem I guess.
 

[Nikolai Kondrashov]

On 07/19/2011 02:25 PM, Adrian Vintu wrote:
>> I'd say you should have just read the documentation instead of
>> relying on
>> luck :)
> well, the documentation suggests values 1 and 4. 1 did not work for me -
> I have no idea why. number 123 is just perfect - it's not in the
> documentation. could be that my google skills are failing me, if there
> is a good explanation of this flag I would love to have it.

GCOV_PREFIX and GCOV_PREFIX_STRIP is a typical pair of variables used to
relocate files. GCOV_PREFIX_STRIP is the number of directory components to
be removed from the start of the source file paths and GCOV_PREFIX is a
prefix to be added to the paths.

So, if your DIR_OUT was /home/adrian/android/out, for example and you wish
to put your files under /data/coverage on the target device, you could
define these variables as such:

# Remove four components, i.e. /home/adrian/android/out
export GCOV_PREFIX_STRIP=4
# Prepend /data/coverage
export GCOV_PREFIX=/data/coverage

Thus an object file on host named
/home/adrian/android/out/debug/target/product/blah/obj/EXECUTABLES/gzip_intermediates/minigzip.o
will get the coverage data output to
/data/coverage/debug/target/product/blah/obj/EXECUTABLES/gzip_intermediates/minigzip.gcda
on the device.

[Adrian Vintu]

Hi Nick,

I am having troubles getting coverage from an apk. I have checked that the debug/coverage .so files are included, but I cannot get any feedback.

In logcat I have seen that java!? uses dlopen to load the libs. You mentioned that some tweaking needs to be done in this case. Can you please detail on that?

Thank you,
Best regards,
Adrian

[Nikolai Kondrashov]

On 08/08/2011 03:21 PM, Adrian Vintu wrote:
> I am having troubles getting coverage from an apk. I have checked that
> the debug/coverage .so files are included, but I cannot get any feedback.
>
> In logcat I have seen that java!? uses dlopen to load the libs. You
> mentioned that some tweaking needs to be done in this case. Can you
> please detail on that?

The problem is that glibc remembers which library registered which atexit
handler and calls them on appropriate dlclose calls. Libgcov relies on this.
Bionic doesn't register this relation (although it is perfectly capable of
storing and checking it). As a result, gcov atexit handler gets called only
when the main program exits, i.e. when the handler is gone already.

There is a way to workaround the segfault by force-linking atexit
implementation, which behaves similarly to glibc, to all of your shared
libraries.

Like this one:

#include <stddef.h>

extern int __cxa_atexit(void (*func)(void *), void *arg, void *dso);

extern void *__dso_handle;

int
atexit(void (*func)(void))
{
return __cxa_atexit((void (*)(void *))func, NULL, &__dso_handle);
}

Make a library of this and then force-link it to your shared libraries with:

LOCAL_WHOLE_STATIC_LIBRARIES += libatexit

[Chris Stratton]

On Tuesday, July 19, 2011 8:44:19 AM UTC-4, Nikolai Kondrashov wrote:

So, if your DIR_OUT was /home/adrian/android/out, for example and you wish
to put your files under /data/coverage on the target device, you could
define these variables as such:

# Remove four components, i.e. /home/adrian/android/out
export GCOV_PREFIX_STRIP=4
# Prepend /data/coverage
export GCOV_PREFIX=/data/coverage

Jumping in rather late, but thought I'd point out that /data/coverage is not a path prefix which unprivileged (ie, apk-delivered) code is likely to be allowed to write to on a real device.   Allowable paths would be something under the app's files directory or the external storage (with appropriate manifest permission).  Since these are slightly device-dependent, it would probably be necessary to build with approximate knowledge of the specific target device.

[Nikolai Kondrashov]

On 08/08/2011 08:39 PM, Chris Stratton wrote:
> Jumping in rather late, but thought I'd point out that /data/coverage is
> not a path prefix which unprivileged (ie, apk-delivered) code is likely
> to be allowed to write to on a real device.

Thanks, Chris. I didn't have the chance to develop apps for Android yet.
Sorry if there is any confusion.

[Adrian Vintu]

Many thanks Nick, I will try this and let you know on the result.

Best regards,
Adrian

[Praveen Jain]

 

Hi

 

I was looking for a code coverage tool for Android apps and I foud this group.

I know it is not for app but atleast if you can point me to right direction.

 

Please help me with suggestions / tool.

My app has native code also.

 

Regards

-Praveen

[Evgeny]

Hi Everyone,

I'm trying to get coverage for native Android code as follows:

1. Build app with '-fprofile-arcs', '-ftest-coverage' LOCAL_CFLAGS flags and 'lgcov', 'lgcc' LOCAL_LDFLAGS flag (gcno files are created during this process)
2. Run app on Android API16 emulator (need exacltly this API version) (gcda files are created on the emulator)
3. pull files using adb such a way gcda and corresponding gcno are in the same directory
4. run customised lcov mentioned there (android branch)

As a result I get for every coverage-related file:

*.gcno:version '404*', prefer '408*'
*.gcda:version '404*', prefer '408*'
geninfo: WARNING: gcov did not create any files for ... *.gcda

gcov and gcc versions are the same.

Are my steps correct? If steps are correct, have you any idea of how to fix these issues?

[Anu Varshini]

Could someone please tell me the steps to implement code coverage (gcov) in android O.