Using NYTProf for code coverage?

[Mark Stosberg]

Greetings,

I was tasked with working on code coverage for a large project, but had
difficulty getting Devel::Cover to run. We had Devel::NYTProf handy,
and I realized that although that tool focuses on profiling, it produces
data that appears that it can be used for code coverage instead.

Having one tool that could be used for both profiling and code coverage
seems like a nice win. Have other people looked into using
Devel::NYTProf this way? Is there a reason why it would be undesirable?
Is Devel::Cover still the go-to tool for code coverage?

Below are some notes on how I used Devel::NYTProf for code coverage.

Mark

###

I was able to produce a report which looked like this:

0 0 Config::compile_date
0 0 Config::config_re
0 0 Config::config_sh
0 0 Config::config_vars
0 0 Config::header_files
0 0 Config::launcher
0 0 Config::local_patches
0 0 Config::myconfig
0 0 Cwd::chdir
0 0 Cwd::fast_abs_path
0 0 Data::Dumper::qquote
1 1 Cwd::getcwd
1 1 JSON::XS::encode
7 1 File::Find::CORE:closedir

It's the number of calls, places called from, and the subroutine.

To generate, in Apache add the following:

MaxClients 1
MaxRequestsPerChild 0

and this to the mod_perl startup.pl:

my $time = `/bin/date +%Y-%m-%dT%H.%M.%S.%N`;
chomp($time);
$ENV{NYTPROF} =
"file=/tmp/nytprof/nytprof.$time.out:addpid=1:endatexit=1";

require Devel::NYTProf::Apache;

Next, run your web request and a file should be created for parent and
child:

nytprof.2013-02-08T13.18.18.209426295.out.31929
nytprof.2013-02-08T13.18.18.209426295.out.31949

Now, merge the files: nytprofmerge -v --out=nytprof-merged.out
nytprof.2013-02-0*.

Create the report: covverage.pl --file nytprof-merged.out --out
merged_coverage --minimal and parse the data. Here is a script to parse:

cat *.tsv | perl -ne 'print if /\t/' | perl -a -ne
'print("$F[0]\t$F[1]\t$F[-1]\n") if $F[0] =~ /\d/' | sort -k 2,2 | sort
-u | grep -v ::BEGIN | grep -v __ANON | sort -n

This should give you something like the above.

[David Cantrell]

On Wed, Feb 27, 2013 at 08:49:44AM -0500, Mark Stosberg wrote:

> I was tasked with working on code coverage for a large project, but had
> difficulty getting Devel::Cover to run. We had Devel::NYTProf handy,
> and I realized that although that tool focuses on profiling, it produces
> data that appears that it can be used for code coverage instead.
>
> Having one tool that could be used for both profiling and code coverage
> seems like a nice win. Have other people looked into using
> Devel::NYTProf this way? Is there a reason why it would be undesirable?
> Is Devel::Cover still the go-to tool for code coverage?

Devel::Cover tells you a lot more than just whether a line of code was
executed. Consider a line like this:

do_something if($foo || $bar);

Devel::Cover will tell you which of the three possible conditions were
covered by your tests, those being:
* neither $foo nor $bar are true;
* $foo is false, $bar is true;
* $foo is true and we don't care about $bar

I'm surprised that you can bet NYTProf working but not Cover - in my
experience, they either both work easily or both fail horribly.

--
David Cantrell | Bourgeois reactionary pig

Vegetarian: n: a person who, due to malnutrition caused by
poor lifestyle choices, is eight times more likely to
catch TB than a normal person

[Mark Stosberg]

On 02/28/2013 06:02 AM, Christian Walde wrote:

> On Wed, 27 Feb 2013 14:49:44 +0100, Mark Stosberg <ma...@summersault.com>
> wrote:
>
>> I was tasked with working on code coverage for a large project, but had
>> difficulty getting Devel::Cover to run.
>

> Can you maybe go into details as to why it won't run, or maybe condense
> it to a repeatable case? PJCJ is, with help from others, actively
> working on D::C and fixing bugs, so reports of broken things are quite
> useful. :)

Thanks for all the replies. Sounds like the consensus is that I should
expect Devel::Cover to work, and get help if it doesn't. I'll give it
another shot.

I appreciated David Cantrell's additional detail about the additional
value that Devel::Cover provides.

Mark

[Paul Johnson]

On Thu, Feb 28, 2013 at 10:19:57AM -0500, Mark Stosberg wrote:

> Thanks for all the replies. Sounds like the consensus is that I should
> expect Devel::Cover to work, and get help if it doesn't. I'll give it
> another shot.

You can get help here or, if you've narrowed things down to a bug,
please post it to github.

--
Paul Johnson - pa...@pjcj.net
http://www.pjcj.net

[Christian Walde]

On Wed, 27 Feb 2013 14:49:44 +0100, Mark Stosberg <ma...@summersault.com>
wrote:

> I was tasked with working on code coverage for a large project, but had
> difficulty getting Devel::Cover to run.

Can you maybe go into details as to why it won't run, or maybe condense it
to a repeatable case? PJCJ is, with help from others, actively working on
D::C and fixing bugs, so reports of broken things are quite useful. :)

--
With regards,
Christian Walde