coverage of test-unit

[Ben Goodrich]

So, I did this

http://ltp.sourceforge.net/coverage/lcov.php

on test-unit to ostensibly see how well test-unit is covering the Stan code. The results (attached) can be seen by unzipping and pointing a web browser to lcov/index.html . They are a bit hard to interpret because lcov does everything relative to the subdirectories in STAN_HOME/test , but it is harder to interpret them that favorably with less than 66% coverage by line, less than 70% coverage by function, and less than 30% coverage by branch. Possibly the incomplete fwd stuff is dragging us down somewhat.

Ben

[Bob Carpenter]

These lcov reports look great (in terms of how they're presented,
not how Stan fared!).

I'm not surprised at the coverage. We should definitely
try for 100% coverage. As always, it's a matter of priorities.
I'd be happy to prioritize more testing and refactoring of
the code that we have. I'm going to start on gm/ast today,
which has very few tests currently.

I believe the next agrad/fwd pull request is blocked on
binomial_coefficient_log, where it's failing in the code branch
for inputs > 1000-ish.

Can we somehow restrict reports to src/stan/** suffixes?
Or remove anything with lib/**? I'm finding it hard to see
our files among all the Boost and Eigen files, which we don't
try to test.

Any idea how lcov decides what is covered? Some of the
things that are listed as not covered, like agrad/rev/op,
are base classes that are used all over the place in tests.
We should make mock objects that extend them and test those.
So the question is how we would make sure that counts as
coverage.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Ben Goodrich]

On Sunday, July 7, 2013 2:03:21 PM UTC-4, Bob Carpenter wrote:

Can we somehow restrict reports to src/stan/** suffixes?
Or remove anything with lib/**?  I'm finding it hard to see
our files among all the Boost and Eigen files, which we don't
try to test.

There are a bunch of options, but I do not yet see how to do that.
 

Any idea how lcov decides what is covered?  Some of the
things that are listed as not covered, like agrad/rev/op,
are base classes that are used all over the place in tests.
We should make mock objects that extend them and test those.
So the question is how we would make sure that counts as
coverage.

I don't really know how gcov works or how lcov parses the gcov output. I do think everything is more intended to be used with a conventional binary program rather than a header only library that is tested by thousands of unit tests. But at least we have something now and should be able to measure our coverage progress.

Ben

[Daniel Lee]

Resurrecting an old thread.

Should we try to get code coverage running on Jenkins? Ben, what did you do to make it work? Could you resurrect it?

[Ben Goodrich]

On Monday, August 3, 2015 at 1:56:15 PM UTC-4, Daniel Lee wrote:

Resurrecting an old thread.

Should we try to get code coverage running on Jenkins? Ben, what did you do to make it work? Could you resurrect it?

I don't even remember doing this two years ago, but according to the link I sent

http://ltp.sourceforge.net/coverage/lcov.php

it is pretty much just a matter of adding --coverage to CFLAGS and generating the report at the end.

Ben

[Bob Carpenter]

That would be great to have full test coverage. I swear if
I ever start another project it'll start that way. I've been
trying to add complete coverage for everything I write going
forward, but we have a lot of backfilling to do.

- Bob

> For more options, visit https://groups.google.com/d/optout.