[RFC] New wiki entry related to CMake/CTest/CDash

[Thomas Paviot]

Hi,

I just opened a new wiki entry: https://github.com/tpaviot/oce/wiki/Testing-suite.

It contains some basic explanations about running the testsuite, measuring code coverage and performing a dynamic analysis. For the latter, you'll need valgring installed (I uploaded such a dynamic analysis to http://my.cdash.org/index.php?project=OCE: you can see how exhaustive is the result!).

Feel free to edit/improve this page.

Thomas

[Thomas Paviot]

2011/8/27 Thomas Paviot <tpa...@gmail.com>

I uploaded to http://my.cdash.org/index.php?project=OCE a complete report including tests+code coverage+dynamic analysis.

The "Code covergae" report is very insteresting. It shows that our few tests cover up to 19% of OCE_MODEL (the tests cover 35158 OCE lines of code). A detailed report is available categorized in Low/Medium/Satisfactory code coverage. 'Low' coverage should be first adressed in the tests to develop.

Note that these reports have a short lifetime: only 24 hours. They'll be available untill tomorrow.

Thomas

[D. Barbier]

On 2011/8/27 Thomas Paviot <tpa...@gmail.com>:

This is really awesome, but 24 hours is definitely too short to
investigate those issues.
Do you know if the same reports can be send by email? Maybe we could
create another mailing list for this purpose?

Denis

[Thomas Paviot]

2011/8/27 D. Barbier <bou...@gmail.com>

I apologize, I was absolutely wrong: results are not deleted after 24h and the history is saved on the cdash.org server for *30* days. By default, when connecting http://my.cdash.org/index.php?project=OCE, you have the 'Current' tests ('Current' actually means 'today'). But you can browse previous results by clicking "Previous" or choose the day from the "Calendar" section (I discovered that  few minutes ago).

There is however a limitation with the free account I registered on mycdash.org: there is a maximum daily quota upload (as far as I remeber it's 100Mb, unfortunately I reached the quota for the last 24hours).
 

Do you know if the same reports can be send by email?

The XML files sent by CTest are available from the folder /Testing in your cmake-build directory. The DynamicAnalysis.xml file is human readable, however it's less convenient.
 

Maybe we could
create another mailing list for this purpose?

It would be really difficult to manage: each report is different according to the arch/compiler/mode(Release, debug etc.)/branch/.

If 30 days is enough for you, I suggest we go on with mycdash.org (unless you know any other free cdash server provider). Code coverage XML files are quite huge. We just have to take care not to submit more than one code coverage report per day and it should be ok (i.e. we would not go over the quota).
 

Denis

Thomas