valrgrind or xml reporter plugin

[Deuberger]

I'm trying to find a way to integrate xml output from running valgrind
into Hudson's reporting and notification. I couldn't find a plugin
that already supports this. Does anyone have any ideas that doesn't
involve writing a plugin from scratch?

Thanks.

- Daniel

[Jon Schewe]

I hacked together a script that turned each valgrind error into a junit
test failure. That works reasonably well.

[Deuberger]

On Jan 12, 9:58 pm, Jon Schewe <jpsch...@mtu.net> wrote:
> I hacked together a script that turned each valgrind error into a junit
> test failure. That works reasonably well.

Are you at liberty to share the script or an example of the junit xml
so that I can see the mapping? I'm working on developing an xsl file
to use with the Xunit plugin.

Thanks,

- Daniel

[Jon Schewe]

Attached is the output of valgrind for a simple memory error and the
junit output that I create from it.

--
Jon Schewe | http://mtu.net/~jpschewe
If you see an attachment named signature.asc, this is my digital
signature. See http://www.gnupg.org for more information.

[asmundo]

hi Jon and Daniel,

I am also interesting in interpreting the output of valgrind but I was
wandering if you had looked in to extending the warnings-plugin. IMHO
it is better match. I was wandering if any of you have done some
investigation in this direction. Unfortunately my java, plugin arch
knowledge is low.

Regards,
Asmund

[Deuberger]

On Feb 22, 11:36 am, asmundo <asmu...@gmail.com> wrote:
> I am also interesting in interpreting the output of valgrind but I was
> wandering if you had looked in to extending the warnings-plugin. IMHO
> it is better match.  I was wandering if any of you have done some
> investigation in this direction.  Unfortunately my java, plugin arch
> knowledge is low.

Thanks for the idea; I had not looked into that. Unfortunately, I
have not been able to work on this for a little while due to other
higher priorities. It is still on my list of things to do as soon as
I can get a breather though.

The current idea is to develop an XSL file to translate the Valgrind
XML output to Junit XML and use it with the XUnit plugin. The author
of the XUnit plugin is also interested in integrating the XSL with the
actual plugin so that it can support Valgrind out of the box. This
seems like the quickest approach for now, which means its the most
likely route forward.

In the long run it would be nice to have a plugin with more specific
support for Valgrind, which would probably require some Java coding.
This could possibly use/extend the suite that the warnings plugin is a
part of (http://wiki.jenkins-ci.org/display/JENKINS/Static+Code
+Analysis+Plug-ins), the warnings plugin itself, or another plugin
such as the plot plugin.

Whenever I get back around to this, I'll try to do a more complete
analysis of the best route forward (which in my case might be the
quickest route forward). Whatever I do, I'll post back here, but that
may take a while depending on how busy I stay with other priorities.

If you or someone else gets to this first though, please let me know.
I have not written any plugins myself, but my perception is that the
learning curve is low assuming you know Java.

Thanks for the idea!

- Daniel

[Åsmund Østvold]

Hi Daniel,

I have no time to work on this at the moment but I wanted to share with you my findings.  The actual parsing class should probably only be a variant of [1] (72 lines). And from "Adding support for new warning types" on [2] it looks like a recipe that I may be able to follow. Then again the devil is hidden in the details..... 

I will report back if I do start work on this. 

Regards,

Asmund

1] https://github.com/jenkinsci/warnings-plugin/blob/master/src/main/java/hudson/plugins/warnings/parser/GccParser.java

2] http://wiki.jenkins-ci.org/display/JENKINS/Warnings+Plugin

[Joris Koster]

Hi all,

 

I’m not sure if this was already pointed out in some earlier posts, but here http://www.tonicebrian.com/2010/10/15/continuous-integration-for-c-using-hudson/ you can find a python script that inspired me greatly J to have memory-analysis runs reported in Hudson, sorry, Jenkins. It might not be as nice as having XSL do the transforms, but practically this already works for me.

 

Regards,

Joris

 

-----Original Message-----
From: jenkins...@googlegroups.com [mailto:jenkins...@googlegroups.com] On Behalf Of Åsmund Østvold
Sent: Friday, February 25, 2011 9:16 AM
To: Deuberger
Cc: jenkins...@googlegroups.com
Subject: Re: valrgrind or xml reporter plugin

 

Hi Daniel,

 

I have no time to work on this at the moment but I wanted to share with you my findings. The actual parsing class shouldprobablyonly be avariantof [1] (72 lines). And from "Adding support for new warning types" on [2] it looks like a recipe that I may be able to follow. Then again the devil is hidden in the details.....

 

I will report back if I do start work on this.

 

Regards,

Asmund

 

1]https://github.com/jenkinsci/warnings-plugin/blob/master/src/main/java/hudson/plugins/warnings/parser/GccParser.java

2]http://wiki.jenkins-ci.org/display/JENKINS/Warnings+Plugin

 

 

On Thu, Feb 24, 2011 at 2:48 PM, Deuberger <daniel.n...@gmail.com> wrote:

 

CONFIDENTIALITY: This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium.

[Deuberger]

Joris, thanks for the link. It looks great and might be the quickest
way to get something up an running. I'll try it out as soon as I get
some time.

Asmund, thanks for sharing your findings and please do let me know if
you make some good progress. I'll do the same for you by posting back
here.

Also, I will probably still try to write the XSL when I get the chance
because that would be a nice simply integration. Lastly, it would
probably be a good idea to make a wiki page for Valgrind explaining
these different options and work that has already been done. If
someone else doesn't beat me to it, I'll try to do that when I get a
chance as well.

Thanks everyone!

- Daniel

[Daniel Neuberger]

Attached is an XSLT file for transforming Valgrind's memcheck XML
output to a JUnit similar format suitable for the Jenkin's XUnit
plugin.

Greg, I think you can integrate this with XUnit to add support for Valgrind.

Jon, thanks for your example; it was helpful to me in developing the XSLT.

Asmund, I still think making a plugin more tailored to valgrind could
be useful. That being said I think this was a much easier path
forward for now and in the future. I'd still be interested if you
make any progress though on your front.

Notes regarding the XSLT:
- The file is not 100% complete in the sense that it does fully
transform everything that could be in valgrind's xml output. That
being said, it should be fairly complete for most cases. See
/docs/internals/xml-output.txt from valgrind's source if anyone is
interested it catching the edge cases.
- The resulting format is in generally that of JUnit and works for the
XUnit plugin, but is not 100% valid. If anyone is interested in
filling in the gaps the correct schema is
http://windyroad.org/dl/Open%20Source/JUnit.xsd.
- I made the stack trace include links to the appropriate valgrind
documentation based on the error type so that users could quickly get
information on what the errors mean. There's no way to make the links
pretty because it's preformatted text, but Jenkins will at least turn
the URLs into links. If anyone notices problems with my mapping from
error kinds to documentation, please let me know.

I hope that is helpful. If anyone has suggestions for improvements or
notices bugs, please let me know. I'll also try to keep the group
updated if I make significant improvements.

Thanks to all!

- Daniel

[Grégory Boissinot]

Thanks Daniel for your initial XSL.
Regarding the transformation result: why don't you create a testcase
for each error section?

-
Grégory

[Deuberger]

On Apr 15, 6:19 pm, Grégory Boissinot wrote:
> Regarding the transformation result: why don't you create a testcase
> for each error section?

I did that initially, but then realized that causes an inconsistent
number of test cases in your Jenkins Test results and reports.
Sometimes it would look like you have way more tests than you do and
then if there were no errors, it would appear as if the tests were not
even being run. This way there should always be one valgrind test per
test executable. Also, in my experience, multiple errors are often
related and its useful to see them all in context with each other.

I'm for suggestions though. Thanks.

- Daniel