[JIRA] (JENKINS-56510) Provide generic issue parser for custom tools

Issue Type:	Improvement
Assignee:	Ulli Hafner
Components:	warnings-ng-plugin
Created:	2019-03-11 21:47
Priority:	Minor
Reporter:	Derrick Gibelyou

I have a handful of custom tools to generate custom reports on my code base and/or the compiled results. Right now I am abusing the output format of PMD to get close to the output I want. However, it would be nice if I could generate a file that mapped directly to the fields used by the plugin.

This might also be useful for people who do not have admin access to write groovy parsers. They could do some local analysis, then generate this file format.

An example of the input file format:

 
                                                                <issues>
    <issue>
        <category>Category 1</category>
        <type>Type 1</type>
        <severity>WARNING_LOW</severity>
        <message>Message 1</message>
        <lineStart>0</lineStart>
        <lineEnd>0</lineEnd>
        <columnStart>0</columnStart>
        <columnEnd>0</columnEnd>
        <lineRanges/>
        <description>Description 1</description>
        <moduleName>Module 1</moduleName>
        <packageName>-</packageName>
        <fileName>-</fileName>
    </issue>
    <issue>
        <category>Category 2</category>
        <type>Type 2</type>
        <severity>WARNING_HIGH</severity>
        <message>Message 2</message>
        <lineStart>0</lineStart>
        <lineEnd>0</lineEnd>
        <columnStart>0</columnStart>
        <columnEnd>0</columnEnd>
        <lineRanges/>
        <description>Description 2</description>
        <moduleName>Module 2</moduleName>
        <packageName>-</packageName>
        <fileName>-</fileName>
    </issue>
</issues>
</code>
 
                                                            

If this is of general interest I can post a PR.

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

ullrich.hafner@gmail.com (JIRA)

unread,

Mar 11, 2019, 7:32:03 PM3/11/19

to jenkinsc...@googlegroups.com

Ulli Hafner edited a comment on

This is a good idea! If the parser would be part of warnings-ng (and not analysis-model) then Jenkins XML reading logic (XStream) could be used, see [AnalysisResult.java ]( | https://github.com/jenkinsci/warnings-ng-plugin/blob/master/src/main/java/io/jenkins/plugins/analysis/core/model/AnalysisResult.java#L471 ) ] .

ullrich.hafner@gmail.com (JIRA)

unread,

Mar 11, 2019, 7:32:03 PM3/11/19

to jenkinsc...@googlegroups.com

Ulli Hafner commented on

This is a good idea! If the parser would be part of warnings-ng (and not analysis-model) then Jenkins XML reading logic (XStream) could be used, see [AnalysisResult.java](https://github.com/jenkinsci/warnings-ng-plugin/blob/master/src/main/java/io/jenkins/plugins/analysis/core/model/AnalysisResult.java#L471).

derrick.gibelyou@imsar.com (JIRA)

unread,

Mar 16, 2019, 11:57:02 PM3/16/19

to jenkinsc...@googlegroups.com

Derrick Gibelyou commented on

That is an interesting idea.

Could that cause problems since it would be de-serializing XML generated by a different tool? I don't have any experience with XStream. Does it handle missing fields well? I wouldn't expect the external tools to populate all the possible fields of an issue.

ullrich.hafner@gmail.com (JIRA)

unread,

Mar 17, 2019, 6:09:02 AM3/17/19

to jenkinsc...@googlegroups.com

Ulli Hafner commented on

Yes, it handles missing fields quite well. And the serialization fallbacks are under our control, see https://github.com/jenkinsci/warnings-ng-plugin/blob/master/src/main/java/io/jenkins/plugins/analysis/core/model/IssueStream.java.

ullrich.hafner@gmail.com (JIRA)

unread,

Mar 22, 2019, 10:31:02 AM3/22/19

to jenkinsc...@googlegroups.com

Ulli Hafner assigned an issue to Unassigned

Jenkins /

Change By:	Ulli Hafner
Labels:	help-wanted newbie-friendly
Assignee:	Ulli Hafner

ullrich.hafner@gmail.com (JIRA)

unread,

Mar 28, 2019, 5:39:02 AM3/28/19

to jenkinsc...@googlegroups.com

Ulli Hafner updated an issue

Jenkins /

Change By:	Ulli Hafner
Priority:	Minor Major

rfurch@hm.edu (JIRA)

unread,

Mar 28, 2019, 2:05:11 PM3/28/19

to jenkinsc...@googlegroups.com

Raphael Furch assigned an issue to Raphael Furch

Jenkins /

Change By:	Raphael Furch
Assignee:	Raphael Furch

rfurch@hm.edu (JIRA)

unread,

Apr 3, 2019, 11:12:02 AM4/3/19

to jenkinsc...@googlegroups.com

Raphael Furch commented on

Ulli Hafner I don't know exactly what to do here. Can you give me a hint please?

ullrich.hafner@gmail.com (JIRA)

unread,

Apr 3, 2019, 11:53:03 AM4/3/19

to jenkinsc...@googlegroups.com

Ulli Hafner commented on

We need a new parser class e.g. `AnalysisReport` (subclass of `ReportScanningTool`) in the warnings-ng project, see `Detekt` for an example. This class creates a new parser e.g. of new class `ReportParser` (extending `IssueParser`) in the `createParser` method. The `parse` method of this class should use the same code as `AnalysisResult.readIssues()`. The code in `AnalysisResult` already reads a report with several issues, so we basically need to reuse or extract that part.

ullrich.hafner@gmail.com (JIRA)

unread,

Apr 3, 2019, 11:55:02 AM4/3/19

to jenkinsc...@googlegroups.com

Ulli Hafner edited a comment on

ullrich.hafner@gmail.com (JIRA)

unread,

Apr 3, 2019, 11:56:02 AM4/3/19

to jenkinsc...@googlegroups.com

Ulli Hafner edited a comment on

# We need a new parser tool class e.g. {{AnalysisReport}} (subclass of {{ReportScanningTool}}) in the warnings-ng project, see {{Detekt}} for an example. This class creates a new parser in the {{createParser}} method.
# We need a new parser class e.g. of new class {{ReportParser}} (extending {{IssueParser}}) in the {{createParser}} method . The {{parse}} method of this class should use the same code as {{AnalysisResult.readIssues()}}. The code in {{AnalysisResult}} already reads a report with several issues, so we basically need to reuse or extract that part.

dev@jmini.fr (JIRA)

unread,

Apr 18, 2019, 2:29:02 AM4/18/19

to jenkinsc...@googlegroups.com

Jeremie Bresson commented on

I have similar needs, I have made a proposal for a JSON based format in JENKINS-57098.

ullrich.hafner@gmail.com (JIRA)

unread,

Apr 18, 2019, 6:30:01 PM4/18/19

to jenkinsc...@googlegroups.com

Ulli Hafner started work on

Change By:	Ulli Hafner
Status:	Open In Progress

ullrich.hafner@gmail.com (JIRA)

unread,

Apr 22, 2019, 12:41:02 PM4/22/19

to jenkinsc...@googlegroups.com

Ulli Hafner updated an issue

Jenkins /

Change By:	Ulli Hafner

I have a handful of custom tools to generate custom reports on my code base and/or the compiled results. Right now I am abusing the output format of PMD to get close to the output I want. However, it would be nice if I could generate a file that mapped directly to the fields used by the plugin.

This might also be useful for people who do not have admin access to write groovy parsers. They could do some local analysis, then generate this file format.

An example Current Serialization of the input file format an {{Issue}} instance using XStream :
{code:xml}
< issues ?xml version="1.1" encoding="UTF-8"? >
    <issue>
        <category> Category 1 category </category>
        <type> Type 1 type </type>
        <severity> WARNING_LOW HIGH </severity>
        <message> Message 1 message </message>
        <lineStart> 0 1 </lineStart>
        <lineEnd> 0 2 </lineEnd>
        <columnStart> 0 3 </columnStart>
        <columnEnd> 0 4 </columnEnd>
        <lineRanges >
    <lineRange>
      <start>5< / start >
        < end>6</end>
    </lineRange>
  </lineRanges>
  <id>63d61b1f-0cac-4e31-8bb9-f390ed1acfe8</id>
  < description> Description 1 description </description>
        < reference>reference</reference>
  <origin>origin</origin>
  < moduleName> Module 1 module-name </moduleName>
        <packageName> package - name </packageName>
        <fileName> file - name </fileName>
    < fingerprint>fingerprint< / fingerprint>
</ issue>
{code}

Current Serialization of a {{Report}} instance using XStream:

{code}
    < ?xml version="1.1" encoding="UTF-8"?>
<analysisReport>
  <elements class="linked-hash-set">
    < issue>
        <category> Category 2 category </category>
        <type> Type 2 type </type>
        <severity> WARNING_HIGH HIGH </severity>
        <message> Message 2 1 </message>
        <lineStart> 0 1 </lineStart>
        <lineEnd> 0 2 </lineEnd>
        <columnStart> 0 3 </columnStart>
        <columnEnd> 0 4 </columnEnd>
        <lineRanges >
        <lineRange>
          <start>5< / start >
        < end>6</end>
        </lineRange>
      </lineRanges>
      <id>c3b984a3-7f67-4332-953b-27d3943f232c</id>
      < description> Description 2 description </description>
        < reference>reference</reference>
      <origin>origin</origin>
      < moduleName> Module module-name</moduleName>
      <packageName>package-name</packageName>
      <fileName>file-name</fileName>
      <fingerprint>fingerprint</fingerprint>
    </issue>
    <issue>
      <category>category</category>
      <type>type</type>
      <severity reference="../../issue/severity"/>
      <message> 2</ message>
      <lineStart>1</lineStart>
      <lineEnd>2</lineEnd>
      <columnStart>3</columnStart>
      <columnEnd>4</columnEnd>
      <lineRanges>
        <lineRange>
          <start>5</start>
          <end>6</end>
        </lineRange>
      </lineRanges>
      <id>fbf2fee0-292f-4991-bd06-d8c5b13ace93</id>
      <description>description</description>
      <reference>reference</reference>
      <origin>origin</origin>
      < moduleName> module-name</moduleName>
        <packageName> package - name </packageName>
        <fileName> file - name </fileName>
    < fingerprint>fingerprint< / fingerprint>
    </ issue>
</ issues elements >
< infoMessages>
    <string>info< / code string >
  </infoMessages>
  <errorMessages>
    <string>error</string>
  </errorMessages>
  <duplicatesSize>0</duplicatesSize>
</analysisReport>
{code}

If this is of general interest I can post a PR.

ullrich.hafner@gmail.com (JIRA)

unread,

Apr 22, 2019, 12:42:02 PM4/22/19

to jenkinsc...@googlegroups.com

Ulli Hafner updated an issue

Jenkins /