warnings-ng-plugin-devenv/analysis-model/bin/go.sh missing a step?

34 views
Skip to first unread message

Simon Matthews

unread,
Jun 2, 2022, 6:34:45 PM6/2/22
to Jenkins Developers
I have now got this to build and run in Jenkins (I installed the latest version of Docker and the docker-compose-plugin so that I could use the scripts without modification), however, when I modify the analysis-model (to add a parser) and run ./bin/go.sh in the analysis-model directory, the script seems to be missing a step.

The script has:
...
echo Copying module analysis-model version ${version} to analysis-model-api-plugin Jenkins plugin

cd ../analysis-model-api-plugin || { echo "Wrong directory"; exit 1; }

exec ./go.sh $version
...

Looking at the first and second second go.sh, neither appear to actually copy any files from the file "analysis-model-api.jar" (or any "analysis-model-*SNAPSHOT.jar" file)  to the JENKINS_HOME directory.

Note that the documentation refers to running "go.sh" in the analysis-model directory, but the only "go.sh" file in this directory is actually "./bin/go.sh".

Simon

Ullrich Hafner

unread,
Jun 3, 2022, 4:15:35 AM6/3/22
to JenkinsCI Developers
Am 03.06.2022 um 00:34 schrieb Simon Matthews <simon.d....@gmail.com>:

I have now got this to build and run in Jenkins (I installed the latest version of Docker and the docker-compose-plugin so that I could use the scripts without modification),

Good to see. I need to spend some time to investigate why docker compose does not work in the same way as docker-compose.

however, when I modify the analysis-model (to add a parser) and run ./bin/go.sh in the analysis-model directory, the script seems to be missing a step.

The script has:
...
echo Copying module analysis-model version ${version} to analysis-model-api-plugin Jenkins plugin

cd ../analysis-model-api-plugin || { echo "Wrong directory"; exit 1; }

exec ./go.sh $version
...

Looking at the first and second second go.sh, neither appear to actually copy any files from the file "analysis-model-api.jar" (or any "analysis-model-*SNAPSHOT.jar" file)  to the JENKINS_HOME directory.


The first go.sh calls mvn install that installs the jar in your local maven repository.

The second go.sh packages the jar from the local maven repository as Jenkins JPI file and deploys it to JENKINS_HOME using the deploy.sh command 

Note that the documentation refers to running "go.sh" in the analysis-model directory, but the only "go.sh" file in this directory is actually "./bin/go.sh".

Thanks for spotting. I forgot to update the documentation after reorganizing the scripts. Should be fixed now by https://github.com/uhafner/warnings-ng-plugin-devenv/commit/e6040034dfb7cffa1e761b12c0e007264380db8b


Simon

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5fbb2fb1-f9de-4b83-b8e9-84650fcd5ca0n%40googlegroups.com.

Simon Matthews

unread,
Jun 3, 2022, 2:35:36 PM6/3/22
to Jenkins Developers
On Friday, June 3, 2022 at 1:15:35 AM UTC-7 ullrich...@gmail.com wrote:

Looking at the first and second second go.sh, neither appear to actually copy any files from the file "analysis-model-api.jar" (or any "analysis-model-*SNAPSHOT.jar" file)  to the JENKINS_HOME directory.


The first go.sh calls mvn install that installs the jar in your local maven repository.

It looks like the Jenkins running under Docker is not picking up my modified version of analysis-model. I don't see the parser that I added.

I assume that I have missed a step somewhere.

I have added files for the <MyParser>SoftwareDescriptor.java, I have added the reference in the ParserRegistry.java, I have added the reference in ParsersTest.java and an appropriate log file.

In analysis-model, I have run "./bin/go.sh" and in warnings-ng-plug, I have run "./bin/go.sh", then in the warnings-ng-plugin-devenv directory, I have re-started "./jenkins.sh".

If I configure a job, I can add "Record compiler warnings and static analysis results" as a post-build step, but my new parser is not in the list. What could I have missed?

Simon

Simon Matthews

unread,
Jun 3, 2022, 8:49:44 PM6/3/22
to Jenkins Developers
When I re-ran and looked at the transcript a little deeper, I think that the parser I built is not testing properly, so maybe there is an issue with it. I'll have to investigate further. There is an error flagged, but it had scrolled off the screen and, at the end, I got the message that the build had completed. I'll go back to the start to see if I can figure it out. It's odd, because, in my stand-alone analysis-model directory, the tests ran clean with my new parser.

Simon

Ullrich Hafner

unread,
Jun 6, 2022, 5:46:19 PM6/6/22
to JenkinsCI Developers
The parser is not listed in the UI, because the parser is not registered in the warnings plugin yet. 
But you can invoke the parser already using:

recordIssues tool: analysisParser(pattern: 'your-report-file.xml', analysisModelId: 'your-id')
Does that work?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

Simon Matthews

unread,
Jun 6, 2022, 9:03:26 PM6/6/22
to Jenkins Developers

On Monday, June 6, 2022 at 2:46:19 PM UTC-7 ullrich...@gmail.com wrote:

Thanks for your reply.

To update on progress, I had not previously added the analysis-model/src/test/java/edu/hm/hafner/analysis/parser/<my new parser>ParserTest.java file. I have now added that and the unit tests are passing.

The parser is not listed in the UI, because the parser is not registered in the warnings plugin yet. 
But you can invoke the parser already using:

recordIssues tool: analysisParser(pattern: 'your-report-file.xml', analysisModelId: 'your-id')
Does that work?


It's not clear to me how I run that command, or what file I add it to.

Simon


Ullrich Hafner

unread,
Jun 7, 2022, 5:45:58 AM6/7/22
to JenkinsCI Developers
In your Jenkinsfile of your job where you want to use the parser. Or are you using a Freestyle project? 

I think in Freestyle projects I forgot to add a dynamic selection of the parser ID in the UI up to now. I should file an issue for this bug. In this case you need to register your parser in the warnings plugin as well...

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

Simon Matthews

unread,
Jun 7, 2022, 1:54:25 PM6/7/22
to Jenkins Developers
On Tuesday, June 7, 2022 at 2:45:58 AM UTC-7 ullrich...@gmail.com wrote:
In your Jenkinsfile of your job where you want to use the parser. Or are you using a Freestyle project? 

I think in Freestyle projects I forgot to add a dynamic selection of the parser ID in the UI up to now. I should file an issue for this bug. In this case you need to register your parser in the warnings plugin as well...

Can you tell me how to work around this? At least to show that my parser is working?

Simon

Ullrich Hafner

unread,
Jun 7, 2022, 3:10:22 PM6/7/22
to JenkinsCI Developers
Workaround 1: use a pipeline and not a Freestyle Job

Simon Matthews

unread,
Jun 7, 2022, 9:21:35 PM6/7/22
to Jenkins Developers
On Tuesday, June 7, 2022 at 12:10:22 PM UTC-7 ullrich...@gmail.com wrote:
Workaround 1: use a pipeline and not a Freestyle Job


I tried to follow  Workaround 2.

I created a file to register the tool in warnings-ng by creating the file: warnings-ng-plugin/plugin/src/main/java/io/jenkins/plugins/analysis/warnings/<my parser>.java

I copied another file from the same directory and modified to reference my parser.  I note that almost all the other files in this directory extend AnalysisModelParser, not ReportScanningTool, as the documentation example does, so I copied one that extends AnalysisModelParser and modified this.

The documentation also talks about a pom.xml file, but I could not find a similar example of a similar file. I am not sure if this is relevant to what I am trying to do.

I  started docker in the warnings-ng-plug-devenv directory using the "jenkins.sh" script and then ran "./go.sh" in this directory. It appeared to finish, although there is an error right at the end:
...
removed 'docker/volumes/jenkins-home/plugins/warnings-ng.jpi.version_from_image'
'analysis-model-api-plugin/target/analysis-model-api.hpi' -> 'docker/volumes/jenkins-home/plugins/analysis-model-api.jpi'
cp: cannot stat 'code-coverage-api-plugin/target/*hpi': No such file or directory

I have noticed that I appear to have 2 docker containers running:

simon@jenkins-u:/jenkins/warnings-ng-plugin-devenv$ docker container list
CONTAINER ID   IMAGE                                          COMMAND                  CREATED        STATUS          PORTS                                                                                             NAMES
bd4d0076d1eb   warnings-ng-plugin-devenv_java11-agent         "/docker-entrypoint.…"   26 hours ago   Up 35 minutes   22/tcp, 0.0.0.0:8001->8001/tcp, :::8001->8001/tcp                                                 warnings-ng-plugin-devenv-java11-agent-1
4e87949d4ec7   warnings-ng-plugin-devenv_jenkins-controller   "/sbin/tini -- /usr/…"   26 hours ago   Up 35 minutes   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 50000/tcp   warnings-ng-plugin-devenv-jenkins-controller-1

For good measure, I restarted the container with "docker-compose restart", but I still don't see my parser in the list.

Any ideas what I might have missed? Or is my mistake to extend AnalysisModelParser instead of ReportScanningTool?

My work on the analysis-model can be found in https://github.com/SimonMatthews-BP/analysis-model

Ullrich Hafner

unread,
Jun 8, 2022, 7:00:48 AM6/8/22
to JenkinsCI Developers
When you look at the installed plugins in your Jenkins, are there the correct snapshot versions deployed?

It should look like:


The analysis-model code looks good (it should not be part of your master branch, it should be be part of a new branch though).
The warnings plugin fork has no changes yet?


Simon Matthews

unread,
Jun 8, 2022, 12:30:03 PM6/8/22
to Jenkins Developers
Looks like I am missing the updated warnings-ng plugin:
Screenshot at 2022-06-08 09-28-02.png

Simon Matthews

unread,
Jun 8, 2022, 1:04:44 PM6/8/22
to Jenkins Developers
Update: I manually ran the copy:
cp -fv warnings-ng-plugin/plugin/target/*hpi $JENKINS_HOME/plugins/warnings-ng.jpi

Something is triggering the "set -e" in the deploy.sh script to make it exit before this copy happens.

Screenshot at 2022-06-08 09-57-06.png


Still no sign of my new parser:
Screenshot at 2022-06-08 10-01-35.png

Simon Matthews

unread,
Jun 8, 2022, 1:56:37 PM6/8/22
to Jenkins Developers
More on what is stopping deploy.sh

I see this error message, which I had erroneously ignored previously:
cp: cannot stat 'code-coverage-api-plugin/target/*hpi': No such file or directory

The directory 'code-coverage-api-plugin/target/' doesn't exist, yet everything looks good building the warnings-ng:

[INFO] --- maven-install-plugin:2.4:install (default-install) @ warnings-ng-plugin-devenv ---
[INFO] Installing /jenkins/warnings-ng-plugin-devenv/pom.xml to /home/simon/.m2/repository/edu/hm/hafner/warnings-ng-plugin-devenv/UNVERSIONED/warnings-ng-plugin-devenv-UNVERSIONED.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Java coding style 2.24.0-SNAPSHOT .................. SUCCESS [ 19.703 s]
[INFO] Static Analysis Model and Parsers 10.11.0-SNAPSHOT . SUCCESS [ 49.094 s]
[INFO] Analysis Model API Plugin 10.11.0-SNAPSHOT ......... SUCCESS [ 12.007 s]
[INFO] Code Coverage API Plugin 3.0.0-SNAPSHOT ............ SUCCESS [ 15.591 s]
[INFO] UI Tests of Code Coverage Plugin UNVERSIONED ....... SUCCESS [ 13.469 s]
[INFO] Aggregator for Code Coverage Plugin UNVERSIONED .... SUCCESS [  0.051 s]
[INFO] Forensics API Plugin 1.14.0-SNAPSHOT ............... SUCCESS [  9.788 s]
[INFO] Git Forensics Plugin 1.9.0-SNAPSHOT ................ SUCCESS [ 10.822 s]
[INFO] UI Tests of Git Forensics Plugin UNVERSIONED ....... SUCCESS [ 12.355 s]
[INFO] Aggregator for Git Forensics Plugin UNVERSIONED .... SUCCESS [  0.007 s]
[INFO] Warnings Next Generation Plugin 9.13.0-SNAPSHOT .... SUCCESS [ 44.076 s]
[INFO] UI Tests of Warnings Plugin UNVERSIONED ............ SUCCESS [ 15.580 s]
[INFO] Aggregator for Warnings Plugin UNVERSIONED ......... SUCCESS [  0.008 s]
[INFO] Warnings plugin development environment UNVERSIONED  SUCCESS [  0.006 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:27 min
[INFO] Finished at: 2022-06-08T10:51:29-07:00
[INFO] ------------------------------------------------------------------------
Removing plugins in docker/volumes/jenkins-home

Ullrich Hafner

unread,
Jun 8, 2022, 6:44:23 PM6/8/22
to JenkinsCI Developers

Am 08.06.2022 um 19:56 schrieb Simon Matthews <simon.d....@gmail.com>:

More on what is stopping deploy.sh

I see this error message, which I had erroneously ignored previously:
cp: cannot stat 'code-coverage-api-plugin/target/*hpi': No such file or directory


Ups, the path has been changed. I fixed that now. 
Ok, that looks good. (You just need the plugins analsis-model-api and warnings-ng, the other ones are not required in your PR).

Then you see if the parser already works.

Simon Matthews

unread,
Jun 8, 2022, 8:36:03 PM6/8/22
to Jenkins Developers
On Wednesday, June 8, 2022 at 3:44:23 PM UTC-7 ullrich...@gmail.com wrote:

Am 08.06.2022 um 19:56 schrieb Simon Matthews <simon.d....@gmail.com>:

More on what is stopping deploy.sh

I see this error message, which I had erroneously ignored previously:
cp: cannot stat 'code-coverage-api-plugin/target/*hpi': No such file or directory


Ups, the path has been changed. I fixed that now. 


I did that. No errors.

I pushed my changes to branches ("add_bps") in
and

What's the next step?


Reply all
Reply to author
Forward
0 new messages