SonarQube GitHub Plugin With Jenkins

[hay...@gmail.com]

Hello,

We are very interested in leveraging the GitHub integration on my team:

http://docs.sonarqube.org/display/PLUG/GitHub+Plugin

We use Jenkins as our CI Server and already have the GitHub Pull Request Builder working there.  We have also been using SonarQube for analysis, but have not had much luck with the preview or incremental modes and GitHub integration.  Current versions are SonarQube 5.1.1 and GitHub Plugin version 1.0.1.

We have the plug-in installed on SonarQube and have our GitHub Enterprise API configured.  We have attempted using the SonarQube Runner in Jenkins and Maven to get this to work following your travis.sh script as a guide:

https://github.com/SonarSource/sonarqube/blob/master/travis.sh

We see the Sonar integration show up on GitHub, but it is always successful (we are working with large, legacy code bases and expect to see a lot of output).

The thing that is consistent between our javascript and java analysis is that we don't seem to be getting any files indexed and the GitHub Plugin InputFile Cache sensor is used instead of Javascript or Java sensors we are used to seeing during a full analysis.

00:01:57.532 13:12:31.109 INFO  - 0 files indexed
00:01:57.533 13:12:31.110 INFO  - 25784 files ignored because of inclusion/exclusion patterns
00:01:57.535 13:12:31.112 INFO  - Quality profile for js: Sonar way
00:01:57.758 13:12:31.334 DEBUG - 'SCM Sensor' skipped in preview mode
00:01:57.758 13:12:31.335 DEBUG - 'CPD Sensor' skipped in preview mode
00:01:57.759 13:12:31.336 DEBUG - Sensors : Lines Sensor (wrapped) -> GitHub Plugin InputFile Cache
00:01:57.759 13:12:31.336 INFO  - Sensor Lines Sensor (wrapped)
00:01:57.759 13:12:31.336 INFO  - Sensor Lines Sensor (wrapped) (done) | time=0ms
00:01:57.759 13:12:31.336 INFO  - Sensor GitHub Plugin InputFile Cache
00:01:57.759 13:12:31.336 INFO  - Sensor GitHub Plugin InputFile Cache (done) | time=0ms
00:01:57.759 13:12:31.336 DEBUG - No previous analysis, skipping issue tracking
00:01:57.763 13:12:31.340 INFO  - 
00:01:57.763 
00:01:57.763 -------------  Issues Report  -------------
00:01:57.763 
00:01:57.763   No file analyzed
00:01:57.763 
00:01:57.763 -------------------------------------------
00:01:57.763 
00:01:57.763 
00:01:57.763 13:12:31.340 INFO  - Export issues to /data/jenkins/workspace/mygea-mygeaviation-pullrequests/.sonar/sonar-report.json
00:01:57.766 13:12:31.343 INFO  - ANALYSIS SUCCESSFUL
00:01:57.767 13:12:31.344 DEBUG - Post-jobs : GitHub Pull Request Issue Publisher
00:01:57.767 13:12:31.344 INFO  - Executing post-job class org.sonar.plugins.github.PullRequestIssuePostJob

Are we missing something simple?

Any assistance on this would be much appreciated!  This is an awesome plug-in and we think it will really help us get our technical debt under control.

Best Regards,

Josh

[Julien Lancelot]

Hi Josh,

I think the thing that will be able to help you is : 

00:01:57.533 13:12:31.110 INFO  - 25784 files ignored because of inclusion/exclusion patterns

You should check your configuration to check why all files are excluded. Please have a look at this page for more information : http://docs.sonarqube.org/display/SONAR/Narrowing+the+Focus

Regards,

Julien LANCELOT | SonarSource

http://sonarsource.com

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/168e5865-ab53-4580-a02c-9838346fb697%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hays, Joshua (GE Aviation, US)]

Hello Julien,

 

Thank you for your very quick response!

 

We are using the same settings for our full scans, so we thought we had ruled this out.  When we do a full scan the log looks like this:

 

00:02:56.184 21:58:21.810 INFO  - 88 files indexed

00:02:56.184 21:58:21.811 INFO  - 25784 files ignored because of inclusion/exclusion patterns

00:02:58.742 21:58:24.369 INFO  - Quality profile for js: Sonar way

00:02:58.944 21:58:24.571 DEBUG - Sensors : Lines Sensor (wrapped) -> QProfileSensor -> InitialOpenIssuesSensor -> ProjectLinksSensor -> VersionEventsSensor -> JavaScriptSquidSensor -> SCM Sensor (wrapped) -> org.sonar.plugins.javascript.lcov.UTCoverageSensor@4f760f35 -> org.sonar.plugins.javascript.lcov.ITCoverageSensor@4fe08954 -> CPD Sensor (wrapped)

00:02:58.944 21:58:24.571 INFO  - Sensor Lines Sensor (wrapped)

00:02:58.980 21:58:24.607 INFO  - Sensor Lines Sensor (wrapped) (done) | time=36ms

00:02:58.981 21:58:24.607 INFO  - Sensor QProfileSensor

00:02:58.986 21:58:24.613 INFO  - Sensor QProfileSensor (done) | time=6ms

00:02:58.986 21:58:24.613 INFO  - Sensor InitialOpenIssuesSensor

00:02:59.047 21:58:24.674 INFO  - Sensor InitialOpenIssuesSensor (done) | time=61ms

00:02:59.047 21:58:24.674 INFO  - Sensor ProjectLinksSensor

00:02:59.056 21:58:24.683 INFO  - Sensor ProjectLinksSensor (done) | time=9ms

00:02:59.056 21:58:24.683 INFO  - Sensor VersionEventsSensor

00:02:59.079 21:58:24.706 INFO  - Sensor VersionEventsSensor (done) | time=23ms

00:02:59.079 21:58:24.706 INFO  - Sensor JavaScriptSquidSensor

00:02:59.462 21:58:25.089 INFO  - 88 source files to be analyzed

00:03:03.567 21:58:29.191 DEBUG - Updating semaphore batch-mygea-ux-pullrequests

00:03:08.724 21:58:34.351 INFO  - 88/88 source files have been analyzed

 

The same 25784 lines are ignored, but analysis is completed for the remaining files and more sensors are loaded.

 

Let me know if you need any additional  information to help with this.

 

Josh

[Julien Lancelot]

Then I've no idea what's your issue. But I'm sure our batch analysis expert will be able to help !

He will answer you when he'll be available.

Regards,

Julien LANCELOT | SonarSource

http://sonarsource.com

[Julien HENRY]

Hi Josh,

The SQ GitHub plugin will automatically filter input files to only analyze files that are part of the pull request using GitHub API. For example we use result from:

https://api.github.com/repos/SonarSource/sonarqube/pulls/430/files

Can you check that your pull request has some modified source files?

Also it would be interesting if you could share full log of the analysis (send it to me privately if you want).

++

Julien

[Hays, Joshua (GE Aviation, US)]

Hello Julien,

 

This got us to the answer!  It appears to have been working as designed all along.  We had modifications, but they were in pom.xml files, readme files, etc.  We made a functional change and saw everything light up on the GitHub Pull Request.

 

Thank you for pushing us in the right direction.  We are looking forward to making this part of our workflow!

 

Best Regards,

Josh

 

From: Julien HENRY [mailto:julien...@sonarsource.com]
Sent: Friday, July 31, 2015 11:06 AM
To: SonarQube
Cc: Hays, Joshua (GE Aviation, US); Decrane, Jacob (GE Aviation, US); hay...@gmail.com
Subject: Re: SonarQube GitHub Plugin With Jenkins

 

Hi Josh,

[dries.van...@gmail.com]

Or you sure of this? I took a quick peek in the code and I didn't see any code to only analyse the changed files in de PullRequest / to get the files out of the PR (maybe I missed that code). I see code that checks if the file is in the PR to be able set a comment on that file.

I say this because a pull request of only 2 changed files last like 8 minutes in our project. It seems a whole lot more files are getting scanned. We are wondering why it is getting so long. In the output we clearly see:

"00:48:56.015 INFO  - 13148 files indexed

00:48:56.015 INFO  - 4252 files ignored because of inclusion/exclusion patterns

...

12912/12912 source files have been analyzed"

I would expect only to see "2 files indexed".

Greetings

Dries

[mohan.sid...@gmail.com]

I have the same issue. Analysis is done but I m not able to comment the issues on the github.

this is the lines I m getting

'sonar.sourceEncoding'.
14:23:15.891 WARN  - Invalid character encountered in file /var/lib/jenkins/workspace/Rosterfy/vendor/Aws/data/cloudhsm/2014-05-30/docs-2.json.php at line 3 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
14:23:19.434 WARN  - Invalid character encountered in file /var/lib/jenkins/workspace/Rosterfy/news_story/wordpress/wp-includes/Text/Diff/Engine/string.php at line 13 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
14:23:20.043 INFO  - 8829 files indexed
14:23:20.207 INFO  - Quality profile for php: Sonar way
14:23:20.604 INFO  - JaCoCoSensor: JaCoCo report not found : /var/lib/jenkins/workspace/target/jacoco.exec
14:23:20.605 INFO  - JaCoCoItSensor: JaCoCo IT report not found: /var/lib/jenkins/workspace/target/jacoco-it.exec
14:23:20.714 INFO  - Sensor NoSonar Sensor
14:23:20.796 INFO  - Sensor NoSonar Sensor (done) | time=82ms
14:23:20.796 INFO  - Sensor PHP sensor
14:23:21.181 INFO  - 13 source files to be analyzed
14:23:23.430 INFO  - 13/13 source files have been analyzed
14:23:23.438 INFO  - No PHPUnit test report provided (see 'sonar.php.tests.reportPath' property)
14:23:23.438 INFO  - No PHPUnit unit test coverage report provided (see 'sonar.php.coverage.reportPath' property)
14:23:23.438 INFO  - No PHPUnit integration test coverage report provided (see 'sonar.php.coverage.itReportPath' property)
14:23:23.438 INFO  - No PHPUnit overall coverage report provided (see 'sonar.php.coverage.overallReportPath' property)
14:23:23.438 INFO  - Sensor PHP sensor (done) | time=2642ms
14:23:23.438 INFO  - Sensor Analyzer for "php.ini" files
14:23:23.441 INFO  - Sensor Analyzer for "php.ini" files (done) | time=3ms
14:23:23.442 INFO  - Load server issues
14:23:28.280 INFO  - Load server issues (done) | time=4838ms
14:23:28.281 INFO  - Performing issue tracking
14:23:30.499 INFO  - 10267/10267 components tracked
14:23:30.500 INFO  - ANALYSIS SUCCESSFUL
14:23:30.502 INFO  - Executing post-job GitHub Pull Request Issue Publisher
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
Total time: 29.646s
Final Memory: 15M/432M
INFO: ------------------------------------------------------------------------

[tawal...@gmail.com]

Did you ever resolve this issue?  I am having an issue where analysis results are not reported to github as either inline comments or global comments.  I am pretty sure the code analysis is being performed, but new issues (like adding inline comments) are not being reported in github.  Instead I always get a  green check mark and  SonarQube reported no issues