SonarQube Scanner for MSBuild in multi-language ANT based project

[Vassilena Treneva]

Hello,

I am using the sonarqube scanner for msbuild in a multi-language project, containing java, c++ and c# sub-projects (using the scanner for c# only as it appears to be mandatory if you want to publish c# analysis)

The entire build is ant based - java and c++ results are published together in a single sonar project name/key using ANT sonar tasks.

Is there a way to accomplish this for my c# build?

Based on my experience with the runner the answer is NO, because the runner publishes internally and does not delegate that option to the build.

I am looking for advice because I would like to have a single publish point for this multi-language build.

Thanks!

[G. Ann Campbell]

Hi,

You're right that you need to use the SonarQube Scanner for MSBuild if you want to (correctly) analyze C#.

And to get all your languages analyzed into the same project, they'll need to be analyzed together. I.e. via the SonarQube Scanner for MSBuild. Using Ant (hooboy!)

If you absolutely must use Ant, then your best bet is to <exec/> the SonarQube Scanner for MSBuild.

But that's only a requirement if you're triggering your jobs by hand from a command line. If you're using some sort of CI system, then I'd set up steps to use Ant to handle the things you must do in Ant, and to use the SonarQube Scanner for MSBuild directly for the analysis.

HTH,

Ann

[tre...@gmail.com]

HI Ann,

Many thanks for your input!

Unfortunately our build is ANT based, and I cannot/don't want to escape from ANT :)

Actually, I believe it is pretty common to use ANT for such tasks. 

Do you know if there is a plan to provide better integration for a multi-language projects with ANT based builds?  

[G. Ann Campbell]

Hi,

Multi-language analysis has been an automatic, core feature for a long, long time now. On the other hand, this mashup of build technologies is not something we directly support (Or endorse! Or recommend!) and I don't anticipate any future work on our part to make this easier.

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

https://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/aCsk2Ktaeik/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/7157af37-8609-47e3-8ca4-2fa55990ff0e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[michael....@gmail.com]

Ann,

I would like to encourage SonarQube to rethink their position on this.  I know that I am frozen on a old version of SonarQube and SonarQube Scanner because we have code that is not built my MSBuild as well.  Although, our use case is not as complicated as mentioned by Vasilena (I don't think that combination is that outlandish), it is C# and JavaScript\Typescript.  Lately, I have been contemplating forking the latest version of the scanner to restore the functionality - which is probably something neither of us really want.  

I understand that in 90% of the case .NET shops can\should use the MSBuild scanner as it makes sense.  However, that assumes that the front end devs use Visual Studio for their development needs.  Our front end devs use tools that they are comfortable with WebStorm, Atom, Sublime, etc... Not Visual Studio.  Their build process is built off of their tool chain, not tied to a Visual Studio solution.  The orchestration of the components of the build is managed by the build system.

With Sonar Scanner we could start a scan and collect metrics for each technology under one SonarQube project.  Is this possible when combining different technologies using Sonar Scanner for MSBuild?  The Scanner for MSBuild says that it is the recommended way not the only way.

Michael Fischer

On Wednesday, December 6, 2017 at 8:20:23 AM UTC-7, G. Ann Campbell wrote:

Hi,

Multi-language analysis has been an automatic, core feature for a long, long time now. On the other hand, this mashup of build technologies is not something we directly support (Or endorse! Or recommend!) and I don't anticipate any future work on our part to make this easier.

Ann

[G. Ann Campbell]

Hi Michael,

You're right; SonarQube Scanner for MSBuild is the recommended way, not the only way. And since you're not using a straight MS tool chain, I invite you to try using the vanilla scanner. It might work fine for you. But our experience to date has been that it's very difficult to set up C# analyses correctly by hand. Which is why we try to steer everyone to the SonarQube Scanner for MSBuild.

But launching a C# analysis from Ant?... I doubt we'll be circling back to a direct way to do that.

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

https://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/aCsk2Ktaeik/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/1b9842e8-240d-4a92-9961-5a1be3863e75%40googlegroups.com.

[michael....@gmail.com]

Ann,

OK, I tried the latest version of SonarQube with the latest scanner.  I can see it indexing the CSharp files correctly as 'cs' but then I get to this in the log file using default settings of a fresh install:

14:56:28.069 DEBUG: Sensors : PythonXUnitSensor -> C# -> Python Squid Sensor -> SonarJavaXmlFileSensor -> XML Sensor -> PHP sensor -> Analyzer for "php.ini" files -> TypeScript Sensor -> TypeScript LCOV Coverage Sensor -> JavaScript Squid Sensor -> C# Tests Coverage Report Import -> [Deprecated] C# Integration Tests Coverage Report Import -> C# Unit Test Results Import -> Zero Coverage Sensor -> CPD Block Indexer

.....

14:56:31.454 INFO: Sensor C# [csharp]

14:56:31.470 INFO: Importing analysis results from c:\src\core-platform\null\output-cs

14:56:31.501 INFO: Sensor C# [csharp] (done) | time=47ms

Googling that brings up links (for example) that say since 6.0 the C# plug-in only supports the msbuild scanner.  This is why I haven't upgraded my production environment.  All my tests have failed. Can you confirm that the vanilla scanner should still work?  The scanner I tried is sonar-scanner-cli-3.0.3.778-windows.zip.  It is possible that I have reconfigured it but I have not seen any documentation on how to ensure C# works with the vanilla scanner.  Be glad to be pointed to some.

We have also been starting to use .NET Core on Linux but since 6.6 the C# plugin has failed if detected on Linux or MacOs systems.  I'm not sure why that would be.

Michael Fischer

[G. Ann Campbell]

Hi Michael,

I'm really not following the thread in your message. From my experience the SQ Scanner for MSBuild will pick up _every_ file listed in your project files. If it's missing non-.cs files then you should try listing them as 'included'.

As for the platform question, the latest versions support analysis on non-Windows environments, but only the latest. (I'm not even confident it's been released yet. Check the release notes.)

BTW, I'm on vacation until Jan. and answering this from my phone. Don't be surprised if I don't respond again until 2018.

Ann

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/aCsk2Ktaeik/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/71cd4707-fbe5-4b0b-98c9-e86f0d9e9cb9%40googlegroups.com.

[michael....@gmail.com]

Perhaps I misunderstood your previous comment,  "I invite you to try using the vanilla scanner. It might work fine for you. But our experience to date has been that it's very difficult to set up C# analyses correctly by hand."  I assumed that this meant to try the non "For MS-Build" SonarScanner.  I did not think it would work but gave it another try.  

Either way.  Happy Holidays and I will look for your response in the new year.

Michael Fischer

To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

[G. Ann Campbell]

Hi Michael,

I guess that's what I get for answering from my phone (without reviewing the context in the thread).

Re the log messages you listed... I'm not sure what the point was. I don't see any errors or even warnings. You've turned on debug logging there, which is why you get a list of all possible sensors. Only the ones relevant to files found in the project will actually fire during the analysis. I assumed from your inclusion of the log messages that you had some problem with the analysis & thought those messages relevant. 

Re analysis of C# on Linux... yeah MS supports C# on non-Windows now, but we didn't until the most recent releases.

Re the vanilla scanner... you say the recommended way won't work for you because you're using a non-standard tool chain. So okay, try a non-standard analysis. But I have strong doubts and reservations & as soon as you report problems with that, the answer is always going to be "do a standard analysis". And BTW, just because you have additional files that aren't built by MSBuild (that's what I get from your first message in this thread) that doesn't mean you can't use the SonarQube Scanner for MSBuild, just that you'll need to make sure those extra files are included (e.g. <Content Include="src\**\*.js" />) in a project in your solution.

Ann

[michael....@gmail.com]

Anne,

I hope that you had a good vacation.  Let me try to clarify and get us on the same page. 

I think you are spot on about what I was trying to show in the log.  I just wasn't explicit about it.  There isn't anything reported when the C# scanner runs it just finishes in 47ms.  My assertion is that the vanilla scanner no longer looks at the C# files.  I see that it detects files as CS but when the scanner runs there is no debugging output just as you mention.  

Latest scanner

14:56:31.454 INFO: Sensor C# [csharp]

14:56:31.470 INFO: Importing analysis results from c:\src\core-platform\null\output-cs

14:56:31.501 INFO: Sensor C# [csharp] (done) | time=47ms

versus the JavaScript sensor (in same log)

14:57:21.035 INFO: Sensor JavaScript Squid Sensor [javascript]

14:57:21.066 INFO: 13032 source files to be analyzed

                          [..... 19,000+ deleted lines .....]

15:09:40.805 INFO: Unit Test Coverage Sensor is started

15:09:40.852 INFO: 13032/13032 source files have been analyzed

15:09:40.852 INFO: Integration Test Coverage Sensor is started

15:09:40.852 INFO: Overall Coverage Sensor is started

15:09:40.852 INFO: Sensor JavaScript Squid Sensor [javascript] (done) | time=739817ms

or the earlier C# sensor

INFO: Sensor C#

INFO: Analyzer working directory does not exist

INFO: SonarAnalyzer.Scanner needs to be executed: true

INFO: SonarAnalyzer for C# version 1.20.0.1206

.....

I've been wondering if it has something to do with the "\null\" in the "c:\src\core-platform\null\output-cs?"  I am not sure why that is there.

My complaint about the MSBuild solution are as follows.  

• The developers that are not working on CS files do not use visual studio. This forces the scenario that they either have to adopt a tool that they are not comfortable in or someone that uses that tool has to track files that they did not create.  
• It seems that the preferred way is more error prone to skipping files that should be included in analysis while the vanilla scanner is more prone to including files that you might want to exclude.   The later error seems much more preferable to me as log analysis will detect it.
  

My specific question is... Should the scanner (https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner) work on C# files without specifying other values that sonar.pojectKey, sonar.ProjectName, sonar.ProjectVersion, and sonar.sources?  From the logs above this does not seem to be the case. It seems that at some point after version 2.8 that stopped working.  My assumption has been that this was intentional but based on our conversation I am unsure now.

If this does not make it any clearer I'll just assume that the MSBUILD scanner is the only way that works and move on.

Michael Fischer

[G. Ann Campbell]

Hi Michael,

After double-checking with the .NET team... it seems I've sent you on a wild goose chase, for which I apologize.

Formerly this would have worked, but 

SonarQube Scanner used MSBuild 12 mode, which is no longer implemented

SonarC# 6.0 dropped the functionality

So... it seems that we just don't support your use case.


Sorry,
Ann

[michael....@gmail.com]

Anne,

No worries.  Now we are communicating!  

This brings me back to my original post where I requested that SonarQube reconsider the deprecation of this feature.  Is there a better way to communicate a feature request such as this?  I know that the MSBUILD scanner makes sense in pure .NET shops but not all shops are pure .NET and have build processes outside of the solution / proj file scheme that MSBUILD uses.  It would be nice to have a scanner that works in those scenarios.

Thank you for all of your efforts on my behalf.

Michael Fischer

[G. Ann Campbell]

Hi again,

So going back to your original post, you said

 Lately, I have been contemplating forking the latest version of the scanner to restore the functionality

and it's not clear to me what functionality you're talking about restoring, nor in your most recent post

  I requested that SonarQube reconsider the deprecation of this feature.  

what feature you think has been deprecated.

As we've (finally!) established, the ability to analyze C# with the vanilla scanner has been removed. If you're asking for the return of that... well, I think you're going to be disappointed. However, it's possible that the .NET team will be working to make multi-language analysis easier to do this year.

To be crystal clear, are you saying that even your C# files are not built with MSBuild? Because as I said earlier, I can tell you from personal experience that it is possible to analyze other languages with the SonarQube Scanner for MSBuild. 

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

https://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/aCsk2Ktaeik/unsubscribe.

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/5e503f0e-adce-4051-b0f8-65a60a057ac7%40googlegroups.com.

[michael....@gmail.com]

Well, thank you for looking into this for me.  I actually had not provided much details about my scenario as I didn't want to complicate with unnecessary details.  I am simply saying that regardless of our individual use cases there are people that have challenges with the direction that SonarQube has taken.  Both Vassilena and myself have stated that SonarQube does NOT meet our needs anymore due to the removal of scanning of C# from the other scanners.

My goal is that I have products that are written in several different languages (at least 4), built by multiple build systems (2), includes custom steps on multiple sever OS's.  To get a holistic view I can get artifacts into a common folder and kick off analysis.  In my humble opinion I should be able to initiate this scan with any of your scanners in a technology agnostic manner. I was able to do that previously and now I cannot.  There may be perfectly good reasons for this but I haven't seen them shared with the community. 

Yes, we do build our C# code using MSBUILD but I can easily think of cases where I would not use MSBUILD to build C# code where I would want to use the C# scanner.  

That being said, we can, and will continue to use SonarLint at the individual developer level.  Sadly, we will not be able to get the reporting that we were looking for from SonarQube without being frozen on an old version of the server and scanner.  Thank you again for your patience during this long thread.

Michael Fischer

[dinesh.bo...@sonarsource.com]

Micheal,

I would recommend you to use the  "vanilla" SonarQube Scanner to analyze the Javascript and Typescript code of your project, and the SonarQube Scanner for MSBuild for the C# part.

This should be straightforward to do, but indeed will lead to 2 separate projects in SonarQube.

If a unified view of the two SonarQube projects is really required, I would aggregate the 2 projects into one "Application" using the Portfolio plugin which comes with the Enterprise edition of SonarQube.

Kind regards,

Dinesh

[Amaury Leve]

Hi,

As @Ann and @Dinesh have recommended the best solution for now is to analyze them as 2 projects and use the application feature of the enterprise edition.

Cheers,

Amaury

On Thu, Mar 1, 2018 at 4:32 AM <sha...@gmail.com> wrote:

Hi Ann, 

If the project is multi language (E.g java/c#), how do we get the results for both languages into one project? 

I notice that Sonar MSbuild only scans c# not Java . I tried running Sonar msbuild and the vanilla sonar-scanner sequentially with the same project key/name but they simply overwrite the results. They do not aggregate them. 

What do you suggest?

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/23c24d74-fbaa-47e0-9507-e0fc35cb18f8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Amaury Levé | SonarSource

Software Developer - .Net Team

http://sonarsource.com

Are you using SonarLint in your IDE? 

[G. Ann Campbell]

Hi,

Alternately, build your Java first, include it in one of your project files (<Content Include="src\**\*.java" />) and pass the java binaries location in as a parameter to the begin command (/d:sonar.java.binaries=build/classes).

HTH,

Ann

To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/23c24d74-fbaa-47e0-9507-e0fc35cb18f8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.