Sonar project result differ even if branch same

[Reddeppa]

Hi,

Trying to understand result in SonarQube.

I have 2 project defined with 2 branches :'master' and 'someX'
Now both branch are in sync but sonar show completely different result. Not sure why ?

In sync” I mean I merge them every 1-2 days.  So for there are difference on a daily basis but both branch get align often.

Any help is extremely appreciated.

Thank you.

BR/
Reddeppa.

[nicolas...@sonarsource.com]

Hi Reddeppa,

I doubt you will get any concrete recommendations without providing any concrete details/examples, nor insight into what you have diagnosed so far. If the codebases are different then it's normal you might get different results. If the codebases are the same then here's how you could proceed if you see different results:

• compare analysis logs to see if project was scanned differently
  
• compare project settings to make sure that the same settings apply
  
• pick one specific issue which is reported on one branch but not on the other, and correlate that with the actual code analysed + what you're seeing in the analysis logs
  

You can consider these as generic first steps that will help you narrowing down differences between distinct analysis.

Best regards,

Nicolas

[Reddeppa]

Hi Nicolas,

Thank you.

I will check as suggested below and let you know.

/Reddeppa

[Reddeppa]

Hi Nicolas,

Could you please let me know on the line from your previous comment:

• Compare analysis logs to see if project was scanned differently :Not sure what to look for ?
• compare project settings to make sure that the same settings apply : Please elaborate what should check for ?

The code is merge everyday && both ways
e.g. master branch contains all team code and each team work and their own branch
So 'someX' is my team and I merge 'someX'  branch to master and master to 'someX' 5 times a day
And this is not new – it has been like this for weeks now

Is this statement true?
Merging branches does not mean that the files referred to by the analysis are the same.

Thank you.

BR/
Reddeppa

[Reddeppa]

Hi,

Is there a way to reset the each branch to new period ? e.g to align both  master and the new branch ?

BR/
Reddeppa

[G. Ann Campbell]

Hi Reddeppa,

It's not clear what you're asking.

Ann

[Reddeppa]

Hi,

As I mentioned in my post, Its regarding the SonarQube Analysis result.

I have 2 project defined with 2 branches :'master' and 'someX' .Now both branch are in sync but sonar show completely different result. “In sync” I mean I merge them every 1-2 days.  So for there are difference on a daily basis but both branch get align often.

Please refer to attached dash boards for both ranches :'master' and 'someX'  and let us know possible reasons for this different results

Thank you.

BR/
Reddeppa

[G. Ann Campbell]

Hi,

Can you narrow down what you mean by "completely different result"? From your screenshots, I see different issue counts and technical debt values, as well as differences in the Function Distribution / Complexity graph. Those IMO are expected since presumably there are code changes in the branch that are not reflected in master (otherwise, why have a branch?).

I also see differences in the Quality Gate condition values, but again I find that unremarkable.

However, if you're expecting to see a narrower gap between the versions in terms of issue counts or coverage, then I would examine my project settings if I were you:

* Are the projects using the same quality profile?

* Do they have the same exclusions set in the UI? In their analysis parameters?

If those things are the same from project to project, then I'd start narrowing down where in the project the differences occur.

Ann

[Reddeppa]

Hi Ann,

Thank you.

We are at the end of project so only TR fixes are in branch so this is why we merge often and why both branch should be aligned

 

The settings and I already check them and did not find any difference .

o   Everything is default under : Project-->Administration-->General Settings-->Edit project settings-->Java-->Checkstyle

o   Quality profiles are the same

o   Quality gates are the same

o   …

o   Everything seams to be the same for configuration

 

·         Exclusion is done via maven pom.xml so yes they have the same

o   sonar.login=xxx…

o   sonar.host.url=https://sonarqube.xxx.xxx.

o   sonar.projectDescription="Master ${BRANCH} -- ${RPMS_NAME}"

o   sonar.branch=${RPMS_NAME}

o   sonar.projectVersion=${RPMS_NAME}

 

o   sonar.login=xxx…

o   sonar.host.url=https://sonarqube.xxx.xxx.

o   sonar.projectDescription="someX ${BRANCH} -- ${RPMS_NAME}"

o   sonar.branch=${RPMS_NAME}

o   sonar.projectVersion=${RPMS_NAME}

 

BR/
Reddeppa

[G. Ann Campbell]

Hi Reddeppa,

Let's back up a little: 

* What, specifically are the unexpected differences? 

* Do they occur in code that is the same from master to branch or different?

Ann

[Reddeppa]

Hi Ann,

The unexpected differences are the Bugs count. The Bugs count is different in 'Master' branch and 'someX' branch.

Master Branch:694 Bugs
The other branch:1313 Bugs

Yes, they occurred in code that is the same from master to branch or different.
I can see using the TREE – XXX-MAIN as 1 issue in master and 3 in the other branch :-(

Thanks,
Reddeppa

[G. Ann Campbell]

Hi Reddeppa,

We might be getting somewhere. Do you provide byte code (class files and dependency jars) to both analyses? 

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

http://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/9fN-R4ZTUmc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/43155c82-2bad-4bf5-b29a-33da67f087a2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Reddeppa]

Hi

 

Both project are build in the same way (using same maven pom files)

 

The only difference will be the machine where they run (but both machine have the same software / clone VM)

Both jobs are running from Jenkins job configured the same way except for the  following

·          

o   sonar.login=20a…

o   sonar.host.url=https://sonarqube.xxx.xxx.xxx

o   sonar.projectDescription="Master ${BRANCH} -- ${RPMS_NAME}"

o   sonar.branch=${RPMS_NAME}

o   sonar.projectVersion=${RPMS_NAME}

 

o   sonar.login=20a…

o   sonar.host.url=https://sonarqube.xxx.xxx.xxx

o   sonar.projectDescription="someX ${BRANCH} -- ${RPMS_NAME}"

o   sonar.branch=${RPMS_NAME}

o   sonar.projectVersion=${RPMS_NAME}

 

The only difference is the time creation of the project.

Thanks,

Reddeppa

To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

[G. Ann Campbell]

Hi Reddeppa,

Do your analysis logs say anything about "Excluded" sources or tests?

Also, can you go to the Issues page of your Master project and check the number of issues marked False Positive and Won't Fix?

Does the total represent the number of missing master issues?

Ann

[Reddeppa]

Hi Ann,

I could see the difference in 'False Positive' and 'Won't Fix' in Issue page in the other branch when compared to Master.

From the other Branch:


From the master branch:

Master:< [INFO] [INFO] 186/186 source files have been analyzed

---

other Branch:> [INFO] 186/186 source files have been analyzed

Thanks,
Reddeppa

[G. Ann Campbell]

Hi Reddeppa,

Just in case it's not clear: your Branch Open + Won't Fix ~= Master Open.

Marking an issue Won't Fix removes it from the open issue counts. Which is why the Branch open count is lower.

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

http://sonarsource.com

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/a4f9776b-82c2-4d2a-bb0f-a87184a51426%40googlegroups.com.

[Reddeppa]

Hi Ann,

In the branch, Open (Unresolved=20,829) + Won't Fix (335)= 21,164

In Master: Open (Unresolved=21,165) + Won't Fix (0)= 21,165

Thanks,
Reddeppa

[Reddeppa]

Hi Ann,

I see the close resemblance with many numbers and some are related to the leak period

•    How do I compare 2 branch?
What I want to do is to compare 2 branch (project) to see if we improve or not.
So since branch and master are almost aligned I expect them to show same result.

•    Can we reset the “leak period"?
Would this make both branch the same?


So the question is:
•    How do we get both branch aligned?

--------------------

And I'm still confused about the number of bug

Those are all based/close branch to master.

But there are still major difference between master and branch (example   # bug  vs vulnerability).

Is this difference due to the leak period?

# bug  vs vulnerability
https://sonarqube.xxx.xxx.xxx/component_measures/?id=com.xxx.mdn%3Amdn-parent%3master      707      vs 663
https://sonarqube.xxx.xxx.xxx/component_measures/?id=com.xxx.mdn%3Amdn-parent%branch       1,324    vs 460


Thanks,
Reddeppa

[G. Ann Campbell]

Hi Reddeppa,

To start, both master and branch have (nearly) the same number of issues. The difference is how they're flagged. Apparently your developers have been very diligent about curating the issues in the branch without doing the same work in Master. I find this surprising, since I would expect the master to be the long-lived entity where you would want the most accurate data, i.e. where you would want to do your Won't Fix marking. But that's a workflow question.

•    How do I compare 2 branch?
What I want to do is to compare 2 branch (project) to see if we improve or not.
So since branch and master are almost aligned I expect them to show same result.

If they were exactly aligned, I would expect the same result too. But they're almost aligned, which I suppose explains the 1-issue difference between them. 

Since you're trying to understand if you've improved (in the branch?) you can do a raw issue count comparison by summing Open + Won't Fix for both versions. As it stands, it looks like you've improved.

Beyond that, though, you should probably sit down with your developers and discuss their use of Won't Fix. The fact is that they appear to be examining individual issues and deciding that they're not relevant/important for your situation. Assuming it's thoughtfully done, this work to mark issues Won't Fix also reflects an "improvement" as you all come to better understand the quality of your application.

If you absolutely insist on seeing the same numbers in both places then why bother having a branch? 

 

•    Can we reset the “leak period"?

Sure

 

Would this make both branch the same?

No

 

So the question is:
•    How do we get both branch aligned?

They already are as closely aligned as they're going to be without the extra work to mark the same issues Won't Fix in master that have been marked in the branch.

[Reddeppa]

Hi Ann,

Thanks a lot for the detailed information.

Could you please let me know, how do I reset the leak period?

BR/
Reddeppa

[G. Ann Campbell]

Hi Reddeppa,

It's in the docs.

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

http://sonarsource.com

--

You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/9fN-R4ZTUmc/unsubscribe.

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/3fa5f07f-32d2-47b7-8aba-537f65fe7037%40googlegroups.com.

[Reddeppa]

Hi Ann,

Thanks very much.

BR/
Reddeppa

To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

[Reddeppa]

Hi Ann,

I don't see exact steps (Administration > General Settings > Leak > Leak Period) as mentioned in the link you have provided, but there is one more link available in the docs for the setting leak period (Administration > General Settings > General > Differential Views > Leak Period).

Thanks,
Reddeppa

To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

[G. Ann Campbell]

Hi Reddeppa,

That's the global / default leak period.

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

http://sonarsource.com

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/bc023b4b-fc0b-4ac8-8db6-75dbffcaa414%40googlegroups.com.

[Reddeppa]

Ok, Thank you Ann.

BR/
Reddeppa

[Reddeppa]

Hi Ann,

I changed the differential view on Branch and did not notice any difference

            

I just changed the leak period on master to match: 60 days

 

I also tried to set the period 4 – but does not seams to work : to compare with a version

Thanks,
Reddeppa.

[G. Ann Campbell]

Hi Reddeppa,

I'm not sure what difference you expected to see. As I said earlier, changing the leak period isn't going to make a difference in your total issue count.

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

http://sonarsource.com

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/4181886a-cf27-451c-af53-530e0eb937b6%40googlegroups.com.