Miscalculated "new issues/coverage/TD" between 2 analysis (commits)

[fester...@gmail.com]

Hi guys,

I wrote a few days ago about analysis between 2 git commits. The purpose is to get the metrics on the difference - coverage on new code, issues on new code, etc. 

I managed to run it successfully trough jenkins with 1 analysis with previous date on the first commit from the branch (the one it is rebased on) and a second analysis on the last commit. 

So sonar shows me the "new code" metrics, but most of the time it is showing me wrong values :( For example ... I change 1 line of code, or a pom.xml file and sonar shows me coverage on new code around 50-60% which ofcourse is wrong.

However I am not sure if the problem is in sonar or in my jenkins environment ...

I am launching maven install and after this a maven sonar analysis, before the build starts I am cleaning the jenkins workspace and make a fresh checkout of my project. I am running the maven sonar analysis from my top level maven pom (multimodule java project), I tried to do it with -o option for offline and also without it, but the result is the same. 1/2 ot my analysis show wrong coverage on new code, new issues, new TD

Anyone having an idea what I can do? I'll try to reproduce it with smaller project as the one I am using is a big one, but decided to ask here first.

Thanks!

[fester...@gmail.com]

Forgot to mention ... I am using Jacoco maven plugin to gather coverage data. I did not investigate if it is showing the same percentage as sonar, I am giving it a try at the moment.

[G. Ann Campbell]

Is your checkout for the date you specify in your analysis parameters? I.e. if sonar.projectDate is Jan 1, are you checking out Jan 1's code?

Ann

[fester...@gmail.com]

Yes, my date is set manually, I am using 2 dates with 1-2 days between them. They're not the actual dates of the checked out code. 

Is sonar finding the "new code" using the date specified? If this is the case I'll use my real dates (made it this way as I thought it won't matter).

Thanks for the quick response Ann!

[G. Ann Campbell]

"New" code is detected using SCM information.

---

G. Ann CAMPBELL | SonarSource

Product Owner

http://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/MDMFycnav0s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/bf3b0b9c-44d8-4a00-aea8-39e8ef0f35ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[fester...@gmail.com]

Ok, than the dates I set should not be a problem. Any idea what else could be the problem? 

[G. Ann Campbell]

They're not the actual dates of the checked out code. 

That^

You need to backdate your checkout as well as your scan. 

---

G. Ann CAMPBELL | SonarSource

Product Owner

http://sonarsource.com

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/76573144-d4e6-4ebd-91ba-aa0b18e1851b%40googlegroups.com.

[fester...@gmail.com]

Sorry I didn't understand what you wrote :) I'll try it with actual dates.

[fester...@gmail.com]

I tried to set actual dates but it's still not accurate enough to use it for my purpose :(

The purpose is (in case someone didn't understand): To find the new issues, new code coverage, new TD in a branch and I am trying to make it happen with 2 analysis - 1 on the commit the branch is rebased on and 1 on the last commit of the branch.

The closest date I can set for a specific branch is the date of the commit it is rebased on (and which will be analysed before the last commit). But for a quite big team we have like 10-20 merges in the master per day, so if I set a date for example 2016-03-07 and I want to analyse a branch which is rebase on a commit with date 2016-03-07 12:00 AM and has a last commit with date 2016-03-07 4:00 PM. So in this case it seems that sonar detects all the code from the day as new code, not only between 12:00AM - 4:00PM, but from 2016-03-07 00:00.

Is there a way to avoid this? sonar.projectDate is with format YYYY-MM-DD so I can't set a more accurate date + time. I know this feature is made to analyze old projects to track trends and should not be used in cases like mine, but still ... is there a way to make this work?

Thanks!

P.S. Happy women's day G. Ann. and to all women here :)

[nicolas...@sonarsource.com]

Hi there,

Taking a step back: new code is identified with SCM info (as Ann said) and a date. From their on it's just an SCM blame. You need to backdate your scan because the date currently used (for the blame) is the date of the previous analysis (and not the commit date of the code previously analysed). This is not optimal and has its limitation, see SONAR-7085 for more details.

Current granularity of sonar.projectDate is indeed YYYY-MM-DD, so while I understand your use-case and the shortcomings you're seeing (due to high-activity on the branch), I don't see any workaround.

On a side note, I'd actually argue that the past is the past. So although your project history might not be completely accurate due to the limitations we're discussing, the most important is that once you're on track, new code and related Quality Gates are more accurately computed. That is the case since internally SonarQube will use a more fine-grained date (the actual clock timestamp of the previous analysis).

Best regards,

Nicolas