SonarLint Analysis Time 96+ Minutes for Single File

[EricIsInTheHaus]

Hello SonarLint Team,
I have a problem with SonarLint when analyzing some files. Normally it works quite fast (20s for files with more than 8000 lines of code) but some files take more or less forever.
One example is a file with 2600 LoC and it takes more than 96 minutes to analyze.  5775105 Milliseconds to be exact. It seems to only happen to files which are derived from the JUnit Class TestCase.
So they implement a lot of test methods. Maybe it is also because of a deep type hierarchy? The problematic file is 9 hierarchy levels below the JUnit Class TestCase.

I am using:

Eclipse Platform Version:  4.6.3.v20170301-0400
SonarLint Version: 3.2.0
Sonarqube Server: 6.5
My System has 24GB of RAM and an 8 core Intel Xeon Processor running windows 7

I have attached a .nps file which can be viewed in the Java VisualVM. I took a snapshot from there after 30 Minutes of analyzing.

Thanks in Advance

Eric

[Julien HENRY]

Hi Eric,

I am very surprised when I read: 

Normally it works quite fast (20s for files with more than 8000 lines of code)

SonarLint is supposed to be really fast. We are talking about less than 1s here for most common situations.

We used to have performance issue in the Java analyzer during classpath initialization step. Could you please:

  - give us the version of SonarJava installed on your server (you are using connected mode right?)

  - enable verbose output in the SonarLint console, and look for duration of the different steps. This should looks like:

JavaClasspath initialization

JavaClasspath initialization (done) | time=0ms

JavaTestClasspath initialization

JavaTestClasspath initialization (done) | time=4ms

Java Main Files AST scan

1 source files to be analyzed

Java Main Files AST scan (done) | time=16ms

Java Test Files AST scan

0 source files to be analyzed

1/1 source files have been analyzed

Java Test Files AST scan (done) | time=5ms

Thanks,

Julien

[Duarte Meneses]

Hi Eric,

Thanks for reporting the problem, and the snapshot is very useful.

I noticed that there are surprisingly long times creating objects. What is the heap size that you assigned to Eclipse?

--
You received this message because you are subscribed to the Google Groups "SonarLint" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/efab2d2e-9f87-464a-8650-52cb1a26d235%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Duarte Meneses | SonarSource

Software Engineer

https://sonarsource.com

Are you using SonarLint in your IDE?

[holger...@gmail.com]

Heap size of Eclipse:
- EricIsInTheHaus: -Xmx4096m
- me: -Xmx6096m

Am Mittwoch, 11. Oktober 2017 11:12:05 UTC+2 schrieb duarte.meneses:

Hi Eric,

Thanks for reporting the problem, and the snapshot is very useful.

I noticed that there are surprisingly long times creating objects. What is the heap size that you assigned to Eclipse?

On 10 October 2017 at 16:23, EricIsInTheHaus <eric...@hotmail.de> wrote:

Hello SonarLint Team,
I have a problem with SonarLint when analyzing some files. Normally it works quite fast (20s for files with more than 8000 lines of code) but some files take more or less forever.
One example is a file with 2600 LoC and it takes more than 96 minutes to analyze.  5775105 Milliseconds to be exact. It seems to only happen to files which are derived from the JUnit Class TestCase.
So they implement a lot of test methods. Maybe it is also because of a deep type hierarchy? The problematic file is 9 hierarchy levels below the JUnit Class TestCase.

I am using:

Eclipse Platform Version:  4.6.3.v20170301-0400
SonarLint Version: 3.2.0
Sonarqube Server: 6.5
My System has 24GB of RAM and an 8 core Intel Xeon Processor running windows 7

I have attached a .nps file which can be viewed in the Java VisualVM. I took a snapshot from there after 30 Minutes of analyzing.

Thanks in Advance

Eric

--
You received this message because you are subscribed to the Google Groups "SonarLint" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/efab2d2e-9f87-464a-8650-52cb1a26d235%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Duarte Meneses]

Thanks. Could you please also provide the version of SonarJava that you are using, as requested by Julien? 

To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/e612f663-6b81-40a4-8021-3e98331d7c7a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[holger...@gmail.com]

The (plugin) project with this unit test was not connected to the SonarQube Server; nethertheless:
SonarJava: 4.12.0.11033
SonarQube Server version: Version 6.5 (build 27846)


below you will find the SonarLint console output. some information:
1. This analysis was done on a different class than the one that Eric mentioned. according to the log it took about 4 minutes on my machine (-Xmx6096m).

2. every 10 seconds this line was printed into the console:
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java

3. I have removed some lines in the output below regarding adding additional package/classpath paths


the verbose output of the SonarLint Console:

Starting SonarLint for Eclipse 3.2.0.201706271328
Check for updates from server 'vispen5308pch.vi.vector.int'
  - Quality profile 'Optimal Java' for language 'Java' updated
Check for updates from server 'vispen5308pch.vi.vector.int' for project 'com.vector.easee.modelmanagement.server'
SonarLint analysis of file /aquintos.prop.common/META-INF/MANIFEST.MF...
Starting standalone SonarLint engine 3.2.0.201706271328...
Found 0 issue(s)
Trigger: EDITOR_OPEN
SonarLint analysis of file /com.vector.easee.modelmanagement.client.test/src/com/vector/easee/modelmanagement/test/client/core/reuse/ReuseParallelTest.java...
Standalone mode (project not bound)
Starting analysis with configuration:
[
  baseDir: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test
  workDir: D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\com.vector.easee.modelmanagement.client.test\org.sonarlint.eclipse.core
  extraProperties: {sonar.java.source=1.8, sonar.java.target=1.8, sonar.libraries=D:\dev\jdk1.8.0_112_x86_64\jre\lib\resources.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\rt.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\jsse.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\jce.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\charsets.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\jfr.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\access-bridge-64.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\cldrdata.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\dnsns.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\jaccess.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\jfxrt.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\localedata.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\nashorn.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunec.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunjce_provider.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunmscapi.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunpkcs11.jar,D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\zipfs.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.core.runtime_3.12.0.v20160606-1342.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\javax.inject_1.0.0.v20091030.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.osgi_3.11.0.v20160603-1336.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.region_1.3.0.v20150929-2134.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.transforms.hook_1.1.0.v20131021-1933.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.weaving.hook_1.1.200.v20150730-1648.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.fx.osgi_2.4.0.201605100504.jar,D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.jetty.osgi.alpn.fragment_9.3.9.v20160517.jar,

------ many more plugin paths of us.... }


  inputFiles: [
    D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java [test]
  ]
]

Available languages:
  * PHP => "php"
  * Python => "py"
  * Java => "java"
  * JavaScript => "js"
Start analysis
Declared extensions of language PHP were converted to php: file:**/*.php,file:**/*.php3,file:**/*.php4,file:**/*.php5,file:**/*.phtml,file:**/*.inc
Declared extensions of language Python were converted to py: file:**/*.py
Declared extensions of language Java were converted to java: file:**/*.java,file:**/*.jav
Declared extensions of language JavaScript were converted to js: file:**/*.js,file:**/*.jsx,file:**/*.vue
Index files
Language of file 'D:/dev/trunk/workspace/com.vector.easee.modelmanagement.client.test/src/com/vector/easee/modelmanagement/test/client/core/reuse/ReuseParallelTest.java' is detected to be 'java'
Setting filesystem encoding: UTF-8
1 files indexed
'PHP sensor' skipped because there is no related file in current project
'Python Squid Sensor' skipped because there is no related file in current project
'JavaScript Squid Sensor' skipped because there is no related file in current project
Execute Sensor: JavaSquidSensor
Configured Java source version (sonar.java.source): 8
JavaClasspath initialization
JavaClasspath initialization (done) | time=352ms
JavaTestClasspath initialization
JavaTestClasspath initialization (done) | time=301ms
----- Classpath analyzed by Squid:
D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\bin
D:\dev\jdk1.8.0_112_x86_64\jre\lib\resources.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\rt.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\jsse.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\jce.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\charsets.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\jfr.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\access-bridge-64.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\cldrdata.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\dnsns.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\jaccess.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\jfxrt.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\localedata.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\nashorn.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunec.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunjce_provider.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunmscapi.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\sunpkcs11.jar
D:\dev\jdk1.8.0_112_x86_64\jre\lib\ext\zipfs.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.core.runtime_3.12.0.v20160606-1342.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\javax.inject_1.0.0.v20091030.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.osgi_3.11.0.v20160603-1336.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.region_1.3.0.v20150929-2134.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.transforms.hook_1.1.0.v20131021-1933.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.weaving.hook_1.1.200.v20150730-1648.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.fx.osgi_2.4.0.201605100504.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.jetty.osgi.alpn.fragment_9.3.9.v20160517.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.osgi.compatibility.state_1.0.200.v20160504-1419.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.osgi.nl_de_4.6.0.v20160813060001.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\vi.jre.oracle.exports_1.0.1.jar
D:\dev\trunk\workspace\vi.jre8.profile\bin
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.common_3.8.0.v20160509-1230.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.core.jobs_3.8.1.201703021017.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.registry_3.6.100.v20160223-2218.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.preferences_3.6.0.v20160120-1756.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.core.contenttype_3.5.100.v20160418-1621.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.equinox.app_1.3.400.v20150715-1528.jar
D:\dev\trunk\workspace\aquintos.core\res\wibu\CodeMeter.jar
D:\dev\trunk\workspace\aquintos.log\lib\obfuscate_annotations.jar




-------- ... many more plugin paths of us





D:\dev\trunk\workspace\aquintos.metric.editor\bin
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.apache.commons.lang_2.6.0.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.apache.httpcomponents.httpcore_4.4.5.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.osgi.services_3.5.100.v20160504-1419.jar
D:\dev\trunk\workspace\.metadata\.plugins\org.eclipse.pde.core\.bundle_pool\plugins\org.eclipse.ui.views_3.8.100.v20160518-1929.jar
-----

Java Main Files AST scan

0 source files to be analyzed

Java Main Files AST scan (done) | time=2ms

Java Test Files AST scan

0/0 source files have been analyzed

1 source files to be analyzed

0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java
0/1 files analyzed, current file: D:\dev\trunk\workspace\com.vector.easee.modelmanagement.client.test\src\com\vector\easee\modelmanagement\test\client\core\reuse\ReuseParallelTest.java

1/1 source files have been analyzed

Java Test Files AST scan (done) | time=233369ms
Execute Sensor: Analyzer for "php.ini" files
Execute Sensor: SonarJavaXmlFileSensor
Found 0 issue(s)
0 entries removed from the store
Done in 234440 ms

[duarte.meneses]

Hi,

Thanks for the additional information.

It seems like a lot of time is spent resolving the class hierarchy involving generics.

Would it be possible to give us the full class hierarchy (including generics) of the class being analyzed?

Thanks.

[holger...@gmail.com]

the parent classes of ReuseParallelTest themselves do not have generic types.
however there are members and methods that have generics/are generified. can you say whether members or methods are affected?
The source code is not open source; thus it is not easily possible to provide you with the full source code.

[EricIsInTheHaus]

One of the biggest problems right now with this issue is, that cancelling the On-The-Fly Analysis takes almost as long as the analysis. So it would be nice to at least be able to cancel the analysis when it takes so long.

Could we make this a feature request? That would be a very helpful workaround before finding the exact problem. Right now this issue is holding us back from using SonarLint in our productive environment. This is because it is not applicable to accept compile blocking mechanisms just because the "wrong" file was opened.

Maybe this helps as a first step solution.

Best regards Eric

[Julien HENRY]

Hi Eric,

Cancelling an analyzer in the middle of a file processing is a complex topic. Not likely to occurs soon.

What would help since you can't share your code (even privately) is to create a minimal reproducer, by progressively removing all unnecessary code/renaming classes/... until you have something:

  - that still shows the issue

  - that you can share (again, you can send it to us privately if you have some concerns about exposing sensitive data)

Thanks,

Julien Henry | SonarSource

Developer

https://sonarsource.com

--

You received this message because you are subscribed to the Google Groups "SonarLint" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/b1c890a1-250f-4239-b71d-325601eed2c1%40googlegroups.com.

[EricIsInTheHaus]

Hey Julien,
thanks for the fast response. We will check if that is possible for us at the moment.
Since the main time in the SonarLint Console seems to be used for analyzing the code, do you have a hint or clue which rule could be causing this problem. We will try to narrow down the issue by switching of more and more rules and see if the problem persists. So if you have an idea what is causing the issue it would save us a lot of time!

Thanks in advance :)

Am Dienstag, 17. Oktober 2017 10:43:39 UTC+2 schrieb Julien HENRY:

Hi Eric,

Cancelling an analyzer in the middle of a file processing is a complex topic. Not likely to occurs soon.

What would help since you can't share your code (even privately) is to create a minimal reproducer, by progressively removing all unnecessary code/renaming classes/... until you have something:

  - that still shows the issue

  - that you can share (again, you can send it to us privately if you have some concerns about exposing sensitive data)

Thanks,

Julien Henry | SonarSource

Developer

https://sonarsource.com

2017-10-17 10:38 GMT+02:00 EricIsInTheHaus <eric...@hotmail.de>:

One of the biggest problems right now with this issue is, that cancelling the On-The-Fly Analysis takes almost as long as the analysis. So it would be nice to at least be able to cancel the analysis when it takes so long.

Could we make this a feature request? That would be a very helpful workaround before finding the exact problem. Right now this issue is holding us back from using SonarLint in our productive environment. This is because it is not applicable to accept compile blocking mechanisms just because the "wrong" file was opened.

Maybe this helps as a first step solution.

Best regards Eric

--
You received this message because you are subscribed to the Google Groups "SonarLint" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+...@googlegroups.com.

[Julien HENRY]

I'm not developer of SonarJava, but be careful when trying to isolate the offending rule. We have very few rules running on test files, and when no rules remains for test files, I think the SonarJava Sensor for test files will be entirely skipped (it means no parsing will occurs at all).

Julien Henry | SonarSource

Developer

https://sonarsource.com

To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/7533bb05-bf04-4744-af13-135d38f463ca%40googlegroups.com.

[Nicolas Peru]

Hi, 

I am a developer of SonarJava. Having a look at your snapshot file it seems the issue comes from semantic analysis and more especially resolving generic substitution when resolving method. 

That is why Duarte asked for the full hierarchy of the class causing the trouble. Note that while you shared the class hierarchy, I would really like to also know the interface hierarchy of those test class (in other words, which interfaces those classes implements and what are they extending as well) so the _entire_ type hierarchy (class + interfaces and their ancestors).

I suspect that the recursion dealing with generic substitution of formal parameters is going crazy (this is what I see from the snapshot) but I would like to understand why to try to make a reproducer and deliver a fix. 

I believe you have a complicated interface hierarchy and the algorithm is naive and inefficient in that case but I need you to confirm this.

Thanks for helping us out.

Cheers,

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/7533bb05-bf04-4744-af13-135d38f463ca%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SonarLint" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/CAD9wyxFrMkSdJPigunmEZSjJcnr8m2zfncaDHwrpX9XOu4SLUQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Nicolas Peru | SonarSource

Senior Developer

https://twitter.com/benzonico

https://sonarsource.com

[holger...@gmail.com]

I have attached a screenshot of the full class hierarchy. There is just one interface; junit.framework.Test ; thus no complicated interface hierarchy according to the testclasses themselves. However the testmethods are "interface-related".
In one parent class there is a method returning a Pair<IEETestModelFactory, MDFOutermostPackage> ; IEETestModelFactory has 4 further super-interfaces each containing many methods. MDFOutermostPackage also has many methods and extends several other interfaces.

[Nicolas Peru]

Hi, 

That may very well be it : looking back at the snapshot the time is spent solving substitution from a call site with a big hierarchy. 

So if you have a complex hierarchy from a type on which you invoke a method you might very well trigger a complicated recursion. At this point that could be really great if you could narrow done the case and provide me with a structure of such a hierarchy. 

I'll try to reproduce the case on my side but I lack a bit of concrete data to construct a proper testcase. 

Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/713cd46c-1995-4048-8f59-5f3f06a1fa27%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nicolas Peru]

I tried to reproduce the problem on my side and I managed to make an improvement but I would require you to test it if it works. 

You can find a patched version of the plugin here https://expirebox.com/download/7b6a441666ebf7877925b065aa535494.html to test if it improves performances (please don't use this version in production, it is not suited for release) (note that this link will only be up for two days).  

(Note that this patched version is based on the latest master, so there might be improvement in some other areas).

Could you test and let us know if performances improves ?

Thanks

[holger...@gmail.com]

the performance improved significantly. previously the analysis of this test class took about 4-5 minutes. now it takes 4-5 seconds.
the class mentioned by Eric in this first post takes about 32 seconds now on my machine.

Thanks a lot.

[Nicolas Peru]

Hi, 

Thanks for the feedback and test,That's great news ! 

Please don't use this version of SonarJava in production, a proper fix will be part of the next release (SonarJava 4.15) linked to this ticket : 

https://jira.sonarsource.com/browse/SONARJAVA-2535

While this is a great improvement 32 seconds on one file still seems a huge amount of time for an analysis : would you be able to send a .nps file on those files taking more than 30 seconds ?  at least to try to see if there are other obvious area of improvement and see where time is spent ? 

Thanks for your help !

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/d0c7cd12-c38f-4daa-a851-1f2c1c9b79fc%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Holger Mensch]

I have attached a JVisualVM sample of the analysis with your patched version of SonarJava.

To unsubscribe from this group and stop receiving emails from it, send an email to sonarlint+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/d0c7cd12-c38f-4daa-a851-1f2c1c9b79fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nicolas Peru]

Hi, 

Thanks for this ! 

I was able to spot another area for improvement : Would you be able to test that patch ? and let me know if it improves things in your case ?https://expirebox.com/download/f5cdb89d21a183fe63de4cd101f1f724.html 

Again this version is not supposed to be used in production and the link will be available for two days.

Thanks again for your help ! That's really appreciated.

Cheers, 

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarlint/d0c7cd12-c38f-4daa-a851-1f2c1c9b79fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Nicolas Peru | SonarSource

Senior Developer

https://twitter.com/benzonico

https://sonarsource.com

--

[Nicolas Peru]

Hi, 

Thanks for the test and new snapshot. So the new improvement made reduced the time to analyse this file and we are now down to a problem of classpath size as far as I can see (There are also some visit of imports declaration that could be spared, which I improved with the ticket mentioned previously). 

What I mean by classpath size problem is that it takes ~100ms to find and load a .class in your classpath which seems quite big from your first logs. 

I am surprised for instance to see in your first logs that you have some jars coming from : 

"workspace\.metadata\.plugins" : are those really required for your project to compile ?

SonarLint (if i'm correct) uses your project configuration to figure out the classpath required for SonarJava. I suspect you have too many jars/paths configured in your classpath on eclipse and that leads to those slow lookup for classes. 

Cheers, 

[holger...@gmail.com]

Hi,

no problem; I am glad that you fixed the issue very fast.
This plugin (and some others...) has quite a lot dependencies; we know about that problem...

I suppose the class path is extended with the target platform path somewhere in "workspace\.metadata\.plugins". thus they are required to compile.