NullPointerException in SonarQube Scanner for Ant after update 6.6

[linus...@pttde.de]

Hi,

I get the following null pointer exception from the SonarQube Scanner for Ant when running an analysis after upgrading to SonarQube 6.6

java.lang.NullPointerException
	at org.sonar.api.batch.fs.internal.DefaultIndexedFile.hashCode(DefaultIndexedFile.java:140)
	at org.sonar.api.batch.fs.internal.DefaultInputFile.hashCode(DefaultInputFile.java:180)
	at java.util.HashMap.hash(HashMap.java:338)
	at java.util.HashMap.put(HashMap.java:611)
	at org.sonar.scanner.scan.DefaultComponentTree.index(DefaultComponentTree.java:40)
	at org.sonar.scanner.scan.filesystem.FileIndexer.indexFileAndParentDir(FileIndexer.java:206)
	at org.sonar.scanner.scan.filesystem.FileIndexer.indexFile(FileIndexer.java:185)
	at org.sonar.scanner.scan.filesystem.FileIndexer.access$400(FileIndexer.java:59)
	at org.sonar.scanner.scan.filesystem.FileIndexer$IndexFileVisitor.lambda$visitFile$0(FileIndexer.java:246)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Do you have any ideas what might be the problem?

Thanks,
Linus Lotz

[Julien HENRY]

Hi Linus,

This might be caused by a file that is located outside of your project baseDir. Could also be related to symlink. Can you give me your platform (Windows / Unix)?

We'll investigate but in the meantime, if you could isolate the offending file, and provide a reproducer, that would help.

Thanks!

Julien

[Linus Lotz]

Hi Julien,
I've removed a symlink that seemed to have caused the Null Pointer Exception, now I get a different error:

/var/lib/jenkins/workspace/commerce-webshop_sonar_test-DUBVT6YB3IFQ6T3FGNMI4TGUTM6E34UGNV5Y2V4VPYOHBSFMWAPA/hybris/bin/platform/resources/ant/sonar.xml:89: java.lang.IllegalStateException: Dir /var/lib/jenkins/workspace/commerce-webshop_sonar_test-DUBVT6YB3IFQ6T3FGNMI4TGUTM6E34UGNV5Y2V4VPYOHBSFMWAPA/hybris/bin/custom/xxxxx/xxxxxfacades/src/de/portaltech/xxxxx/facades/customerticket should be relative to project baseDir

    at org.sonar.scanner.scan.filesystem.InputComponentStore.lambda$getProjectRelativePath$3(InputComponentStore.java:157)

    at java.util.Optional.orElseThrow(Optional.java:290)

    at org.sonar.scanner.scan.filesystem.InputComponentStore.getProjectRelativePath(InputComponentStore.java:157)

    at org.sonar.scanner.scan.filesystem.InputComponentStore.put(InputComponentStore.java:151)

    at org.sonar.scanner.scan.filesystem.ModuleInputComponentStore.doAdd(ModuleInputComponentStore.java:86)

    at org.sonar.api.batch.fs.internal.DefaultFileSystem$Cache.add(DefaultFileSystem.java:209)

    at org.sonar.api.batch.fs.internal.DefaultFileSystem.add(DefaultFileSystem.java:184)

    at org.sonar.scanner.scan.filesystem.FileIndexer.indexFileAndParentDir(FileIndexer.java:204)

    at org.sonar.scanner.scan.filesystem.FileIndexer.indexFile(FileIndexer.java:185)

    at org.sonar.scanner.scan.filesystem.FileIndexer.access$400(FileIndexer.java:59)

    at org.sonar.scanner.scan.filesystem.FileIndexer$IndexFileVisitor.lambda$visitFile$0(FileIndexer.java:246)

    at java.util.concurrent.FutureTask.run(FutureTask.java:266)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

    at java.lang.Thread.run(Thread.java:748)

But this does not make any sense, as the projectBasedir ist set to

/var/lib/jenkins/workspace/commerce-webshop_sonar_test-DUBVT6YB3IFQ6T3FGNMI4TGUTM6E34UGNV5Y2V4VPYOHBSFMWAPA/hybris/bin/custom/xxxxx/xxxxxfacades

and the offending path is:

/var/lib/jenkins/workspace/commerce-webshop_sonar_test-DUBVT6YB3IFQ6T3FGNMI4TGUTM6E34UGNV5Y2V4VPYOHBSFMWAPA/hybris/bin/custom/xxxxx/xxxxxfacades/src/de/portaltech/xxxxx/facades/customerticket

which is inside the basedir

This is run in a docker container on Linux.

Thanks,
Linus

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/c7TNFwbVhVY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/0bab1ebb-40ed-4ec8-8cd0-e60a64995dce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julien HENRY]

Are any of the two paths made of symlinks?

Regarding the first error, I'm still interested to have more details about the symlink (or a reproducer) so that we can at least fail with a more meaningful message.

Thanks,

Julien Henry | SonarSource

Developer

https://sonarsource.com

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/0bab1ebb-40ed-4ec8-8cd0-e60a64995dce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/c7TNFwbVhVY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/cab6c147-a7ba-8cd8-00e2-fcdf969e627c%40pttde.de.

[Linus Lotz]

No the Path does not contain any symlinks.
The symlink was

/var/lib/jenkins/workspace/commerce-webshop_sonar_test-DUBVT6YB3IFQ6T3FGNMI4TGUTM6E34UGNV5Y2V4VPYOHBSFMWAPA/hybris/bin/custom

and linked to:

../../bin/custom

Thanks,
Linus

[Julien HENRY]

I'm not sure to understand. Was the symlink recursive?

Julien Henry | SonarSource

Developer

https://sonarsource.com

[Linus Lotz]

No, It pointed to the directory

/var/lib/jenkins/workspace/commerce-webshop_sonar_test-DUBVT6YB3IFQ6T3FGNMI4TGUTM6E34UGNV5Y2V4VPYOHBSFMWAPA/bin/custom

(the difference is the 'hybris' directory is not included in the path)
I downgraded now to sonarqube 6.5 and it is working again, so there must have been a change between 6.5 and 6.6 that caused this problem.

Thanks,
Linus

[Julien HENRY]

Hi Linus,

I'm still not able to reproduce your initial NPE. Here is my sample project layout:

/

  /another_dir/src/Foo2.java

  /myproject/src/Foo1.java

            /src2 -> ../another_dir/src

I run the scanner from "myproject" directory, with sonar-scanner -Dsonar.projectKey=foo -Dsonar.sources=.

There were indeed some changes in 6.6 regarding file indexing, but if I don't manage to understand/reproduce your issue, it will be hard to fix the potential bug. We are very close to release 6.7 that will be the next LTS, so please help me to reproduce.

Thanks

Julien Henry | SonarSource

Developer

https://sonarsource.com

[Linus Lotz]

Hi Julian,
The Null Pointer Exception was replaced with the IllegalStateException after removing the symlink, so maybe it can only be caused when a IllegalStateExcpetion is caused, where a symlink is in the path?
The project is based on SAP Hybris 6.5 and uses their build scripts. Maybe the issue is related to their ant buildscripts provided by SAP. I'm afraid I cannot share them, as this would probably violate the license terms.

Thanks,
Linus

[Julien HENRY]

I'm not asking for sharing the entire build script. But trying to remove all unnecessary stuff (source code, folders) until you have a very minimal project that still have (any of the two) issue.

Julien Henry | SonarSource

Developer

https://sonarsource.com

[Thomas]

Hi Julien,

I have the same issue as above. I managed to reproduce it in a sample project

# must be unique in a given SonarQube instance

sonar.projectKey=my:project

# this is the name and version displayed in the SonarQube UI. Was mandatory prior to SonarQube 6.1.

sonar.projectName=My project

sonar.projectVersion=1.0

sonar.modules=../custom/module1

../custom/module1.sonar.projectKey=module1

#../custom/module1.sonar.sources=src

# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.

# This property is optional if sonar.modules is set.

sonar.sources=.

# Encoding of the source code. Default is default system encoding

#sonar.sourceEncoding=UTF-8

Thanks

Thomas 

To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/0bab1ebb-40ed-4ec8-8cd0-e60a64995dce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/c7TNFwbVhVY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

[Julien HENRY]

Hi Thomas,

I just made a test with SonarQube 6.6 and (hopefully) the same project configuration you provided, but I receive a different error that is expected due to https://jira.sonarsource.com/browse/SONAR-9835:

INFO: Scanner configuration file: /home/julien/Prog/Softs/sonar-scanner-3.0.1.733/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/julien/tmp/repro/parent/sonar-project.properties
INFO: SonarQube Scanner 3.0.1.733

[...]

INFO: Project key: my:project
INFO: -------------  Scan ../custom/module1

[...]

INFO: Base dir: /home/julien/tmp/repro/custom/module1
INFO: Working dir: /home/julien/tmp/repro/parent/.scannerwork/my_project_module1
INFO: Source paths: .
INFO: Source encoding: UTF-8, default locale: fr_FR
INFO: Index files
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 3.250s
INFO: Final Memory: 47M/208M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarQube Scanner execution
java.lang.IllegalStateException: Dir /home/julien/tmp/repro/custom/module1 should be relative to project baseDir

So again, please provide a better reproducer (with a zip file) or at least share your analysis logs.

Thanks,

Julien Henry | SonarSource

Developer

https://sonarsource.com

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/23603f2b-b7de-438d-b702-ec1349fdd391%40googlegroups.com.

[cc...@smart-trade.net]

Hi Julien,

We have the same problem with gradle projects since upgrading to the 6.7 version.

Here is our flat projects' layout with 'build' is the root project.

build:

 + build.gradle

 + settings.gradle: 

     includeFlat project1, project2, projectx

     rootProject.name = 'build'

project1

  + build.gradle

project2

  + build.gradle

projectx

  + build.gradle

When I tried to call the task "gradlew sonar" from build project, an error occured as following:

FAILURE: Build failed with an exception.

* What went wrong:

Execution failed for task ':sonarqube'.

> Dir C:\ProjectsDir\project1\src\main\java\com\ourcompany\packagename should be relative to project baseDir

* Try:

Run with --info or --debug option to get more log output.

* Exception is:

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':sonarqube'.

        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:100)

        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:70)

        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:63)

        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)

        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)

        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:88)

        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:52)

        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)

        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)

        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)

        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)

        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.run(DefaultTaskGraphExecuter.java:248)

        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)

        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)

        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:197)

        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:107)

        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:241)

        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:230)

        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.processTask(DefaultTaskPlanExecutor.java:124)

        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.access$200(DefaultTaskPlanExecutor.java:80)

        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:105)

        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:99)

        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.execute(DefaultTaskExecutionPlan.java:625)

        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.executeWithTask(DefaultTaskExecutionPlan.java:580)

        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:99)

        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)

        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)

        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)

Caused by: java.lang.IllegalStateException: Dir C:\ProjectsDir\project1\src\main\java\com\ourcompany\packagename should be relative to project baseDir

        at org.sonar.scanner.scan.filesystem.InputComponentStore.lambda$getProjectRelativePath$3(InputComponentStore.java:157)

        at org.sonar.scanner.scan.filesystem.InputComponentStore.getProjectRelativePath(InputComponentStore.java:157)

        at org.sonar.scanner.scan.filesystem.InputComponentStore.put(InputComponentStore.java:151)

        at org.sonar.scanner.scan.filesystem.ModuleInputComponentStore.doAdd(ModuleInputComponentStore.java:86)

        at org.sonar.api.batch.fs.internal.DefaultFileSystem$Cache.add(DefaultFileSystem.java:209)

        at org.sonar.api.batch.fs.internal.DefaultFileSystem.add(DefaultFileSystem.java:184)

        at org.sonar.scanner.scan.filesystem.FileIndexer.indexFileAndParentDir(FileIndexer.java:204)

        at org.sonar.scanner.scan.filesystem.FileIndexer.indexFile(FileIndexer.java:185)

        at org.sonar.scanner.scan.filesystem.FileIndexer.access$400(FileIndexer.java:59)

        at org.sonar.scanner.scan.filesystem.FileIndexer$IndexFileVisitor.lambda$visitFile$0(FileIndexer.java:246)

Could you please suggest us a solution for this restrict?

I find that you had a solution for maven project in this jira: https://jira.sonarsource.com/browse/MSONAR-164

How about gradle project?

Or do you plan to support back compatibility in the next version 6.8?

Thanks in advance,

Chinh Ho

[Julien HENRY]

Hi Chinh Ho,

Thanks for the pointer about "includeFlat" feature. This is something I was not aware of. I have created a ticket to fix the Scanner for Gradle to support such projects out of the box:

https://jira.sonarsource.com/browse/SONARGRADL-48

I also mentioned a workaround in the ticket, that you can use in the meantime.

++

Julien Henry | SonarSource

Developer

https://sonarsource.com

---

     

How trading firms can leverage technology not just to achieve regulatory compliance but also to improve their performance - Read more

smartTrade Technologies awarded best liquidity aggregation system provider - Read more

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/c7TNFwbVhVY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/6b6b7b0f-ffeb-4adf-9bf7-f6ba0fdf2191%40googlegroups.com.

[cc...@smart-trade.net]

Hi Julien,

Thanks for your prompt response and solution.

It worked like a charm!!

Regards,

Chinh HO