Project Analysis Failures - IllegalStateException

[Sam A]

Hi everyone,

We have recently been experiencing some strange behaviour in SonarQube and was hoping the community could provide some guidance/help on how to troubleshoot the issue. 

We are running SonarQube 6.4 and have noticed that after a while our analysis' all start failing with the below error. Restarting the SonarQube service seems to resolve the issue, however after a while it starts happening again (this has occurred twice in the last fortnight).

Previously everything had been running fine for several months.

The only non-standard Compute Engine config we have made was to set sonar.ce.javaAdditionalOpts=-Xmx768m

I'd really appreciate any suggestions as to what could be the cause, and where to start looking to resolve the issue (doesn't seem to be any similar issue in the SonarQube Google group)

java.lang.IllegalStateException: load called twice for thread 'ce-worker-0' or state wasn't cleared last time it was used

at com.google.common.base.Preconditions.checkState(Preconditions.java:197)

at org.sonar.server.setting.ThreadLocalSettings.load(ThreadLocalSettings.java:137)

at org.sonar.ce.settings.SettingsLoader.start(SettingsLoader.java:43)

at sun.reflect.GeneratedMethodAccessor461.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110)

at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89)

at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84)

at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169)

at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132)

at org.picocontainer.behaviors.Stored.start(Stored.java:110)

at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016)

at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009)

at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767)

at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:143)

at org.sonar.server.computation.task.container.TaskContainerImpl.<init>(TaskContainerImpl.java:44)

at org.sonar.server.computation.task.projectanalysis.container.ContainerFactoryImpl.create(ContainerFactoryImpl.java:32)

at org.sonar.server.computation.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:72)

at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:92)

at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:59)

at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:35)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Thanks in advance,

Sam

[Sebastien Lesaint]

Hello Sam,

I reckon such error is caused by the failure of the previous task that run on Thread ce-worker-0.

Do you have any failing task prior to this error?

Could you share the error stacktrace of it?

Cheers,

Seb'

Sébastien LESAINT | SonarSource
Platform Developer

https://sonarsource.com

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/71c0e344-482f-4638-a77b-51d22e43d3fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sebastie...@sonarsource.com]

============ c/p reply done by email which doesn't seem to have reached the group ============

Hello Sam,

I reckon such error is caused by the failure of the previous task that run on Thread ce-worker-0.

Do you have any failing task prior to this error?

Could you share the error stacktrace of it?

Cheers,

Seb'

[Sam A]

Hi Sebastian,

Thanks for the quick reply. I think I've managed to locate the offending task, here's the stacktrace:

Further investigation has revealed that the offending C# project (which is analysed weekly) is always failing with the same error. I'll have to look at the configuration of the project in more detail and see if I can get to the bottom of why it's failing.

java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:3332)
	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
	at java.lang.StringBuilder.append(StringBuilder.java:136)
	at org.sonar.server.computation.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.appendDuplication(DuplicationDataMeasuresStep.java:132)
	at org.sonar.server.computation.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.appendDuplication(DuplicationDataMeasuresStep.java:125)
	at org.sonar.server.computation.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.processDuplicationBlock(DuplicationDataMeasuresStep.java:114)
	at org.sonar.server.computation.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.createXmlDuplications(DuplicationDataMeasuresStep.java:100)
	at org.sonar.server.computation.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.computeDuplications(DuplicationDataMeasuresStep.java:85)
	at org.sonar.server.computation.task.projectanalysis.step.DuplicationDataMeasuresStep$DuplicationVisitor.visitFile(DuplicationDataMeasuresStep.java:80)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitNode(DepthTraversalTypeAwareCrawler.java:79)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:51)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:98)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:98)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:98)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visitImpl(DepthTraversalTypeAwareCrawler.java:54)
	at org.sonar.server.computation.task.projectanalysis.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:39)
	at org.sonar.server.computation.task.projectanalysis.step.DuplicationDataMeasuresStep.execute(DuplicationDataMeasuresStep.java:67)
	at org.sonar.server.computation.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:64)
	at org.sonar.server.computation.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:52)
	at org.sonar.server.computation.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:75)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:92)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:59)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:35)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)

Cheers,

Sam

[Sebastien Lesaint]

Hi Sam,

I'm assuming you pasted the error which you accessed from the UI.

Could you have a look at the logs? To check whether any other error occurred during the processing of this task.

Have a look at file logs/ce.log.

You can grep on the content of this file on the UUID of the failing task to see all the logs of this single task. (the task uuid is available in the UI).

Cheers,

Sébastien LESAINT | SonarSource
Platform Developer

https://sonarsource.com

--

You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/0d9ae86e-78d0-47f8-8dc2-3769a6bcb5f3%40googlegroups.com.

[Sebastien Lesaint]

Hello,

Following up on this thread. I managed to reproduce the reported error. Here is the ticket fixing it: https://jira.sonarsource.com/browse/SONAR-9973.

Cheers,

Sébastien LESAINT | SonarSource
Platform Developer

https://sonarsource.com

[Sam A]

Hi Sébastien,

Thanks for the update - sorry I didn't get back to you earlier, I've been on leave for the last few weeks and have only just returned. The fix for this item, combined with the new ability to receive notifications for failed background tasks added in 6.6, will certainly make for a much improved experience!

Cheers,

Sam