Failed to pop the queue of analysis reports

[andre.g...@gmail.com]

Hi all,

I currently face a problem in the most current SonarQube release 6.2. The issue is related to background tasks. After an analysis has been triggered during our CI cycle, a background task is scheduled on the server. Unfortunately, there is currently a background task which never finishes and I also cannot cancel or remove it. 

Checking the logs, I could identify the following messages (found in Sonarqube -> Administration -> System -> System Info -> Logs -> Web Server, see below):

I see two issues here:

1. There seems to be something UTF-8 related. 

2. I looks like Sonarqube is unable to connect to localhost:5432. As telnet works, it must be something related to authentication.

I could not find any information on how to resolve these two issues. Anyone has an idea?

Thank you very much,

André

-----

### Error updating database.  Cause: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00

### The error may involve org.sonar.db.ce.CeActivityMapper.insert-Inline

### The error occurred while setting parameters

### SQL: insert into ce_activity (       uuid,       component_uuid,       analysis_uuid,       status,       task_type,       is_last,       is_last_key,       submitter_login,       submitted_at,       started_at,       executed_at,       created_at,       updated_at,       execution_time_ms,       error_message,       error_stacktrace     )     values (       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?,       ?     )

### Cause: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154)

at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:141)

at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:51)

at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:52)

at com.sun.proxy.$Proxy36.insert(Unknown Source)

at org.sonar.db.ce.CeActivityDao.insert(CeActivityDao.java:54)

at org.sonar.ce.queue.CeQueueImpl.remove(CeQueueImpl.java:172)

at org.sonar.server.computation.queue.InternalCeQueueImpl.remove(InternalCeQueueImpl.java:100)

at org.sonar.server.computation.taskprocessor.CeWorkerCallableImpl.executeTask(CeWorkerCallableImpl.java:94)

at org.sonar.server.computation.taskprocessor.CeWorkerCallableImpl.call(CeWorkerCallableImpl.java:57)

at org.sonar.server.computation.taskprocessor.CeWorkerCallableImpl.call(CeWorkerCallableImpl.java:35)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2310)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2023)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:217)

at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:421)

at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:166)

at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:159)

at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)

at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)

at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44)

at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69)

at org.apache.ibatis.executor.ReuseExecutor.doUpdate(ReuseExecutor.java:50)

at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105)

at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71)

at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152)

... 18 common frames omitted

2017.01.27 13:12:51 ERROR ce[AVmsp3eIR7OoMSpDzWfT][o.s.s.c.t.CeWorkerCallableImpl] Failed to pop the queue of analysis reports

org.apache.ibatis.exceptions.PersistenceException: 

### Error querying database.  Cause: org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

### The error may exist in org.sonar.db.ce.CeQueueMapper

### The error may involve org.sonar.db.ce.CeQueueMapper.selectEligibleForPeek

### The error occurred while executing a query

### Cause: org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26)

at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:111)

at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:117)

at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:63)

at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:52)

at com.sun.proxy.$Proxy33.selectEligibleForPeek(Unknown Source)

at org.sonar.db.ce.CeQueueDao.peek(CeQueueDao.java:112)

at org.sonar.server.computation.queue.InternalCeQueueImpl.peek(InternalCeQueueImpl.java:70)

at org.sonar.server.computation.taskprocessor.CeWorkerCallableImpl.tryAndFindTaskToExecute(CeWorkerCallableImpl.java:66)

at org.sonar.server.computation.taskprocessor.CeWorkerCallableImpl.call(CeWorkerCallableImpl.java:51)

at org.sonar.server.computation.taskprocessor.CeWorkerCallableImpl.call(CeWorkerCallableImpl.java:35)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:270)

at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)

at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:211)

at org.postgresql.Driver.makeConnection(Driver.java:407)

at org.postgresql.Driver.connect(Driver.java:275)

at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)

at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)

at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)

at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)

at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)

at org.sonar.db.profiling.NullConnectionInterceptor.getConnection(NullConnectionInterceptor.java:31)

at org.sonar.db.profiling.ProfiledDataSource.getConnection(ProfiledDataSource.java:323)

at org.apache.ibatis.transaction.jdbc.JdbcTransaction.openConnection(JdbcTransaction.java:134)

at org.apache.ibatis.transaction.jdbc.JdbcTransaction.getConnection(JdbcTransaction.java:61)

at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:279)

at org.apache.ibatis.executor.ReuseExecutor.prepareStatement(ReuseExecutor.java:75)

at org.apache.ibatis.executor.ReuseExecutor.doQuery(ReuseExecutor.java:56)

at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:267)

at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:137)

at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:96)

at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:77)

at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:108)

... 17 common frames omitted

Caused by: java.net.ConnectException: Connection refused (Connection refused)

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:589)

at org.postgresql.core.PGStream.<init>(PGStream.java:64)

at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:150)

... 38 common frames omitted

[Sebastien Lesaint]

Hello,

You are clearly having connection issues between SonarQube and the PostgreSQL database. 

In the log you provided, we can see two statements failed.

The first one with " invalid byte sequence for encoding "UTF8": 0x00" error and the second one with a "Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.".

If the database was shutdown while the first statement was running, an odd error such as " invalid byte sequence for encoding "UTF8": 0x00" is not so surprising. For the second statement, the connection to the DB couldn't be opened at all.

You should check your setup and network.

Cheers,

Sébastien LESAINT | SonarSource
Platform Developer

http://sonarsource.com

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/c0b78c20-2b53-42b7-90f7-2a8b74054ea7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[andre.g...@gmail.com]

Hi Sébastien,

Thank you for your reply.

I also thought of a connection issue. But I failed to confirm that. The weird thing is, that the web interface is working (I am able to login and browse through pages / projects). Without a working database config, this wouldn't be possible, right?

My system setup is as follows:

- Debian 8.6 Jessie

- Sonarqube software runs under /opt/sonarqube

- Postgres database runs on the same hos. Database name is "sonarqube", user for access is "sonarqube".

My sonar.properties file in the /conf folder looks like this (stripped-off commented sections and irrelevant parts):

-----

# User credentials.

# Permissions to create tables, indices and triggers must be granted to JDBC user.

# The schema must be created first.

sonar.jdbc.username=sonarqube

sonar.jdbc.password=**********

#----- PostgreSQL 8.x/9.x

# If you don't use the schema named "public", please refer to http://jira.sonarsource.com/browse/SONAR-5000

#sonar.jdbc.url=jdbc:postgresql://localhost/sonar

sonar.jdbc.url=jdbc:postgresql://localhost/sonarqube

-----

pg_hba.conf looks like this:

-----

# TYPE  DATABASE        USER            ADDRESS                 METHOD

local       all                      postgres                                         peer

local       all                      all                                                  peer

host        all                      all                 127.0.0.1/32              md5

host        all                      all                 ::1/128                      md5

host        all                      all                 0.0.0.0/0                   md5

-----

my postgresql.conf looks like this (extract):

-----

listen_addresses = '*'

port = 5432                             

max_connections = 100           

-----

I tried to connect to the database from the host itself using:

postgres> psql -h localhost sonarqube sonarqube (enter)

The connection could be established successfully.

Is there anything other that I am missing?

Thank you very much.

Best regards,

André

To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

[Sebastien Lesaint]

Hello André,

Nothing obvious come to sight from the information you provide.

Maybe I missed it, but I don't remember you telling whether this problem is a one time thing or recurrent (even systematic).

In the former case, I'd just go with the BD being shutdown/restarted at the wrong time.

Cheers,

Sébastien LESAINT | SonarSource
Platform Developer

http://sonarsource.com

To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/7a6994a8-8756-4b9d-a31c-6c13eee3a422%40googlegroups.com.