Sonar Upgrade memory errors

551 views
Skip to first unread message

Stas Roshkolets

unread,
Sep 15, 2016, 10:10:22 AM9/15/16
to SonarQube
Hello,
During Sonar upgrade from version 4.5.6 to 5.5 we get errors about memory lack.

First time error was:
2016.09.15 12:47:57 ERROR web[o.a.t.u.n.NioEndpoint]
java.lang.OutOfMemoryError: Java heap space

Second time after setting sonar.web.javaOpts to -Xmx1536m -Xms128m we got:
GC overhead limit exceeded

May you advice something regarding this?

Freddy Mallet

unread,
Sep 15, 2016, 10:47:12 AM9/15/16
to Stas Roshkolets, SonarQube
In both cases this is due to a lack of memory. Do you have the ability to try setting -Xmx to 2048m ?

How many projects/issues do you have in your SonarQube instance ? 
Cheers
Freddy

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/4b277624-0276-40a5-bf92-980f848d33df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Freddy MALLET | SonarSource
Product Director & Co-Founder
http://sonarsource.com

Stas Roshkolets

unread,
Sep 15, 2016, 10:58:56 AM9/15/16
to SonarQube, stas...@gmail.com
There are a lot of projects.
dmp file for DB Backup  is 32Gb, so there are a lot of stuff.
Where should i set -Xmx?
sonar.web.javaOpts or sonar.ce.javaOpts or sonar.search.javaOpts? Or all together?
Should i set only -Xmx or it must be the full settings like in commented lines of conf files. For example, #sonar.web.javaOpts=-Xmx768m -XX:MaxPermSize=160m -XX:+HeapDumpOnOutOfMemoryError ?

Simon Brandhof

unread,
Sep 15, 2016, 12:03:08 PM9/15/16
to Stas Roshkolets, SonarQube
You didn't provide the logs for "GC overhead limit exceeded". Assuming error is in web server, you should uncomment sonar.web.javaOpts and keep the default values of other arguments, for example:
sonar.web.javaOpts=-Xmx1G -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true

Regards


For more options, visit https://groups.google.com/d/optout.
--
Simon BRANDHOF | SonarSource
Tech Lead & Co-Founder
http://twitter.com/SimonBrandhof

Stas Roshkolets

unread,
Sep 18, 2016, 8:11:51 AM9/18/16
to SonarQube, stas...@gmail.com
Hello,
We set javaOpts as:
sonar.web.javaOpts=-Xmx4096m -Xms3072m  -XX:MaxPermSize=1024m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true

But still get an error:
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid15343.hprof ...
Heap dump file created [3878543380 bytes in 16.926 secs]
2016.09.18 14:00:16 INFO  web[o.s.s.d.m.PlatformDatabaseMigration] DB migration failed | time=4909632ms
2016.09.18 14:00:16 ERROR web[o.s.s.d.m.PlatformDatabaseMigration] DB Migration or container restart failed. Process ended with an exception
org.jruby.exceptions.RaiseException: (StandardError) An error has occurred, all later migrations canceled:

Java heap space

Simon Brandhof

unread,
Sep 19, 2016, 2:52:32 AM9/19/16
to Stas Roshkolets, SonarQube
Can you provide the full logs please ?


For more options, visit https://groups.google.com/d/optout.

Stas Roshkolets

unread,
Sep 19, 2016, 3:27:20 AM9/19/16
to SonarQube, stas...@gmail.com
Hi,
Log attached.
sonar.log

Simon Brandhof

unread,
Sep 19, 2016, 5:48:47 AM9/19/16
to Stas Roshkolets, SonarQube
This upgrade step transforms sources of all files. Each file is loaded in memory, one by one. It seems that you have a file which content does not fit in memory. Could you check what's the biggest file in your db (version 4.5.6) ? SQL request is:

SELECT p.kee, dbms_lob.getlength(ss.data) FROM snapshot_sources ss
INNER JOIN snapshots s ON s.id=ss.snapshot_id
INNER JOIN projects p ON p.id=s.project_id
ORDER BY dbms_lob.getlength(ss.data) desc

Stas Roshkolets

unread,
Sep 19, 2016, 7:33:06 AM9/19/16
to SonarQube, stas...@gmail.com
There are some files with DBMS_LOB.GETLENGTH(SS.DATA) equal (null), and then :

2554533
2554533
2554533
2345008
2345008
2345008
2205904
2205904
2205904
2025661
2025661
2025661
1934594
1934293
1928174
1928174
1928173
1389431
1389431
1389431
1254767
1254767
1254767
...

Simon Brandhof

unread,
Sep 19, 2016, 9:07:17 AM9/19/16
to Stas Roshkolets, SonarQube
Nothing wrong here. 
The file java_pid15343.hprof is probably too big for sharing. Is it possible to have a look at it on your side ?

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Stas Roshkolets

unread,
Sep 19, 2016, 9:13:23 AM9/19/16
to SonarQube, stas...@gmail.com
-rw-------  1 sonar1 aimsys 489M Sep 15 12:47 java_pid24697.hprof
drwx------  2 sonar1 aimsys 4.0K Sep 18 12:37 logs
drwx------ 10 sonar1 aimsys 4.0K Sep 18 13:59 .
-rw-------  1 sonar1 aimsys 3.7G Sep 18 14:00 java_pid15343.hprof

This is binary file so I can not open it.
What exactly do you need to know about this file?

Simon Brandhof

unread,
Sep 19, 2016, 10:06:19 AM9/19/16
to Stas Roshkolets, SonarQube
This file is a heap dump. It can be read by tools like VisualVM.
Before going further, can you check that system can allocate 4Gb to the Java process ? Note that "-Xms3072m -XX:MaxPermSize=1024m" should be removed.



For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages