Sonar Upgrade memory errors

[Stas Roshkolets]

Hello,
During Sonar upgrade from version 4.5.6 to 5.5 we get errors about memory lack.

First time error was:

2016.09.15 12:47:57 ERROR web[o.a.t.u.n.NioEndpoint]

java.lang.OutOfMemoryError: Java heap space

Second time after setting sonar.web.javaOpts to -Xmx1536m -Xms128m we got:
GC overhead limit exceeded

May you advice something regarding this?

[Freddy Mallet]

In both cases this is due to a lack of memory. Do you have the ability to try setting -Xmx to 2048m ?

How many projects/issues do you have in your SonarQube instance ? 

Cheers

Freddy

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/4b277624-0276-40a5-bf92-980f848d33df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Freddy MALLET | SonarSource

Product Director & Co-Founder
http://sonarsource.com

[Stas Roshkolets]

There are a lot of projects.
dmp file for DB Backup  is 32Gb, so there are a lot of stuff.
Where should i set -Xmx?
sonar.web.javaOpts or sonar.ce.javaOpts or sonar.search.javaOpts? Or all together?
Should i set only -Xmx or it must be the full settings like in commented lines of conf files. For example, #sonar.web.javaOpts=-Xmx768m -XX:MaxPermSize=160m -XX:+HeapDumpOnOutOfMemoryError ?

[Simon Brandhof]

You didn't provide the logs for "GC overhead limit exceeded". Assuming error is in web server, you should uncomment sonar.web.javaOpts and keep the default values of other arguments, for example:
sonar.web.javaOpts=-Xmx1G -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true

Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/aa070ad3-5885-480d-af7a-230134663392%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Simon BRANDHOF | SonarSource
Tech Lead & Co-Founder
http://twitter.com/SimonBrandhof

[Stas Roshkolets]

Hello,
We set javaOpts as:
sonar.web.javaOpts=-Xmx4096m -Xms3072m  -XX:MaxPermSize=1024m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true

But still get an error:

java.lang.OutOfMemoryError: Java heap space

Dumping heap to java_pid15343.hprof ...

Heap dump file created [3878543380 bytes in 16.926 secs]

2016.09.18 14:00:16 INFO  web[o.s.s.d.m.PlatformDatabaseMigration] DB migration failed | time=4909632ms

2016.09.18 14:00:16 ERROR web[o.s.s.d.m.PlatformDatabaseMigration] DB Migration or container restart failed. Process ended with an exception

org.jruby.exceptions.RaiseException: (StandardError) An error has occurred, all later migrations canceled:

Java heap space

[Simon Brandhof]

Can you provide the full logs please ?

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/54d7646c-2382-49a8-aaea-b498ea04e7a9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Stas Roshkolets]

Hi,
Log attached.

[Simon Brandhof]

This upgrade step transforms sources of all files. Each file is loaded in memory, one by one. It seems that you have a file which content does not fit in memory. Could you check what's the biggest file in your db (version 4.5.6) ? SQL request is:

SELECT p.kee, dbms_lob.getlength(ss.data) FROM snapshot_sources ss
INNER JOIN snapshots s ON s.id=ss.snapshot_id
INNER JOIN projects p ON p.id=s.project_id
ORDER BY dbms_lob.getlength(ss.data) desc

[Stas Roshkolets]

There are some files with DBMS_LOB.GETLENGTH(SS.DATA) equal (null), and then :

2554533

2554533

2554533

2345008

2345008

2345008

2205904

2205904

2205904

2025661

2025661

2025661

1934594

1934293

1928174

1928174

1928173

1389431

1389431

1389431

1254767

1254767

1254767

...

[Simon Brandhof]

Nothing wrong here. 

The file java_pid15343.hprof is probably too big for sharing. Is it possible to have a look at it on your side ?

--

You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/b2548b06-86ae-4e05-804e-4728c7c5fd92%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Stas Roshkolets]

-rw-------  1 sonar1 aimsys 489M Sep 15 12:47 java_pid24697.hprof

drwx------  2 sonar1 aimsys 4.0K Sep 18 12:37 logs

drwx------ 10 sonar1 aimsys 4.0K Sep 18 13:59 .

-rw-------  1 sonar1 aimsys 3.7G Sep 18 14:00 java_pid15343.hprof

This is binary file so I can not open it.
What exactly do you need to know about this file?

[Simon Brandhof]

This file is a heap dump. It can be read by tools like VisualVM.

Before going further, can you check that system can allocate 4Gb to the Java process ? Note that "-Xms3072m -XX:MaxPermSize=1024m" should be removed.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/15a2f853-9e1e-4652-9576-0fe7f3a1399a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.