Memory usage with Java 8
45 views
Skip to first unread message
Toby G
Jul 27, 2018, 1:30:49 PM7/27/18
to Union Platform
I converted over to Java 8 (using OpenJDK Java, as I was before on Java 7),
and apparently memory usage has soared!
Before the conversion, my system had been running for 30 days consecutively,
with memory usage growing slowly over that time.
Toward the end, the memory usage looked like this:
MB used 57.062 free 23.722 all 80.784
Last night I switched over to Java 8, and it has now been running for less than 24 hours.
And it looks like this:
Memory: used 80.895 + free 24.213 = 105.108 MB
Never before in almost 6 years have I seen numbers that high.
Using the utility free it's pretty much the same story:
Before (with Java 7): total 1024, used ~300 free ~600
After (with Java 8): total 1024, used ~500 free ~400
It's fortunate I recently moved from a 512 MB VPS to one with 1024 MB
or this might be a severe problem.
What is going on here?
Might the Oracle Java 8 be a little less of a memory hog?
Has anyone else observed anything similar to this?
and apparently memory usage has soared!
Before the conversion, my system had been running for 30 days consecutively,
with memory usage growing slowly over that time.
Toward the end, the memory usage looked like this:
MB used 57.062 free 23.722 all 80.784
Last night I switched over to Java 8, and it has now been running for less than 24 hours.
And it looks like this:
Memory: used 80.895 + free 24.213 = 105.108 MB
Never before in almost 6 years have I seen numbers that high.
Using the utility free it's pretty much the same story:
Before (with Java 7): total 1024, used ~300 free ~600
After (with Java 8): total 1024, used ~500 free ~400
It's fortunate I recently moved from a 512 MB VPS to one with 1024 MB
or this might be a severe problem.
What is going on here?
Might the Oracle Java 8 be a little less of a memory hog?
Has anyone else observed anything similar to this?
Toby G
Jul 27, 2018, 7:55:10 PM7/27/18
to Union Platform
I think I will need to go back to Java 7 (at least for a while)
As the day played out, the memory usage is going like this:
07-26 23:55:57,134 - used 16.655 + free 4.765 = 21.420 MB
07-27 00:45:25,876 - used 41.790 + free 8.546 = 50.336 MB
07-27 01:40:42,180 - used 40.739 + free 10.829 = 51.568 MB
07-27 02:27:30,248 - used 38.966 + free 16.134 = 55.100 MB
07-27 04:38:10,310 - used 40.782 + free 21.350 = 62.132 MB
07-27 06:36:25,860 - used 40.831 + free 28.905 = 69.736 MB
07-27 07:31:13,787 - used 50.721 + free 28.635 = 79.356 MB
07-27 10:46:17,894 - used 46.330 + free 36.126 = 82.456 MB
07-27 11:53:10,980 - used 65.702 + free 24.794 = 90.496 MB
07-27 12:23:13,720 - used 80.895 + free 24.213 = 105.108 MB
07-27 16:40:02,819 -: used 74.902 + free 30.206 = 105.108 MB
07-27 17:24:17,405 -: used 87.718 + free 28.806 = 116.524 MB
07-27 18:21:28,866 -: used 96.556 + free 19.968 = 116.524 MB
07-27 18:59:39,753 -: used 95.055 + free 21.469 = 116.524 MB
07-27 19:40:59,452 -: used 91.540 + free 28.965 = 120.504 MB
That just doesn't fly - it seems to just gobble up memory like there's no tomorrow.
If anyone reads this (which appears to hardly be the case), and has any suggestions, let me know, please!
As the day played out, the memory usage is going like this:
07-26 23:55:57,134 - used 16.655 + free 4.765 = 21.420 MB
07-27 00:45:25,876 - used 41.790 + free 8.546 = 50.336 MB
07-27 01:40:42,180 - used 40.739 + free 10.829 = 51.568 MB
07-27 02:27:30,248 - used 38.966 + free 16.134 = 55.100 MB
07-27 04:38:10,310 - used 40.782 + free 21.350 = 62.132 MB
07-27 06:36:25,860 - used 40.831 + free 28.905 = 69.736 MB
07-27 07:31:13,787 - used 50.721 + free 28.635 = 79.356 MB
07-27 10:46:17,894 - used 46.330 + free 36.126 = 82.456 MB
07-27 11:53:10,980 - used 65.702 + free 24.794 = 90.496 MB
07-27 12:23:13,720 - used 80.895 + free 24.213 = 105.108 MB
07-27 16:40:02,819 -: used 74.902 + free 30.206 = 105.108 MB
07-27 17:24:17,405 -: used 87.718 + free 28.806 = 116.524 MB
07-27 18:21:28,866 -: used 96.556 + free 19.968 = 116.524 MB
07-27 18:59:39,753 -: used 95.055 + free 21.469 = 116.524 MB
07-27 19:40:59,452 -: used 91.540 + free 28.965 = 120.504 MB
That just doesn't fly - it seems to just gobble up memory like there's no tomorrow.
If anyone reads this (which appears to hardly be the case), and has any suggestions, let me know, please!
Toby G
Jul 29, 2018, 12:35:16 PM7/29/18
to Union Platform
And a few more numbers:
From using top I get these numbers for "VIRT" and "RES" for the java process:
Java 7: after about 26 hours of running: 1398380 197840 (1.3 GB Virt, 195 MB Res)
Java 8: after about 20 hours of running: 2447096 545940 (2.4 GB Virt, 540 MB Res)
That is no small difference!
There are command line parameters you can use (java -X...) to limit memory usage,
but it seems kind of odd to need to use them now, after going some 6 years without that need.
From using top I get these numbers for "VIRT" and "RES" for the java process:
Java 7: after about 26 hours of running: 1398380 197840 (1.3 GB Virt, 195 MB Res)
Java 8: after about 20 hours of running: 2447096 545940 (2.4 GB Virt, 540 MB Res)
That is no small difference!
There are command line parameters you can use (java -X...) to limit memory usage,
but it seems kind of odd to need to use them now, after going some 6 years without that need.
Toby
Sep 25, 2018, 5:00:23 PM9/25/18
to Union Platform
Not that anyone appears to be reading any of this, but at least it's a place to vent a little.
I put some limits on Java, running with
java -Xmx80m -XX:MaxMetaspaceSize=80m
and that seemed to put a cap on memory usage
(although it was still way higher than with Java 7).
And then after chugging along just fine for about 11 days,
the server got more clients than usual (although well under the
max that was reached in the past), and it all came crashing down
due to "OutOfMemory on Metaspace".
First time that happened in over 6 years of running, mostly on Java 7.
Congratulations Java/Nashorn developers ... splendid job there wasting memory!
I put some limits on Java, running with
java -Xmx80m -XX:MaxMetaspaceSize=80m
and that seemed to put a cap on memory usage
(although it was still way higher than with Java 7).
And then after chugging along just fine for about 11 days,
the server got more clients than usual (although well under the
max that was reached in the past), and it all came crashing down
due to "OutOfMemory on Metaspace".
First time that happened in over 6 years of running, mostly on Java 7.
Congratulations Java/Nashorn developers ... splendid job there wasting memory!
Toby
Oct 28, 2018, 5:54:21 PM10/28/18
to Union Platform
Well, I have finally given up on Java 8 (at least for one of my projects).
Even with " java -Xmx80m -XX:MaxMetaspaceSize=96m " specified to limit the memory use,
it just grows, seemingly without bound. The first one, limiting the heap, DOES seem to be effective,
in that the heap stops growing just under 80 MB. I don't know about the other one,
but I do know that the longer the process runs, the more memory it gobbles up.
It is possible I have a memory leak in my code, but there is no sign of that when running on Java 7.
There, without any limiting specs, the heap tops out at about 72 MB,
and the process stays at around 200 MB for the resident size and 1.4 GB for the virtual size.
All those numbers are much lower than the SAME CODE running on Java 8.
It would appear that these issues are related to using Union accounts, which this project does use.
I have another, smaller one, that doesn't use Union accounts, and the stats there -
even with Java 8 in use are like this:
It has been running continuously for 42 days (the other one would never make it anywhere near that),
The heap is at around 30 MB, the resident memory is 181 MB and virtual is 2.2 GB.
As with the other one, total memory used is growing, but very slowly.
Total in-use memory on the system was in the 200-MB range at the start,
and now is up to 366 MB. It will run out eventually, too, but after a much longer time.
Maybe these issues will be fixed in later releases of Java, but I may be on Java 7 forever otherwise,
which cannot be a good thing.
Even with " java -Xmx80m -XX:MaxMetaspaceSize=96m " specified to limit the memory use,
it just grows, seemingly without bound. The first one, limiting the heap, DOES seem to be effective,
in that the heap stops growing just under 80 MB. I don't know about the other one,
but I do know that the longer the process runs, the more memory it gobbles up.
It is possible I have a memory leak in my code, but there is no sign of that when running on Java 7.
There, without any limiting specs, the heap tops out at about 72 MB,
and the process stays at around 200 MB for the resident size and 1.4 GB for the virtual size.
All those numbers are much lower than the SAME CODE running on Java 8.
It would appear that these issues are related to using Union accounts, which this project does use.
I have another, smaller one, that doesn't use Union accounts, and the stats there -
even with Java 8 in use are like this:
It has been running continuously for 42 days (the other one would never make it anywhere near that),
The heap is at around 30 MB, the resident memory is 181 MB and virtual is 2.2 GB.
As with the other one, total memory used is growing, but very slowly.
Total in-use memory on the system was in the 200-MB range at the start,
and now is up to 366 MB. It will run out eventually, too, but after a much longer time.
Maybe these issues will be fixed in later releases of Java, but I may be on Java 7 forever otherwise,
which cannot be a good thing.
Derek
Jun 27, 2019, 12:36:06 PM6/27/19
to Union Platform
Hi Toby,
I received another report of someone having memory issues with Java 8 (but not Java 7). I'll take a look at the Account angle you've raised. While we run a server on Java 8 (1.8.0_121) that processes millions of connections over months without memory issues that use case doesn't involve Accounts.
Thanks!
Derek
Toby
Jul 13, 2019, 10:50:53 AM7/13/19
to Union Platform
Thank you! It is nice to see a little spark of life on this board (especially from the Union team!) -
sometimes I think I'm (almost) the only one here.
Toby
Jul 13, 2019, 11:05:12 AM7/13/19
to Union Platform
One other possible clue:
It is not simply the USE of Accounts for logging in, since that happens all the time (hundreds of times per day).
It appears that memory usage goes up especially when new Accounts are CREATED
(which happens much less often - at most a few times per day).
I would note that memory usage also grows using Java 7, but much more slowly.
It appears to become 'too much' after (roughly) a week, whereas with Java 8,
it's more like a matter of hours.
Here is a snapshot from the last few days running Java 7:
07-06 19:14:03,137 - used 9.117 + free 8.255 = 17.372 MB ... immediately after restart
07-06 19:45:48,213 - used 33.617 + free 12.331 = 45.948 MB ... a few rooms added (and more will be added over time)
07-06 20:19:35,419 - used 23.462 + free 23.994 = 47.456 MB
07-06 21:29:22,234 - used 33.643 + free 29.821 = 63.464 MB
07-06 23:40:10,036 - used 49.005 + free 14.395 = 63.400 MB
07-07 08:41:12,581 - used 35.234 + free 28.230 = 63.464 MB
07-07 09:18:00,355 - used 51.914 + free 12.658 = 64.572 MB
07-07 13:46:52,308 - used 45.920 + free 21.668 = 67.588 MB
07-07 19:48:04,057 - used 32.559 + free 36.701 = 69.260 MB
07-07 23:18:32,855 - used 44.374 + free 26.362 = 70.736 MB
07-09 13:10:00,137 - used 38.904 + free 36.788 = 75.692 MB
07-09 15:24:49,090 - used 45.249 + free 32.279 = 77.528 MB
07-10 09:24:22,704 - used 53.016 + free 25.024 = 78.040 MB
07-10 12:05:26,549 - used 31.482 + free 46.622 = 78.104 MB
07-10 13:08:41,774 - used 54.757 + free 23.283 = 78.040 MB
07-10 13:55:10,725 - used 57.071 + free 21.033 = 78.104 MB
07-10 15:08:27,701 - used 48.065 + free 29.975 = 78.040 MB
07-10 17:17:04,279 - used 51.214 + free 26.890 = 78.104 MB
07-10 17:51:59,760 - used 31.124 + free 46.916 = 78.040 MB
07-11 10:21:59,756 - used 65.456 + free 13.660 = 79.116 MB
07-12 16:34:45,435 - used 68.883 + free 13.973 = 82.856 MB
07-12 17:47:55,067 - used 45.863 + free 37.317 = 83.180 MB
(time to restart when total goes over (conservatively) 80 MB, I believe, as that's when problems can start to occur)
07-13 03:40:16,787 - used 9.419 + free 7.725 = 17.144 MB
It is not simply the USE of Accounts for logging in, since that happens all the time (hundreds of times per day).
It appears that memory usage goes up especially when new Accounts are CREATED
(which happens much less often - at most a few times per day).
I would note that memory usage also grows using Java 7, but much more slowly.
It appears to become 'too much' after (roughly) a week, whereas with Java 8,
it's more like a matter of hours.
Here is a snapshot from the last few days running Java 7:
07-06 19:14:03,137 - used 9.117 + free 8.255 = 17.372 MB ... immediately after restart
07-06 19:45:48,213 - used 33.617 + free 12.331 = 45.948 MB ... a few rooms added (and more will be added over time)
07-06 20:19:35,419 - used 23.462 + free 23.994 = 47.456 MB
07-06 21:29:22,234 - used 33.643 + free 29.821 = 63.464 MB
07-06 23:40:10,036 - used 49.005 + free 14.395 = 63.400 MB
07-07 08:41:12,581 - used 35.234 + free 28.230 = 63.464 MB
07-07 09:18:00,355 - used 51.914 + free 12.658 = 64.572 MB
07-07 13:46:52,308 - used 45.920 + free 21.668 = 67.588 MB
07-07 19:48:04,057 - used 32.559 + free 36.701 = 69.260 MB
07-07 23:18:32,855 - used 44.374 + free 26.362 = 70.736 MB
07-09 13:10:00,137 - used 38.904 + free 36.788 = 75.692 MB
07-09 15:24:49,090 - used 45.249 + free 32.279 = 77.528 MB
07-10 09:24:22,704 - used 53.016 + free 25.024 = 78.040 MB
07-10 12:05:26,549 - used 31.482 + free 46.622 = 78.104 MB
07-10 13:08:41,774 - used 54.757 + free 23.283 = 78.040 MB
07-10 13:55:10,725 - used 57.071 + free 21.033 = 78.104 MB
07-10 15:08:27,701 - used 48.065 + free 29.975 = 78.040 MB
07-10 17:17:04,279 - used 51.214 + free 26.890 = 78.104 MB
07-10 17:51:59,760 - used 31.124 + free 46.916 = 78.040 MB
07-11 10:21:59,756 - used 65.456 + free 13.660 = 79.116 MB
07-12 16:34:45,435 - used 68.883 + free 13.973 = 82.856 MB
07-12 17:47:55,067 - used 45.863 + free 37.317 = 83.180 MB
(time to restart when total goes over (conservatively) 80 MB, I believe, as that's when problems can start to occur)
07-13 03:40:16,787 - used 9.419 + free 7.725 = 17.144 MB
Reply all
Reply to author
Forward
0 new messages