Something has gone horribly wrong

[Toby]

I have documented my run-away memory problems with Java 8, and so I have stayed with Java 7.

For quite some time, everything ran quite well, and while memory use would grow slowly, I could have my app running continuously for weeks at a time.  No more.

Have I just been lucky these past 8 1/2 years?

Without making any changes that I know of, recently memory use is higher than it has been, and after just a few days, things get "gummed" up in some way, and users cannot connect to the server. 

So ... a Union restart is required.  But not even the stopserver command (locally on the server, no less) can connect, so I have to kill the Java process and restart manually.

I have no idea how to debug or resolve this problem due to Union being pretty much a black box, and so I will do daily restarts at a slack time.

There appears to be plenty of memory available on the system - after a fresh start, it's at around 50% overall (20% used by Union), and at worst seems to well under 60%.

I hope that I can get some help here, as there is no other possibility.

[Chris R]

Hey Toby,

I've been experiencing the same issues - server memory blows up slowly until it fails.

I'm a Java newbie, so I tried using Java VisualVM to understand the problem. 

Didn't really help but I noticed that using the tool reset the server memory. 

From there I googled a bit and found this:

https://www.theserverside.com/video/5-ways-to-force-Java-garbage-collection

The simplest method listed is the first:

---

1. Call System.gc()

Developers can call System.gc() anywhere in their code to instruct the JVM to prioritize garbage collection. When a developer calls this method -- and there isn't an extreme load on the JVM -- a Java GC cycle will happen within seconds.

---

I've been using a variant on the example server module provided by Union that profiles memory, altered slightly to output to both log and console.

I added the hack code above to run every time the module writes current memory to the log/console.

So far so good. Memory stays normal. Seems too good to be true, but I thought I'd pass it along.

See attached zip file containing union.xml and server module directory.

[Chris R]

Forgot to mention - I am testing it on two Windows 10 servers locally today - one running Java 8 on Windows 10 Pro, the other running Java 7 on Windows Home.

Memory stays low on both.

After tearing my hair out for weeks over this, I am astounded that one line of code could (seemingly) fix the issue. 

I'm assuming the hack might introduce issues of its own. Time will tell.

[Toby]

That's quite interesting Chris - and I'll have to try that sometime.

The issue that I brought up in February (wow! 8 months ago) has long been resolved .... by moving to a different server!

I am running Union on a VPS, which means I'm sharing a server with who-knows-what other applications.

It ran just fine for some years, and then ran into some mysterious problem that just really messed everything up.

Not knowing what else to do, I asked the host to move me to a different server (which they call a 'node'),

and just like that everything went back to normal.

With Java 7, memory usage grows, but very slowly.  I can run anywhere from a few days to a few weeks

without having to restart.  Your gc() work around might extend those periods even longer, or help with

the runaway memory use I experienced with Java 8.

Java is now many more versions ahead  of those, but that's another story.

Glad to see that at least one other person is reading this forum and occasionally posting.