Problems limiting groovyserv's max memory

20 views
Skip to first unread message

ale...@inqloud.com

unread,
Feb 19, 2018, 2:00:11 PM2/19/18
to GroovyServ
I start groovyserv manually and I see that my memory limit is set for the java process -Xmx384m.
Some time later the memory limit seems to disappear as I've seen that the java process consumes much more memory.
Is this happening after a client restarts the server? Why would my client restart a server if the server is running?

Basically I am having trouble with ensuring a few java related environment variables, i.e. -Xmx and -Dfile.encoding=UTF-8

Ideas?

nobeans

unread,
Feb 19, 2018, 9:21:13 PM2/19/18
to groov...@googlegroups.com
Hi,

How did you specify the arguments?
I've tried now as follows:

```
$ JAVA_OPTS="-Xmx128m -Dfile.encoding=UTF-8 " groovyserver -r
(a space is required in order to avoid https://github.com/kobo/groovyserv/issues/79)

$ jcmd 35315 VM.command_line
35315:
VM Arguments:
jvm_args: -Xmx128m -Dfile.encoding=UTF-8 -Djava.awt.headless=true .....

$ build/executables/bin/groovyclient -e "println('Hello')"
Hello

$ jcmd 35315 VM.command_line
35315:
VM Arguments:
jvm_args: -Xmx128m -Dfile.encoding=UTF-8 -Djava.awt.headless=true .....
--> still OK.

$ groovyclient -Cr -v
....
--> This command restarts the server process.

$ jcmd 35315 VM.command_line
35315:
java.io.IOException: No such process

$ jcmd 36161 VM.command_line
36161:
VM Arguments:
jvm_args: -Djava.awt.headless=true ....
```

It's working expectedly.

Did you use any options like '-Cr' with groovyclient?

--
You received this message because you are subscribed to the Google Groups "GroovyServ" group.
To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Yasuharu Nakano

Alexey Panteleev

unread,
Feb 19, 2018, 11:25:23 PM2/19/18
to groov...@googlegroups.com
I start the server like this:

>groovyserver -r -v -t 100 --allow-from <IP>

The java options are defined in my environment:

JAVA_OPTS=-Xmx384m
JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF-8

Then groovyclient can be called many times by procmail (for email filtering)

>groovyclient -cp $GROOVYSERV/lib/* $GROOVYSERV/bin/MyApp.groovy

I used -Cenv LANG for a while too but not anymore.

After a while I find out that the server has lost the file.encoding=UTF-8 setting and -XMx because some special encoding done by our app stops working properly and the server’s java server runs to 4Gb of memory.

The client must be restarting the server but perhaps the environment is not set up when restarted by the client from procmail.
But I don’t understand why the server has to be restarted if I started it beforehand? And if it has to be restarted how to ensure the correct JAVA_ environment settings?

Thank you

-Alexey
> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Yasuharu Nakano
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to groovyserv+...@googlegroups.com.

Alexey Panteleev

unread,
Feb 19, 2018, 11:33:46 PM2/19/18
to groov...@googlegroups.com
To be exact the groovy client is called this way

>groovyclient -cp $GROOVYSERV/lib/* $GROOVYSERV/bin/MyApp.groovy -s "$HDR_FROM" -t $ORIG_TO $ROBOT_MODE_ARG

$ROBOT_MODE_ARG is mainly empty but for some emails it is “-r” and meant to be an option for MyApp.groovy

I wonder if “-r” maybe somehow treated as “-Cr”? Possible?

Alexey Panteleev

unread,
Feb 21, 2018, 2:15:17 PM2/21/18
to groov...@googlegroups.com
Java process is running away:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11691 user 20 0 4068220 1.753g 6876 S 59.5 89.7 149:25.12 java

even though -Xmx384m was specified

user 11691 1 5 Feb19 ? 02:30:38 /usr/lib/jvm/default-java/bin/java -Xms128m -Xmx384m -server -Djava.awt.headless=true -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -classpath /usr/share/groovy2/lib/groovy-2.4.5.jar -Dscript.name=/usr/bin/groovy -Dprogram.name=groovy -Dgroovy.starter.conf=/usr/share/groovy2/conf/groovy-starter.conf -Dgroovy.home=/usr/share/groovy2 -Dtools.jar=/usr/lib/jvm/default-java/lib/tools.jar org.codehaus.groovy.tools.GroovyStarter --main groovy.ui.GroovyMain --conf /usr/share/groovy2/conf/groovy-starter.conf --classpath /usr/local/groovyserv-1.1.0/lib/*:. -e groovyx.groovyserv.GroovyServer.main(args) -- 1961 69.55.236.68 true -v


ps also shows

groovyclient -cp /home/user/tools/groovyserv/lib/filter-1.0-SNAPSHOT-jar-with-dependencies.jar /home/user/tools/groovyserv/bin/MyApp.groovy -s <...> -t <...> -r

nobeans

unread,
Feb 21, 2018, 8:56:39 PM2/21/18
to groov...@googlegroups.com
>groovyserver -r -v -t 100 --allow-from <IP>

I think you don't need to specify `--allow-from` option if you don't run a groovyserver process in the different computer.


> I wonder if “-r” maybe somehow treated as “-Cr”? Possible?

No. Just only '-Cr' or '-Crestart-server' will restart a server process.


I think that the server process just died for some reason. The "groovyclient" command try to run a new server process when there is no server process. That's why the process lost the environment variables you specified.

To know the reason which the server process was down, how about trying to run a server process with "-v,--verbose" option? The option makes a server process emit logs to `~/.groovy/groovyserv/groovyserver-<PID>.log`. However, because the amount of log is too many, pay attention to the diskful error or reduction in performance.


>>> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+unsubscribe@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>>> --
>>> Yasuharu Nakano
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to groovyserv+unsubscribe@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>

--
You received this message because you are subscribed to the Google Groups "GroovyServ" group.
To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Yasuharu Nakano

Alexey Panteleev

unread,
Feb 21, 2018, 11:48:13 PM2/21/18
to groov...@googlegroups.com
I have examined my existing log file I see many many repeating exceptions. Perhaps they’re cause memory leaks?

2018/02/21 20:36:09,990 [ERROR] (java.lang.ThreadGroup[name=system,maxpri=10]) (RequestWorker:168) Failed to send the exit status: 5
groovyx.groovyserv.exception.GServIOException: Failed to send exit status
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at groovyx.groovyserv.ClientConnection.sendExit(ClientConnection.groovy:110)
at groovyx.groovyserv.ClientConnection.sendExit(ClientConnection.groovy)
at groovyx.groovyserv.ClientConnection$sendExit$0.call(Unknown Source)
at groovyx.groovyserv.RequestWorker.closeSafely(RequestWorker.groovy:166)
at groovyx.groovyserv.RequestWorker.closeSafely(RequestWorker.groovy)
at java.lang.reflect.Method.invoke(Method.java:498)
at groovyx.groovyserv.RequestWorker.terminated(RequestWorker.groovy:177)
at java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:713)
at java.util.concurrent.ThreadPoolExecutor.shutdown(ThreadPoolExecutor.java:1404)
at java.util.concurrent.ThreadPoolExecutor.lambda$finalize$0(ThreadPoolExecutor.java:1491)
at java.security.AccessController.doPrivileged(Native Method)
at java.util.concurrent.ThreadPoolExecutor.finalize(ThreadPoolExecutor.java:1492)
at java.lang.System$2.invokeFinalize(System.java:1270)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
Caused by: java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.lang.reflect.Method.invoke(Method.java:498)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019)
at groovyx.groovyserv.ClientConnection$_sendExit_closure1.doCall(ClientConnection.groovy:106)
at java.lang.reflect.Method.invoke(Method.java:498)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019)
at groovy.lang.Closure.call(Closure.java:426)
at groovy.lang.Closure.call(Closure.java:442)
at groovyx.groovyserv.ClientConnection.sendExit(ClientConnection.groovy:103)
... 20 more
(sanitized)
2018/02/21 20:36:09,998 [ERROR] (java.lang.ThreadGroup[name=system,maxpri=10]) (IOUtils:44) Failed to close
java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at java_io_Closeable$close$0.call(Unknown Source)
at java_io_Closeable$close$0.call(Unknown Source)
at groovyx.groovyserv.utils.IOUtils.close(IOUtils.groovy:42)
at groovyx.groovyserv.utils.IOUtils$close$1.call(Unknown Source)
at groovyx.groovyserv.ClientConnection.close(ClientConnection.groovy:132)
at java_io_Closeable$close.call(Unknown Source)
at java_io_Closeable$close$0.call(Unknown Source)
at groovyx.groovyserv.utils.IOUtils.close(IOUtils.groovy:42)
at groovyx.groovyserv.utils.IOUtils$close$1.call(Unknown Source)
at groovyx.groovyserv.RequestWorker.closeSafely(RequestWorker.groovy:170)
at groovyx.groovyserv.RequestWorker.closeSafely(RequestWorker.groovy)
at java.lang.reflect.Method.invoke(Method.java:498)
at groovyx.groovyserv.RequestWorker.terminated(RequestWorker.groovy:177)
at java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:713)
at java.util.concurrent.ThreadPoolExecutor.shutdown(ThreadPoolExecutor.java:1404)
at java.util.concurrent.ThreadPoolExecutor.lambda$finalize$0(ThreadPoolExecutor.java:1491)
at java.security.AccessController.doPrivileged(Native Method)
at java.util.concurrent.ThreadPoolExecutor.finalize(ThreadPoolExecutor.java:1492)
at java.lang.System$2.invokeFinalize(System.java:1270)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
(sanitized)
> >>> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+...@googlegroups.com.
> >>> For more options, visit https://groups.google.com/d/optout.
> >>>
> >>>
> >>>
> >>> --
> >>> Yasuharu Nakano
> >>>
> >>> --
> >>> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> >>> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> >>> To unsubscribe from this group and all its topics, send an email to groovyserv+...@googlegroups.com.
> >>> For more options, visit https://groups.google.com/d/optout.
> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups "GroovyServ" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Yasuharu Nakano
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to groovyserv+...@googlegroups.com.

nobeans

unread,
Feb 25, 2018, 9:30:36 PM2/25/18
to groov...@googlegroups.com
I also sometimes saw the errors in my log files. I think the socket errors are no problem essentially.

Isn't there any errors else?

> >>> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+unsubscribe@googlegroups.com.

> >>> For more options, visit https://groups.google.com/d/optout.
> >>>
> >>>
> >>>
> >>> --
> >>> Yasuharu Nakano
> >>>
> >>> --
> >>> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> >>> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> >>> To unsubscribe from this group and all its topics, send an email to groovyserv+unsubscribe@googlegroups.com.

> >>> For more options, visit https://groups.google.com/d/optout.
> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups "GroovyServ" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Yasuharu Nakano
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to groovyserv+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "GroovyServ" group.
To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Yasuharu Nakano

Alexey Panteleev

unread,
Mar 2, 2018, 2:58:51 AM3/2/18
to groov...@googlegroups.com
I set the following java option -XX:MaxMetaspaceSize=1024m and now I know for sure that groovyserv is causing my jvm to run out of meatspace memory.

>java.lang.OutOfMemoryError: Metaspace

Why is this happening?
> > >>> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+...@googlegroups.com.
> > >>> For more options, visit https://groups.google.com/d/optout.
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Yasuharu Nakano
> > >>>
> > >>> --
> > >>> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> > >>> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> > >>> To unsubscribe from this group and all its topics, send an email to groovyserv+...@googlegroups.com.
> > >>> For more options, visit https://groups.google.com/d/optout.
> > >>
> > >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "GroovyServ" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
> >
> >
> > --
> > Yasuharu Nakano
> >
> > --
> > You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> > To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to groovyserv+...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "GroovyServ" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to groovyserv+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Yasuharu Nakano
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "GroovyServ" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/groovyserv/6F1vuCNVN5g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to groovyserv+...@googlegroups.com.

Alexey Panteleev

unread,
Mar 2, 2018, 3:11:36 AM3/2/18
to groov...@googlegroups.com
I wonder if -cp options the cause of this and the specified classes are loaded again and again?

groovyclient -cp /path/* script.groovy
Reply all
Reply to author
Forward
0 new messages