[Mifos-developer] Memory leakage issue in mifos1.6
Biju K.A
Hi very one,
This is biju from grameenkoota, we are facing a serious issue while testing the mifos 1.6, we are frequently facing an error "heap memory error"
i will explain how we set up the system
We hired a stage server for testing the mifos1.6
HARDWARE
------
Intel Xeon Dual Core 3.0 GHZ * 2CPU
16 GB RAM
146+500 GB HDD
SOFTWARE
---------
OS : Ubuntu 8.04 LTS
web server : apache-tomcat-6.0.24-windows-x86 (binary version)
MYSQL : mysql-advanced-gpl-5.1.45-linux-i686-glibc23(binary version)
application : mifos-v1.6.0
we used one replica of our entire database(160 GB) in the server for the test purpose.
As per the information received from the mailing list we added an environmental veriable in the .profile setup file in the ubuntu like this :
EXPORT JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1536m -Xmx1536m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC"
Still it shows the "heap memory errors", if you use the application continuously
error messages are like this
------------------------------------
javax.servlet.ServletException: Servlet execution threw an exception
org.mifos.security.util.LoginFilter.doFilter(LoginFilter.java:85)
org.mifos.servlet.filters.MifosLogFilter.doFilter(MifosLogFilter.java:67)
org.mifos.framework.persistence.DatabaseInitFilter.doFilter(DatabaseInitFilter.java:51)
root cause
java.lang.OutOfMemoryError: Java heap space
java.io.ByteArrayOutputStream.<init>(Unknown Source)
org.apache.commons.fileupload.DeferredFileOutputStream.<init>(DeferredFileOutputStream.java:131)
org.apache.commons.fileupload.DefaultFileItem.getOutputStream(DefaultFileItem.java:558)
org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:406)
org.apache.struts.upload.CommonsMultipartRequestHandler.handleRequest(CommonsMultipartRequestHandler.java:193)
org.apache.struts.util.RequestUtils.populate(RequestUtils.java:443)
org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:805)
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:203)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1194)
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
org.mifos.security.util.LoginFilter.doFilter(LoginFilter.java:85)
org.mifos.servlet.filters.MifosLogFilter.doFilter(MifosLogFilter.java:67)
org.mifos.framework.persistence.DatabaseInitFilter.doFilter(DatabaseInitFilter.java:51)
--------------------
or error like this:
----------------------
java.lang.OutOfMemoryError: PermGen space
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
etc.........
Can any one please help us with some effective suggestion which will solve the problem with better way.........
Thanks&Regards
Biju. k.a
----------------------------------------
IT Department
Grameen Financial Services Pvt. Ltd
Mobile:+91-09035448060
Email: bi...@grameenkoota.org
Website: www.gfspl.in
Udai Gupta
> -server -Xms1536m -Xmx1536m -XX:NewSize=1024m -XX:MaxNewSize=1024m
> -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC"
EXPORT is not a command in linux (you should use "export")
I think it also matters where you are setting this environment
variable. The scope of environment variable should be visible to
tomcat otherwise mifos will run with low memory and throw OOM Error.
Please make sure you are setting the environment variable correctly.
https://help.ubuntu.com/community/EnvironmentVariables
http://en.wikipedia.org/wiki/Environment_variable
Udai
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
Biju K.A
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
fi
export CATALINA_HOME=/usr/local/tomcat6.0.24
export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.20
export MIFOS_CONF=/usr/local/mifos_config
export JAVA_OPTS="="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1536m -Xmx1536m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC"
mesg n
Biju K.A
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
fi
export CATALINA_HOME=/usr/local/tomcat6.0.24
export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.20
export MIFOS_CONF=/usr/local/mifos_config
mesg n
Udai Gupta
You can test by running a jsp as /tomcat/webapps/mifos/vars.jsp
--------vars.jsp--------------START
JAVA_OPTS : <%=System.getenv("JAVA_OPTS")%>
<br />
CATALINA_OPTS : <%=System.getenv("CATALINA_OPTS")%>
--------vars.jsp--------------END
Check what output you get by this jsp from tomcat.
Biju K.A
thanks for your valuable replay, it worth a lot ..
our old root/.profile was like this:
#-- starts
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
fi
export CATALINA_HOME=/usr/local/tomcat6.0.24
export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.20
export MIFOS_CONF=/usr/local/mifos_config
export JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1536m -Xmx1536m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC"
mesg n
Here we were not added the CATALINA_OPTS , this might the problem,
at present the our config file is like this:
# ~/.profile: executed by Bourne-compatible login shells.
if [ "$BASH" ]; then
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
fi
export CATALINA_HOME=/usr/local/tomcat6.0.24
export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.20
export MIFOS_CONF=/usr/local/mifos_config
export JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1536m -Xmx1536m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC"
mesg n
but can you clarify that the above two environmental variable is enough or require any change in the content of the variable
because we tried with this:
JAVA_OPTS="-Xms1024m -Xmx1024m -XX:PermSize=64m -XX:MaxPermSize=512m";
CATALINA_OPTS="-Xms1024m -Xmx1024m -XX:PermSize=64m-XX:MaxPermSize=512m";
it is not enough to solve the problem for us.
Thanks&Regards
Biju. k.a
----------------------------------------
IT Department
Grameen Financial Services Pvt. Ltd
Mobile:+91-09035448060
Email: bi...@grameenkoota.org
Website: www.gfspl.in
Udai Gupta
Read here what is the JAVA_OPTS and CATALINA_OPTS for tomcat.
http://svn.apache.org/repos/asf/tomcat/trunk/bin/catalina.sh
JAVA_OPTS is used by other applications too, I don't know the exact
reason why JAVA_OPTS was not working for you as it should be. You
don't need both for tomcat JVM memory allocation.
We are running Mifos on our servers but not with as high memory
allocation as you are, and it's working for us.
It would be interesting if you could make tomcat JVM heap dump when
running Mifos with -XX:MaxPermSize=512m and get a PermGen OOM Error.
>CATALINA_OPTS="-Xms1024m -Xmx1024m -XX:PermSize=64m-XX:MaxPermSize=512m";
One note though, you don't have space between two PermSize options.