Re: [Dspace-tech] DSpace memory issue

[Mark H. Wood]

My experience has been that, when Tomcat freezes and has to be killed,
it's run out of PermGen, which is a different pool than the main
heap. Indeed, tuning up PermGen accounts for most of the time I've
spent making Tomcat run well.

--
Mark H. Wood, Lead System Programmer mw...@IUPUI.Edu
Asking whether markets are efficient is like asking whether people are smart.

[Gabriel Dina]

Dear all,

 

We found in our DSpace installation (XMLUI) that JAVA uses a lot of memory for just a few items added in DSpace.

Also, from time to time DSpace freeze the tomcat process and we have to kill it.

 

Do you have some suggestions about this issues?

 

See bellow Java information from Control panel:

 

Java Runtime Environment Version:	1.6.0_29
Java Runtime Environment Vendor:	Sun Microsystems Inc.
Operating System Name:	Linux
Operating System Architecture:	amd64
Operating System Version:	2.6.32-71.29.1.el6.x86_64

Runtime statistics

Available processors:	4
Maximum memory:	1820 MiB
Allocated memory:	981 MiB
Used memory:	250 MiB
Free memory:	730 MiB

Cocoon Info

Cocoon Version:	2.2.0
Cocoon Work Directory:	/var/local/knossos/apache-tomcat-6.0.33/work/Catalina/localhost/xmlui
Cocoon Cache Directory:	/var/local/knossos/apache-tomcat-6.0.33/work/Catalina/localhost/xmlui/cache-dir
Main Cache Size (EHDefaultStore, 0x7dfcabd4):	39 (Clear Cache Immediately)
Transient Cache Size (DefaultTransientStore, 0x57752bea):	52
Transient Cache Size (DefaultTransientStore, 0x4d963c81):	0

 

 

Regards,

Gabriel

 

 

[Tim Donohue]

All,

To Matt's point, we have a few performance tuning recommendations posted
up in the Wiki Docs at:

https://wiki.duraspace.org/display/DSDOC18/Performance+Tuning+DSpace

This page provides hints on how give DSpace more memory to work with if
you keep encountering out-of-memory issues.

https://wiki.duraspace.org/display/DSDOC18/Performance+Tuning+DSpace

If you've already tried some of these options with no/little success,
please let us know on this list.

Also if anyone else on this list has updates to suggest for this
Performance Tuning documentation, please do let us know on this list!

- Tim

On 2/9/2012 9:53 AM, Henry, Matt - OPS District Computer Technician wrote:
> I have had this exact problem while running on windows. After switching
> to linux I ran into it again and fixed it with the following command:
>
> Export CATALINA_OPTS=-Xms1025m -Xmx1024m -XX:MaxPermSize=512m
>
> Honestly, I do not know what Xms and Xmx really stand for or what their
> purpose is. My best guess is that it increases the minimum and maximum
> memory pool of java catalina can use. The MaxPermSize option fixed the
> PermGen out of memory errors for me.
>
> Adjust the numbers as you see fit.
>
> Matt Henry
>
> On 2/9/12 9:42 AM, "Alexandre Magaz Graça"<alexand...@udl.cat> wrote:
>
>> Al 09/02/12 13:37, En/na Gabriel Dina ha escrit:

>>> Dear all,
>>>
>>> We found in our DSpace installation (XMLUI) that JAVA uses a lot of
>>> memory for just a few items added in DSpace.
>>>
>>> Also, from time to time DSpace freeze the tomcat process and we have to
>>> kill it.
>>>
>>> Do you have some suggestions about this issues?
>>>

>>> [...]
>>
>> Hello Gabriel,
>>
>> We're also having memory problems with DSpace. Although we have
>> increased Java heap space up to 1 GB it still ends up running out of
>> memory.
>>
>> Which DSpace version do you have? Do you see a trace like the one bellow
>> in DSpace logs when your Tomcat freezes?
>>
>> ERROR org.dspace.app.xmlui.cocoon.DSpaceCocoonServletFilter @ Serious
>> Error Occurred Processing Request!
>> javax.servlet.ServletException: Servlet execution threw an exception
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicati
>> onFilterChain.java:313)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilter
>> Chain.java:206)
>> at
>> org.dspace.app.xmlui.cocoon.SetCharacterEncodingFilter.doFilter(SetCharact
>> erEncodingFilter.java:111)
>> [...]
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>> at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)
>> at sun.nio.cs.StreamEncoder.<init>(StreamEncoder.java:194)
>> [...]
>>
>> Cheers,
>> Àlex
>>
>> --------------------------------------------------------------------------
>> ----
>> Virtualization& Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> DSpace-tech mailing list
>> DSpac...@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>
>
>
> ------------------------------------------------------------------------------
> Virtualization& Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> DSpace-tech mailing list
> DSpac...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech

[Alexandre Magaz Graça]

Al 09/02/12 13:37, En/na Gabriel Dina ha escrit:

> Dear all,
>
> We found in our DSpace installation (XMLUI) that JAVA uses a lot of
> memory for just a few items added in DSpace.
>
> Also, from time to time DSpace freeze the tomcat process and we have to
> kill it.
>
> Do you have some suggestions about this issues?
>

> [...]

Hello Gabriel,

We're also having memory problems with DSpace. Although we have
increased Java heap space up to 1 GB it still ends up running out of memory.

Which DSpace version do you have? Do you see a trace like the one bellow
in DSpace logs when your Tomcat freezes?

ERROR org.dspace.app.xmlui.cocoon.DSpaceCocoonServletFilter @ Serious
Error Occurred Processing Request!
javax.servlet.ServletException: Servlet execution threw an exception
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:313)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at

org.dspace.app.xmlui.cocoon.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:111)

[Henry, Matt - OPS District Computer Technician]

I have had this exact problem while running on windows. After switching
to linux I ran into it again and fixed it with the following command:

Export CATALINA_OPTS=-Xms1025m -Xmx1024m -XX:MaxPermSize=512m

Honestly, I do not know what Xms and Xmx really stand for or what their
purpose is. My best guess is that it increases the minimum and maximum
memory pool of java catalina can use. The MaxPermSize option fixed the
PermGen out of memory errors for me.

Adjust the numbers as you see fit.

Matt Henry

On 2/9/12 9:42 AM, "Alexandre Magaz Graça" <alexand...@udl.cat> wrote:

>Al 09/02/12 13:37, En/na Gabriel Dina ha escrit:
>> Dear all,
>>
>> We found in our DSpace installation (XMLUI) that JAVA uses a lot of
>> memory for just a few items added in DSpace.
>>
>> Also, from time to time DSpace freeze the tomcat process and we have to
>> kill it.
>>
>> Do you have some suggestions about this issues?
>>
>> [...]
>
>Hello Gabriel,
>
>We're also having memory problems with DSpace. Although we have
>increased Java heap space up to 1 GB it still ends up running out of
>memory.
>
>Which DSpace version do you have? Do you see a trace like the one bellow
>in DSpace logs when your Tomcat freezes?
>
>ERROR org.dspace.app.xmlui.cocoon.DSpaceCocoonServletFilter @ Serious
>Error Occurred Processing Request!
>javax.servlet.ServletException: Servlet execution threw an exception
> at

>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicati
>onFilterChain.java:313)
> at
>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilter
>Chain.java:206)
> at
>org.dspace.app.xmlui.cocoon.SetCharacterEncodingFilter.doFilter(SetCharact

>erEncodingFilter.java:111)
> [...]
>Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)
> at sun.nio.cs.StreamEncoder.<init>(StreamEncoder.java:194)
> [...]
>
>Cheers,
>Àlex
>

>--------------------------------------------------------------------------
>----
>Virtualization & Cloud Management Using Capacity Planning

[Tom De Mulder]

On Thu, 9 Feb 2012, Gabriel Dina wrote:

> We found in our DSpace installation (XMLUI) that JAVA uses a lot of memory
> for just a few items added in DSpace.

Even in the JSPUI there are memory leaks.

We have a nightly cronjob which restarts Tomcat to address the issue, even
though we fixed several of the memory leaks.

--
Tom De Mulder <td...@cam.ac.uk> - Cambridge University Computing Service
+44 1223 3 31843 - New Museums Site, Pembroke Street, Cambridge CB2 3QH
-> 09/02/2012 : The Moon is Waning Gibbous (93% of Full)

[Lewatle Phaladi]

Hi Matt and Dspace Team

Thanks for your time in looking at my problem and I am glad to let you know that your ideas worked, even if I never mentioned your name your effort is appreciated including just reading the message.

Regards,
Lewatle

------------------------------------------------------------------------------

Virtualization & Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

<html><p><font face = "verdana" size = "0.8" color = "navy">This communication is intended for the addressee only. It is confidential. If you have received this communication in error, please notify us immediately and destroy the original message. You may not copy or disseminate this communication without the permission of the University. Only authorized signatories are competent to enter into agreements on behalf of the University and recipients are thus advised that the content of this message may not be legally binding on the University and may contain the personal views and opinions of the author, which are not necessarily the views and opinions of The University of the Witwatersrand, Johannesburg. All agreements between the University and outsiders are subject to South African Law unless the University agrees in writing to the contrary.</font></p></html>

[Gabriel Dina]

Dear all,

Our error log was referring to the PermGen, so after we have increased the
Java memory up to 2 GB and PermSize/PermGen to 256MB, the Tomcat began to be
more responsive and it stopped to freeze (until now).
I'll let you know how things are going.

Thanks and regards,
Gabriel

-----Original Message-----
From: Tim Donohue [mailto:tdon...@duraspace.org]
Sent: Thursday, February 09, 2012 5:59 PM
To: Henry, Matt - OPS District Computer Technician
Cc: Alexandre Magaz Graça; Gabriel Dina; DSpac...@lists.sourceforge.net
Subject: Re: [Dspace-tech] DSpace memory issue

>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
>> icati
>> onFilterChain.java:313)
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
>> ilter
>> Chain.java:206)
>> at
>> org.dspace.app.xmlui.cocoon.SetCharacterEncodingFilter.doFilter(SetCh
>> aract

>> erEncodingFilter.java:111)
>> [...]
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>> at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)
>> at sun.nio.cs.StreamEncoder.<init>(StreamEncoder.java:194)
>> [...]
>>
>> Cheers,
>> Àlex
>>
>> ---------------------------------------------------------------------
>> -----
>> ----
>> Virtualization& Cloud Management Using Capacity Planning Cloud
>> computing makes use of virtualization - but cloud computing also
>> focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> DSpace-tech mailing list
>> DSpac...@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>>
>
>
>
> ----------------------------------------------------------------------

> -------- Virtualization& Cloud Management Using Capacity Planning