[Dspace-tech] DSpace concurrent user access

Jayan Chirayath Kurian

unread,

Aug 24, 2015, 5:14:47 PM8/24/15

to dspac...@lists.sourceforge.net

Hi!

Using Compuware QAload testing tool, our dspace instance (1.4.1 on windows 2003) was tested on our test server ( 1GB ram, 300 GB HD, with around 100,000 items)

Tests were performed to see the support for concurrent access to the dspace instance. It appears like like for 100 current users, 21 users failed. Is there any way to support concurrent access and

what could be the hardware requirements for the same.

thanks,

Jayan

Test 1 (All pass (100 users): combination of submit, first page and search)
------
Submit: 10 users
First page : 50 users 
Search : 40 users

Test 2 (50 users total - 26 fail, 24 pass) - CPU 100%
------
Submit: 50 users
Error Uploading File - Internal System Error


For Test 3, 4, 5 steps followed by concurrent users are 
----------------
accessing http://dspace.ntu.edu.sg/
sleep 180s
accessing Communities\r\n& Collections
sleep 240s
accessing "Fifth Collection"
sleep 240s
accessing "Click on Titles for search"
sleep 240s
click on doc "Access to Information and Participant in"
sleep 60s
click on doc "Open/View"

Test 3 (30 users all pass)
-----
Whole search process
(Timing in attached file DSpace-test1_08202007_121051.bmp)


Test 4 (60 users : 4 errors)
------
whole search process
- Internal System Error" was received
- Link option LINK, value "Fifth Collection" not found. The anchor tag was not found in the HTML document


Test 5 (100 users: 21 errors)
------
whole search process
- Internal System Error" was received
- Link option LINK, value "Fifth Collection" not found. The anchor tag was not found in the HTML document

Tellier, Stephane

unread,

Aug 24, 2015, 5:14:49 PM8/24/15

to Jayan Chirayath Kurian, dspac...@lists.sourceforge.net

Hi,

can you be more specific about the kind of errors you've got from your different tests (what was the output in the log files?)

Here are some suggestions of mine :

1) put more ram on your server and boost the max memory used by tomcat with the -Xmx parameter. The more concurrent user you want to support, the more ram you'll need. This is essential for http sessions, although I have no idea for now if DSpace sessions are using lot of memory or not.

2) raise your bandwidth, this is very important for upload/download features.

Hope it helps, maybe others have some suggestions to.

De: dspace-te...@lists.sourceforge.net de la part de Jayan Chirayath Kurian
Date: lun. 2007-08-20 10:00
À: dspac...@lists.sourceforge.net
Objet : [Dspace-tech] DSpace concurrent user access

Christian Voelker

unread,

Aug 24, 2015, 5:14:50 PM8/24/15

to Jayan Chirayath Kurian, dspac...@lists.sourceforge.net

Hello,

I think your system performs pretty well for up to 40
concurrent users (some less with uploads). I would
check the options that you start your JVM with. Did
you set the -Xmx to let say -Xmx256m? You might gain
more performance from that. I would also run some
tools on the server such as dstat to find out whether
the memory is used or when swapping starts or whether
CPU cycles are scarce.

Bye, Christian

Christian Voelker

unread,

Aug 24, 2015, 5:14:51 PM8/24/15

to Jayan Chirayath Kurian, dspace-tech Tech

Am 20.08.2007 um 16:39 schrieb Jayan Chirayath Kurian:

> In Dspace - bin - dsrun where java was called, i have made -Xmx to
> Xmx1000m.

Phew. That should suffice.

> Is there any other place where i can set Xmx for java. I am using
> DSpace 1.4.1 on windows.
> Earlier i had error coming out with indexing and was solved by
> giving Xmx1000m to dsrun program.

I dont know whether there are other places to tweak Java options
under Windows, but if this works for you I would stick with it.

> please suggest. Which parameter can be increased to give more
> memory to tomcat and postgresql because from the task manager -
> monitor, postgres was consuming lot of memory.

I would give more memory to either tomcat *or* postgres to optimize.
In your case it sounds as if postgres needs more memory, not tomcat.

I might have to do with the queries you are using if these are not
canned queries coded in a link in the UI. You can use explain analyze
to find out whether the database makes use of indices as expected.
But before adding a new index I would discuss the implications of
doing so here on the list. I seems to be a rather drastic change.

I think 1Gig RAM is rather small for a decent server and more RAM
might give you the most bang for the buck. On PCs it is dependent
of the chipset whether more RAM makes the whole machine faster or
*slower*, but even then, the app might work faster on a machine
that is slower in terms of benchmarks, so I would throw in whatever
it can hold. RAM is too cheap to take time for optimization.

Bye, Christian

Jayan Chirayath Kurian

unread,

Aug 24, 2015, 5:14:52 PM8/24/15

to Tellier, Stephane, dspac...@lists.sourceforge.net

Hi!

Concurrent user tests were conducted on dspace instance and error messages from Catalina.log and localhost.log are given below. The test details performed and results are towards the end of this mail. Please suggest.

(1) The Catalina.log shows the following

Aug 20, 2007 12:28:16 PM org.apache.coyote.http11.Http11Protocol pause

INFO: Pausing Coyote HTTP/1.1 on http-80

Aug 20, 2007 12:28:17 PM org.apache.catalina.core.StandardService stop

INFO: Stopping service Catalina

Aug 20, 2007 12:28:17 PM org.apache.catalina.core.StandardWrapper unload

INFO: Waiting for 1 instance(s) to be deallocated

Aug 20, 2007 12:28:18 PM org.apache.catalina.core.StandardWrapper unload

Aug 20, 2007 12:28:22 PM org.apache.catalina.core.StandardWrapper unload

INFO: Waiting for 12 instance(s) to be deallocated

Aug 20, 2007 12:28:28 PM org.apache.coyote.http11.Http11Protocol destroy

INFO: Stopping Coyote HTTP/1.1 on http-80

Aug 20, 2007 12:28:36 PM org.apache.catalina.core.AprLifecycleListener init

INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: E:\Tomcat 6.0\bin;.;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;E:\Java\jdk1.6.0\bin;E:\apache-ant-1.7.0\bin;

Aug 20, 2007 12:28:36 PM org.apache.coyote.http11.Http11Protocol init

INFO: Initializing Coyote HTTP/1.1 on http-80

Aug 20, 2007 12:28:36 PM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 594 ms

Aug 20, 2007 12:28:37 PM org.apache.catalina.core.StandardService start

INFO: Starting service Catalina

Aug 20, 2007 12:28:37 PM org.apache.catalina.core.StandardEngine start

INFO: Starting Servlet Engine: Apache Tomcat/6.0.7

Aug 20, 2007 12:28:37 PM org.apache.catalina.core.StandardHost start

Aug 20, 2007 12:28:39 PM org.apache.coyote.http11.Http11Protocol start

INFO: Starting Coyote HTTP/1.1 on http-80

Aug 20, 2007 12:28:39 PM org.apache.jk.common.ChannelSocket init

INFO: JK: ajp13 listening on /0.0.0.0:8009

Aug 20, 2007 12:28:39 PM org.apache.jk.server.JkMain start

INFO: Jk running ID=0 time=0/31 config=null

Aug 20, 2007 12:28:39 PM org.apache.catalina.startup.Catalina start

INFO: Server startup in 2891 ms

(2) localhost.log shows the following

Aug 20, 2007 11:36:37 AM org.apache.catalina.core.ApplicationDispatcher invoke

SEVERE: Servlet.service() for servlet jsp threw exception

java.io.IOException: Stream closed

at org.apache.jasper.runtime.JspWriterImpl.ensureOpen(JspWriterImpl.java:204)

at org.apache.jasper.runtime.JspWriterImpl.write(JspWriterImpl.java:312)

at org.apache.jasper.runtime.JspWriterImpl.write(JspWriterImpl.java:342)

at org.apache.jasper.runtime.JspWriterImpl.newLine(JspWriterImpl.java:358)

at org.apache.jasper.runtime.JspWriterImpl.println(JspWriterImpl.java:497)

at org.apache.jasper.runtime.JspWriterImpl.println(JspWriterImpl.java:577)

at org.apache.jsp.community_002dlist_jsp.showCommunity(community_002dlist_jsp.java:43)

at org.apache.jsp.community_002dlist_jsp._jspService(community_002dlist_jsp.java:260)

at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:390)

at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:320)

at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:683)

at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:469)

at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:404)

at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:302)

at org.dspace.app.webui.util.JSPManager.showJSP(JSPManager.java:91)

at org.dspace.app.webui.servlet.CommunityListServlet.doDSGet(CommunityListServlet.java:108)

at org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151)

at org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:228)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:212)

at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)

at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:634)

at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:445)

at java.lang.Thread.run(Unknown Source)

Aug 20, 2007 11:36:43 AM org.apache.catalina.core.StandardHostValve custom

SEVERE: Exception Processing ErrorPage[exceptionType=java.lang.Exception, location=/internal-error]

java.lang.IllegalStateException

at org.apache.coyote.Response.reset(Response.java:297)

at org.apache.catalina.connector.Response.reset(Response.java:652)

at org.apache.catalina.connector.Response.reset(Response.java:916)

at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:417)

at org.apache.catalina.core.StandardHostValve.throwable(StandardHostValve.java:271)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:212)

at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)

at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:634)

at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:445)

at java.lang.Thread.run(Unknown Source)

Aug 20, 2007 11:57:50 AM org.apache.catalina.core.StandardWrapperValve invoke

SEVERE: Servlet.service() for servlet community-list threw exception

java.io.IOException: Stream closed

at org.apache.jasper.runtime.JspWriterImpl.ensureOpen(JspWriterImpl.java:204)

at org.apache.jasper.runtime.JspWriterImpl.write(JspWriterImpl.java:312)

at org.apache.jasper.runtime.JspWriterImpl.write(JspWriterImpl.java:342)

at org.apache.jasper.runtime.JspWriterImpl.print(JspWriterImpl.java:468)

at org.apache.jasper.runtime.JspWriterImpl.println(JspWriterImpl.java:576)