Out Of Memory Issue while Batch creation of PURLs

[Landong Zuo]

Hi,


I was doing the scalability test using PURLZV1.6.2 for my local Purl
server. The Batch creation of 100,000 Purls was executed via POST
submission of XML file and single Http post to RESTful API. However,
the ourOfMem exception was encountered in both test, the details are
attached as following text. The job failed at 42578 creation. Could
anyone suggest how this problem can be resolved? Thanks.

Regards,

Landong Zuo





Running job [Index the PURL Data]
WARNING Exception whilst running job in
urn:org:ten60:netkernel:util:cron
WARNING <ex>
<id>Exception during request processing</id>
<message>whilst waiting for result...</message>
<requestid>[SOURCE active:module+resource@active%3Apurl-entries-
reindex+module@urn%3Aorg%3Apurl%
3Aadmin+version@version%3A0.0.1799 in
urn:org:ten60:netkernel:util:cron as com.ten60.netkernel.urii.
IURAspect]</requestid>
<stack>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAsyncRequestHandleImpl.handleException()
line:
161</level>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAsyncRequestHandleImpl.issueCallback()
line:13
4</level>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAsyncRequestHandleImpl.innerReceive()
line:123
</level>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAsyncRequestHandleImpl.receiveAsyncException()
line:107</level>
</stack>
<ex>
<id>Exception during request processing</id>
<message>whilst waiting for result...</message>
<requestid>[SOURCE active:purl-entries-reindex in
urn:org:purl:admin as com.ten60.netkernel.
urii.IURAspect]</requestid>
<stack>

<level>com.ten60.netkernel.scheduler.RequestState.setException() line:
132</level>

<level>com.ten60.netkernel.scheduler.Scheduler.receiveAsyncException()
line:346</level>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAccessorImpl.requestAsync()
line:118</leve
l>

<level>com.ten60.netkernel.scheduler.Scheduler.stateRequestRepresentation()
line:439</le
vel>
</stack>
<ex>
<id>Unhandled Exception in Accessor</id>
<message>org.purl.accessor.IndexAccessor</message>
<requestid>[SOURCE active:purl-search-index
+importType@delta-import in urn:org:purl:sear
ch as com.ten60.netkernel.urii.IURAspect]</requestid>
<stack>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAccessorImpl.requestAsync()
line:115</
level>

<level>com.ten60.netkernel.scheduler.Scheduler.stateRequestRepresentation()
line:439
</level>

<level>com.ten60.netkernel.scheduler.Scheduler.processRequest() line:
253</level>

<level>com.ten60.netkernel.scheduler.Scheduler.innerRequestAsynch()
line:148</level>

</stack>
<ex>
<id>Exception during request processing</id>
<message>whilst waiting for result...</message>
<requestid>[SOURCE active:purl-storage-db-timestamp in
urn:org:purl:search as com.te
n60.netkernel.urii.aspect.StringAspect]</requestid>
<stack>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFHelperImpl.issueSubRequest()
line:
273</level>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFHelperImpl.sourceAspect()
line:362
</level>

<level>org.purl.accessor.IndexAccessor.updateTimestamp() line:109</
level>
<level>org.purl.accessor.IndexAccessor.index()
line:45</level>
</stack>
<ex>
<id>Unhandled Exception in Accessor</id>
<message>org.ten60.docxter2.XRLMapper</message>
<requestid>[SOURCE active:mapper+operator@ffcpl:/
sql/db/links.xml+operand@ffcpl:
/xrl/db-timestamp in urn:org:ten60:netkernel:ext:xrl as
com.ten60.netkernel.urii.aspect.StringAspect
]</requestid>
<stack>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAccessorImpl.requestAsync()
li
ne:115</level>

<level>com.ten60.netkernel.scheduler.Scheduler.stateRequestRepresentation()
line:439</level>

<level>com.ten60.netkernel.scheduler.Scheduler.processRequest() line:
253</le
vel>

<level>com.ten60.netkernel.scheduler.Scheduler.innerRequestAsynch()
line:148
</level>
</stack>
<ex>
<id>Exception during request processing</id>
<message>whilst waiting for result...</
message>
<requestid>[SOURCE active:groovy+operand@ffcpl
%3A/xrl/db-timestamp+operator@
ffcpl%3A/sql/db/db-timestamp.groovy+operator@ffcpl%3A/sql/db/links.xml
in urn:org:ten60:netkernel:ex
t:xrl as com.ten60.netkernel.urii.aspect.StringAspect]</requestid>
<stack>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFHelperImpl.issueSubRequest
() line:273</level>

<level>org.ten60.docxter2.GTPUtils.resolveLink() line:115</level>

<level>org.ten60.docxter2.XRLMapper.processRequest() line:85</level>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAccessorImpl.requestAsync(
) line:107</level>
</stack>
<ex>
<id>Unhandled Exception in Accessor</id>

<message>org.ten60.netkernel.script.accessor.ScriptEngineNKFAccessor</
me
ssage>
<requestid>[SOURCE active:groovy
+operand@ffcpl%3A/xrl/db-timestamp+opera
tor@ffcpl%3A/sql/db/db-timestamp.groovy+operator@ffcpl%3A/sql/db/
links.xml in urn:org:ten60:netkerne
l:ext:script as com.ten60.netkernel.urii.aspect.StringAspect]</
requestid>
<stack>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAccessorImpl.requestAs
ync() line:115</level>

<level>com.ten60.netkernel.scheduler.Scheduler.stateRequestRepresent
ation() line:439</level>

<level>com.ten60.netkernel.scheduler.Scheduler.processRequest() line
:253</level>

<level>com.ten60.netkernel.scheduler.Scheduler.innerRequestAsynch()
line:148</level>
</stack>
<ex>

<id>com.ten60.netkernel.util.NetKernelException</id>
<message>Unhandled Error in Groovy</
message>
<stack>

<level>org.ten60.netkernel.script.engine.GroovyEngine.execute()
line:78</level>

<level>org.ten60.netkernel.script.accessor.ScriptEngineNKFAccess
or.processRequest() line:82</level>

<level>org.ten60.netkernel.layer1.nkf.impl.NKFAccessorImpl.reque
stAsync() line:107</level>

<level>com.ten60.netkernel.scheduler.Scheduler.stateRequestRepre
sentation() line:439</level>
</stack>
<ex>
<id>java.lang.OutOfMemoryError</
id>
<message>Java heap space</message>
<stack/>
</ex>
</ex>
</ex>
</ex>
</ex>
</ex>
</ex>
</ex>
</ex>
in urn:org:ten60:netkernel:util:cron

[David Wood]

Hi Landong,

How much memory do you have available to the process? Perhaps you are just running out of memory because you don't have enough.

Also, we recommend using the latest PURL release v1.6.3.

Regards,
Dave