Stardog 4.0 RC1 - Memory issue after upgrading to 4.0

mohammad....@gmail.com

unread,

Oct 10, 2015, 8:17:24 AM10/10/15

to Stardog

Hi

I get out of memory error quite often after upgrading to 4.0 while using the same ontology with same prop file for stardog:

ERROR 2015-10-10 11:57:17,454 [SPEC-Server-1-29] com.complexible.common.protocols.server.rpc.ServerHandler:exceptionCaught(367): exceptionCaughtServerHandler
java.lang.OutOfMemoryError
	at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:123) ~[?:1.8.0_60]
	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:117) ~[?:1.8.0_60]
	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) ~[?:1.8.0_60]
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) ~[?:1.8.0_60]
	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) ~[?:1.8.0_60]
	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282) ~[?:1.8.0_60]
	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_60]
	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207) ~[?:1.8.0_60]
	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._flushBuffer(WriterBasedJsonGenerator.java:1879) ~[jackson-core-2.4.4.jar:2.4.4]
	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._writeString(WriterBasedJsonGenerator.java:916) ~[jackson-core-2.4.4.jar:2.4.4]
	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator.writeString(WriterBasedJsonGenerator.java:331) ~[jackson-core-2.4.4.jar:2.4.4]
	at com.fasterxml.jackson.core.JsonGenerator._writeSimpleObject(JsonGenerator.java:1516) ~[jackson-core-2.4.4.jar:2.4.4]
	at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:283) ~[jackson-core-2.4.4.jar:2.4.4]
	at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1255) ~[jackson-core-2.4.4.jar:2.4.4]
	at org.openrdf.query.resultio.sparqljson.AbstractSPARQLJSONWriter.writeValue(AbstractSPARQLJSONWriter.java:355) ~[sesame-queryresultio-sparqljson-4.0.0-M1.jar:?]
	at org.openrdf.query.resultio.sparqljson.AbstractSPARQLJSONWriter.handleSolution(AbstractSPARQLJSONWriter.java:181) ~[sesame-queryresultio-sparqljson-4.0.0-M1.jar:?]
	at org.openrdf.query.resultio.sparqljson.SPARQLResultsJSONWriter.handleSolution(SPARQLResultsJSONWriter.java:29) ~[sesame-queryresultio-sparqljson-4.0.0-M1.jar:?]
	at org.openrdf.query.QueryResults.report(QueryResults.java:160) ~[sesame-query-4.0.0-M1.jar:?]
	at org.openrdf.query.resultio.QueryResultIO.write(QueryResultIO.java:634) ~[sesame-queryresultio-api-4.0.0-M1.jar:?]
	at com.complexible.annex.repository.AbstractRepositoryHandler.queryTuples(AbstractRepositoryHandler.java:232) ~[stardog-webconsole-annex-4.0-RC1.jar:?]
	at com.complexible.annex.api.SchemaHandler$RepositoryHandlerStrategy.queryTuples(SchemaHandler.java:480) ~[stardog-webconsole-annex-4.0-RC1.jar:?]
	at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:171) ~[stardog-webconsole-annex-4.0-RC1.jar:?]
	at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:82) ~[stardog-webconsole-annex-4.0-RC1.jar:?]
	at com.complexible.stardog.protocols.http.annex.SchemaService.handleMessage(SchemaService.java:61) ~[stardog-webconsole-annex-4.0-RC1.jar:?]
	at com.complexible.common.protocols.server.rpc.ServerHandler.handleMessage(ServerHandler.java:295) ~[stardog-protocols-api-server-4.0-RC1.jar:?]
	at com.complexible.common.protocols.server.rpc.ServerHandler.channelRead(ServerHandler.java:187) ~[stardog-protocols-api-server-4.0-RC1.jar:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:32) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:299) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:36) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]

Also getting this warning when ever load the ontology:

.
WARN  2015-10-10 11:47:11,217 [StardogServer.WorkerGroup-10] io.netty.channel.DefaultChannelPipeline:warn(151): An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.io.IOException: java.nio.channels.ClosedChannelException
	at com.complexible.stardog.protocols.http.server.HttpMessageEncoder$SendResponseChannelOutputStream.assertNoError(HttpMessageEncoder.java:445) ~[stardog-protocols-http-server-4.0-RC1.jar:?]
	at com.complexible.stardog.protocols.http.server.HttpMessageEncoder$SendResponseChannelOutputStream.close(HttpMessageEncoder.java:363) ~[stardog-protocols-http-server-4.0-RC1.jar:?]
	at com.complexible.stardog.protocols.http.server.HttpMessageEncoder.write(HttpMessageEncoder.java:172) [stardog-protocols-http-server-4.0-RC1.jar:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:633) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:32) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:908) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:960) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:893) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:356) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) [netty-all-4.0.30.Final.jar:4.0.30.Final]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
Caused by: java.nio.channels.ClosedChannelException

Kendall Clark

unread,

Oct 10, 2015, 8:28:36 AM10/10/15

to sta...@clarkparsia.com

Can you share the ontology or any details about yr environment?

--
-- --
You received this message because you are subscribed to the C&P "Stardog" group.
To post to this group, send email to sta...@clarkparsia.com
To unsubscribe from this group, send email to
stardog+u...@clarkparsia.com
For more options, visit this group at
http://groups.google.com/a/clarkparsia.com/group/stardog?hl=en

mohammad....@gmail.com

unread,

Oct 10, 2015, 8:38:47 AM10/10/15

to Stardog

Unfortunately I cannot share the ontology but here are some info about it :

Triples

Axioms

LogicalAxioms

8,744,149

3,230,381

2,557,295

I'm running stardog inside a docker with this args : STARDOG_JAVA_ARGS="-Xmx30g -Xms15g -XX:MaxDirectMemorySize=60g"

The docker host is a debian machine on gcloud n1-highmem-32 (32 vCPUs, 208 GB memory)

We are using java8 update 60

stardog+unsubscribe@clarkparsia.com

Reply all

Reply to author

Forward