"exception in streamer" while doing "stardog data add"

Jacobus Geluk

unread,

Nov 13, 2015, 5:07:03 PM11/13/15

to Stardog

Executing command for loading a DbPedia file on one of the nodes in the Stardog cluster (running as root):

stardog data add snarl://localhost:5821/slap instance-types_en.ttl.bz2

This is what shows up in stardog.log:

ERROR 2015-11-13 10:43:35,055 [StardogServer.WorkerGroup-4] com.complexible.stardog.pack.replication.proxy.Streamer:exceptionCaught(119): exception in streamer

java.io.IOException: Connection reset by peer

at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:1.8.0_51]

at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:1.8.0_51]

at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[?:1.8.0_51]

at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[?:1.8.0_51]

at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[?:1.8.0_51]

at io.netty.buffer.UnpooledUnsafeDirectByteBuf.setBytes(UnpooledUnsafeDirectByteBuf.java:447) ~[netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881) ~[netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:242) ~[netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_51]

Jacobus Geluk

unread,

Nov 13, 2015, 5:10:48 PM11/13/15

to Stardog

After restarting that node, the error message at the command line changes from:

Adding data from file: instance-types_en.ttl.bz2

An error occurred adding RDF to the index: /longdirectoryname/instance-types_en.ttl.bz2

into:

There was a fatal failure during preparation of af9002e2-1f2e-43f1-9dde-3293cce1dc26 org.openrdf.rio.RDFParseException: 65534

Michael Grove

unread,

Nov 16, 2015, 6:40:59 AM11/16/15

to stardog

On Fri, Nov 13, 2015 at 5:10 PM, Jacobus Geluk <jacobu...@gmail.com> wrote:

After restarting that node, the error message at the command line changes from:

Adding data from file: instance-types_en.ttl.bz2

An error occurred adding RDF to the index: /longdirectoryname/instance-types_en.ttl.bz2

into:

There was a fatal failure during preparation of af9002e2-1f2e-43f1-9dde-3293cce1dc26 org.openrdf.rio.RDFParseException: 65534

It's probably the same error in both cases. The DBpedia data dumps are notoriously poor RDF. You have to disabled `strict.parsing` for the database [1] you're loading it into or fix the errors in the original data.

Cheers,

Mike

[1] http://docs.stardog.com/#_configuration_options

On Friday, November 13, 2015 at 5:07:03 PM UTC-5, Jacobus Geluk wrote:

Executing command for loading a DbPedia file on one of the nodes in the Stardog cluster (running as root):

stardog data add snarl://localhost:5821/slap instance-types_en.ttl.bz2

This is what shows up in stardog.log:

ERROR 2015-11-13 10:43:35,055 [StardogServer.WorkerGroup-4] com.complexible.stardog.pack.replication.proxy.Streamer:exceptionCaught(119): exception in streamer

java.io.IOException: Connection reset by peer

at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:1.8.0_51]

at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:1.8.0_51]

at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[?:1.8.0_51]

at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[?:1.8.0_51]

at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[?:1.8.0_51]

at io.netty.buffer.UnpooledUnsafeDirectByteBuf.setBytes(UnpooledUnsafeDirectByteBuf.java:447) ~[netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881) ~[netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:242) ~[netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) [netty-all-4.0.30.Final.jar:4.0.30.Final]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_51]

--
-- --
You received this message because you are subscribed to the C&P "Stardog" group.
To post to this group, send email to sta...@clarkparsia.com
To unsubscribe from this group, send email to
stardog+u...@clarkparsia.com
For more options, visit this group at
http://groups.google.com/a/clarkparsia.com/group/stardog?hl=en

Evren Sirin

unread,

Nov 16, 2015, 1:52:50 PM11/16/15

to Stardog

Note that in the cluster you need to explicitly disable transaction
logging in order to disable strict parsing. So you'd need to create
the database (or change the options later) as follows to load data
with invalid triples:

bin/stardog-admin db create -o transaction.logging=false
strict.parsing=false -n myDB

Best,
Evren

Jacobus Geluk

unread,

Nov 16, 2015, 4:04:17 PM11/16/15

to Stardog

That can be set per database right? So would it be a good approach to load all DbPedia data first in a separate database (in same cluster), with strict parsing off, and then do a federated query to load all content from that database into the actual target database?

Michael Grove

unread,

Nov 17, 2015, 9:13:06 AM11/17/15

to stardog

On Mon, Nov 16, 2015 at 4:04 PM, Jacobus Geluk <jacobu...@gmail.com> wrote:

That can be set per database right?

Correct.

So would it be a good approach to load all DbPedia data first in a separate database (in same cluster), with strict parsing off, and then do a federated query to load all content from that database into the actual target database?

That might work, but it's much better to fix the data in the first place. The queries will be significantly more efficient when run against a single database.