Loading quads too slow

34 views
Skip to first unread message

Daniel Hernandez

unread,
Jul 8, 2015, 12:58:41 PM7/8/15
to 4store support
Hi,

I'm loading a dataset in an nquads file that contains 170 millions of different
named graphs. However, the loading speed is getting lower during the loading:

$ 4s-import data -v -f nquads dataset.nq
removing old data
Reading <file:///home/daniel/dataset.nq>
Pass 1, processed 5000000 triples (5000000)
Pass 2, processed 5000000 triples, 315 triples/s
Pass 1, processed 5000000 triples (10000000)
Pass 2, processed 5000000 triples, 128 triples/s
Pass 1, processed 5000000 triples (15000000)
Pass 2, processed 5000000 triples, 51 triples/s
Pass 1, processed 5000000 triples (20000000)
Pass 2, processed 5000000 triples, 24 triples/s
Pass 1, processed 5000000 triples (25000000)
Pass 2, processed 1970000 triples

I notice that this does not occurs when loading a file with few named graphs.
Why? Is there a faster way to load the data?

Cheers!
Daniel

swh

unread,
Jul 14, 2015, 4:45:10 AM7/14/15
to 4store-...@googlegroups.com, dan...@degu.cl
That's really odd - have you checked /var/log/messages to see if there's any errors?

Have you tried loading a ntriples or turtle file to compare the speed?

Can't think of anything that would make it that slow.
Reply all
Reply to author
Forward
0 new messages