SPARQL query bizarre response times

[Valentino Hudhra]

Hi, 

I've got a synthetic dataset which consists of 10M triples in following pattern:

<http://owl.cs.manchester.ac.uk/Person0> <http://xmlns.com/foaf/0.1/knows> <http://owl.cs.manchester.ac.uk/Person1000000> . 

<http://owl.cs.manchester.ac.uk/Person1> <http://xmlns.com/foaf/0.1/knows> <http://owl.cs.manchester.ac.uk/Person1000001> . 

<http://owl.cs.manchester.ac.uk/Person2> <http://xmlns.com/foaf/0.1/knows> <http://owl.cs.manchester.ac.uk/Person1000002> . 
...

and the FOAF vocabulary.  I query the database with RDFS reasoning, and here's the completion time outputs (from the command line) that I get from each run on 10 repetitions:

Query 1: SELECT (COUNT(*) as ?all) WHERE {?person rdf:type foaf:Person}

Time outputs : 

00:00:07.685

00:00:03.183

00:00:02.819

00:00:02.790

00:00:02.765

00:00:02.803

00:00:02.792

00:00:02.769

00:00:02.752

00:00:02.779

Query 2: SELECT (COUNT(DISTINCT ?person) as ?all) WHERE {?person rdf:type foaf:Person}

Time outputs:

00:00:07.055

00:00:06.071

00:00:05.935

00:00:06.000

00:00:06.287

00:00:06.007

00:00:06.103

00:00:05.971

00:00:05.878

00:00:05.887

 

I have two questions regarding these numbers, if you'd be able to assist:

1. Is this expected? Should the "distinct" in the query make such a difference (more than 2x)? 
2. Is there some sort of caching happening after the first query has run, given that there a noticeable difference between the time of completion of the first run and the consequent ones, in Query 1. Not so much in Query 2, as it seems. 

Thanks a lot,

Valentino

[Mike Grove]

Yes, of course.  In the former case you don't care what the solutions are, you're doing a simple count of the number of them.  Distinct forces you to look at every solution and only count the distinct ones which is significantly more work than simply incrementing a counter.

 

1. Is there some sort of caching happening after the first query has run, given that there a noticeable difference between the time of completion of the first run and the consequent ones, in Query 1. Not so much in Query 2, as it seems. 

Yes, there is some.  There is also JVM warm-up you may need to account for.  This is why you should do some number of warm-up runs before doing the actual benchmarking run.

Cheers,

Mike

Thanks a lot,

Valentino

--
-- --
You received this message because you are subscribed to the C&P "Stardog" group.
To post to this group, send email to sta...@clarkparsia.com
To unsubscribe from this group, send email to
stardog+u...@clarkparsia.com
For more options, visit this group at
http://groups.google.com/a/clarkparsia.com/group/stardog?hl=en