query error

[Conrad Leonard]

I have a mildly complex CONSTRUCT query that runs fine against one of our datasets in Stardog 3.1.1, but on another dataset results in the following message:

This scan does not provide an ordered output and cannot skip through solutions:  IndexScan(order=SPOC, first=http://purl.org/net/grafli/study#7eec478a-c3b0-4fd8-9c62-a050d00b57ab, second=Var(2), third=Var(3))   sort type: SortType[1]

Does that message on its own suggest anything to investigate?

Here is the query and its plan explanation:

Explaining Query:

# Query to get a study and everything downstream of it for presentation in a

# named graph.

#

# Notable things this won't return (that should stay in default graph):

# Controlled vocabulary not referenced by the core subquery

# IcgcCodeList

# Agent

CONSTRUCT

{?S ?P ?O }

WHERE { 

{

{

SELECT

DISTINCT ?S ?P ?O 

WHERE { 

# The core subquery

{

SELECT ?s ?p ?o WHERE {

{

# the study

{ ?s ?p ?o . FILTER (?s = study:7eec478a-c3b0-4fd8-9c62-a050d00b57ab) } 

# things derived from the study

UNION 

{ ?s :wasDerivedFrom* / :isClassifiedBy study:7eec478a-c3b0-4fd8-9c62-a050d00b57ab ; ?p ?o } 

# activities that generated the derived things

UNION

{ ?s (^:wasGeneratedBy) / :wasDerivedFrom+ / :isClassifiedBy study:7eec478a-c3b0-4fd8-9c62-a050d00b57ab ; ?p ?o } 

}

?s rdfs:label ?sLabel

# exclude canary data for presentation

FILTER (!regex(?sLabel, "Serinus serinus"))

}

}

# rebind the core subquery things to outer-level vars

{

?s ?p ?o

BIND(?s AS ?S)

BIND(?p AS ?P)

BIND(?o AS ?O)

}

# also grab exclusions of anything in core subquery

UNION

{

?s :isClassifiedBy ?S .

?S a :Exclusion ;

?P ?O . 

} 

}

}

}

}

The Query Plan:

Reduced [cardinality=28.4M]

  Projection(?subject, ?predicate, ?object) [cardinality=28.4M]

    Distinct [cardinality=28.4M]

      Projection(?S, ?P, ?O) [cardinality=28.4M]

        DirectHashJoin[?s] [cardinality=28.4M]

          Union [cardinality=1.2M]

            Bind((?p AS ?P) (?o AS ?O) (?s AS ?S)) [cardinality=1.2M]

              Scan[SPOC](?s, ?p, ?o) [cardinality=1.2M]

            MergeJoin[?S] [cardinality=3.4K]

              Scan[SPOC](?S, ?P, ?O) [cardinality=1.2M]

              MergeJoin[?S] [cardinality=61]

                Scan[POSC](?S, <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>, <http://purl.org/net/grafli#Exclusion>) [cardinality=303]

                Scan[POSC](?s, <http://purl.org/net/grafli#isClassifiedBy>, ?S) [cardinality=31K]

          Projection(?s, ?p, ?o) [cardinality=1.8M]

            MergeJoin[?s] [cardinality=1.8M]

              Filter(!(Regex(0, "Serinus serinus"))) [cardinality=76K]

                Scan[PSOC](?s, <http://www.w3.org/2000/01/rdf-schema#label>, ?sLabel) [cardinality=152K]

              Union [cardinality=1.1K]

                Bind((<http://purl.org/net/grafli/study#7eec478a-c3b0-4fd8-9c62-a050d00b57ab> AS ?s)) [cardinality=24]

                  Scan[SPOC](<http://purl.org/net/grafli/study#7eec478a-c3b0-4fd8-9c62-a050d00b57ab>, ?p, ?o) [cardinality=24]

                MergeJoin[?s] [cardinality=1.0K]

                  Sort(?s) [cardinality=302]

                    Union [cardinality=302]

                      MergeJoin[?s_const--1552276294-uri-0] [cardinality=273]

                        PropertyPath(?s -> ?s_const--1552276294-uri-0, minLength=0, sorted by=?s_const--1552276294-uri-0)

                          Scan[POSC](?s, <http://purl.org/net/grafli#wasDerivedFrom>, ?s_const--1552276294-uri-0) [cardinality=81K]

                        Scan[POSC](?s_const--1552276294-uri-0, <http://purl.org/net/grafli#isClassifiedBy>, <http://purl.org/net/grafli/study#7eec478a-c3b0-4fd8-9c62-a050d00b57ab>) [cardinality=18]

                      MergeJoin[?s-nested-0] [cardinality=28]

                        Sort(?s-nested-0) [cardinality=94]

                          MergeJoin[?s_const--1552276294-uri-1] [cardinality=94]

                            PropertyPath(?s-nested-0 -> ?s_const--1552276294-uri-1, minLength=1, sorted by=?s_const--1552276294-uri-1)

                              Scan[POSC](?s-nested-0, <http://purl.org/net/grafli#wasDerivedFrom>, ?s_const--1552276294-uri-1) [cardinality=81K]

                            Scan[POSC](?s_const--1552276294-uri-1, <http://purl.org/net/grafli#isClassifiedBy>, <http://purl.org/net/grafli/study#7eec478a-c3b0-4fd8-9c62-a050d00b57ab>) [cardinality=18]

                        Scan[PSOC](?s-nested-0, <http://purl.org/net/grafli#wasGeneratedBy>, ?s) [cardinality=49K]

                  Scan[SPOC](?s, ?p, ?o) [cardinality=1.2M]

I can provide the obfuscated data if you wish.

[Pavel Klinov]

Hi Conrad,

There's the issue 2388 which is already fixed and the fix will be included in the next release.

However, the data would be useful to double check that it's indeed the same problem.

Thanks,

Pavel

--
-- --
You received this message because you are subscribed to the C&P "Stardog" group.
To post to this group, send email to sta...@clarkparsia.com
To unsubscribe from this group, send email to
stardog+u...@clarkparsia.com
For more options, visit this group at
http://groups.google.com/a/clarkparsia.com/group/stardog?hl=en

[Conrad Leonard]

Hi Pavel;

Thanks for the info re #2388. The data are too large to attach in this forum; if you send me a link to sftp site or similar I can upload.

cheers, 

Conrad.