Configuring a Lucene indexed repository using a template

119 views
Skip to first unread message

Pete Edwards

unread,
Sep 30, 2016, 7:51:51 AM9/30/16
to RDF4J Users
I have successfully created a configuration template so I can create Lucene indexed repositories via the console (in RDF4J 1.0.1). I had a bit of a battle with this so thought I would report my findings in case they were of use to others. 

I found the documentation a bit thin on the ground so it took a few experiments and there are a few outstanding questions but I have something working. The documentation on configuration templates (http://rdf4j.org/doc/rdf4j-server-workbench-and-console/rdf4j-console/#Repository_configuration) suggests:

The URIs that are used in the templates are the URIs that are specified by the RepositoryConfig and SailConfig classes of RDF4J’s repository configuration mechanism. The relevant namespaces and URIs can be found in the javadoc of these classes.

I couldn't find what this was referring to and SailConfig doesn't even exist.

It is not yet possible to use configuration templates with workbench so you need to use the console. The default location for templates is ~/.RDF4J/console/templates/ and the only way to change this is to modify console.sh since it overwrites JAVA_OPTS. I named the template lucene.ttl.

For the actual template, I started with the one that Jacek Grzebyta posted a while ago. I worked out how to add the indexed fields;

[] a rep:Repository ;
   rep:repositoryID "{%Repository ID|lucene%}" ;
   rdfs:label "{%Repository title|Lucene indexed native store%}" ;
   rep:repositoryImpl [
      rep:repositoryType "openrdf:SailRepository" ;
      sr:sailImpl [
         sail:sailType "openrdf:LuceneSail" ;
         lucene:indexedfields """
         """ ;
         sail:delegate [
            sail:sailType "openrdf:NativeStore" ;
            ns:tripleIndexes "spoc,posc,opsc";
         ] ;
      ]
   ].


Since you may want to create stores with different indexed fields this can be parameterized:

[] a rep:Repository ;
   rep:repositoryID "{%Repository ID|lucene%}" ;
   rdfs:label "{%Repository title|Lucene indexed native store%}" ;
   rep:repositoryImpl [
      rep:repositoryType "openrdf:SailRepository" ;
      sr:sailImpl [
         sail:sailType "openrdf:LuceneSail" ;
         lucene:indexedfields """{%Indexed Fields%}""" ;
         sail:delegate [
            sail:sailType "openrdf:NativeStore" ;
            ns:tripleIndexes "spoc,posc,opsc";
         ] ;
      ]
   ].

It doesn't seem to be possible to create repositories when the console is connected to a running server so you have to connect to the actual directories instead. When you then create repositories, they are not seen by the server until you restart it. This is a bit annoying but easy enough to resolve once you are aware of it.

After connecting the console to the directories run 
create lucene

Next specify the ID and title. To enter the indexed fields put one on each line, just as in the first example, then enter a single '.' on the line to terminate the input. I discovered that feature by guess work / luck having seen how sparql queries are entered in the console. I guess that the triple quotes in the template are what trigger this behaviour for multiline input.

Additional fields can be parameterized in the template - see the field descriptions here: http://rdf4j.org/javadoc/1.0/org/eclipse/rdf4j/sail/lucene/LuceneSail.html

It would be great if the documentation gave a bit more detail on how to use this. It may be an advanced feature but it was the only way I found to create Lucene indexed repositories without writing code. I guess it is possible to inject the config file directly into the SYSTEM repository but you would still need to know how to create the file.

Pete Edwards

unread,
Sep 30, 2016, 11:42:04 AM9/30/16
to RDF4J Users
Minor update. The template needs the following line in the lucene property section:
lucene:indexDir "/whatever/you/like" ;

I removed it from the template having proved it had no effect. I believe that the property has been replaced by lucenedir but it seems something checks for the existence of indexdir so it must be present. I have raised an issue in github about this: https://github.com/eclipse/rdf4j/issues/593

Jeen Broekstra

unread,
Oct 4, 2016, 8:09:27 PM10/4/16
to rdf4j...@googlegroups.com

Hi Pete,

Thanks for providing these notes, it’s very useful. You’re quite right that the documentation on this is sorely lacking. We are aware of this and are trying to remedy it. Your notes will definitely be useful to help make that happen!

Cheers,

Jeen

-- 
You received this message because you are subscribed to the Google Groups "RDF4J Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rdf4j-users...@googlegroups.com.
To post to this group, send email to rdf4j...@googlegroups.com.
Visit this group at https://groups.google.com/group/rdf4j-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/rdf4j-users/e553d2d7-fc19-43ab-9f53-5be7d32e5339%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages