Re: [orientdb] OrientDB use as a triple store

[Luca Garulli]

Hi,

about (a) look at https://github.com/tinkerpop/blueprints/wiki/Sail-Implementation. About (2) the ; character was between double quotes?

About the NoClassDefFoundError problem are you using the OrientDB 1.3.0 GraphEdition?

Lvc@

On 6 March 2013 06:58, 周真 <zhoul...@gmail.com> wrote:

Hi,
The purpose using Orientdb here is to utitlize it as RDF triple store, and test the performance with Virtuoso.
The problem now is load what kind of data and how to load data into orientdb.
1. load RDB data into OrientDB, and utilize blueprints to query using sparql;
2. load n-triples data into OrientDB, and utilize blueprints to query using sparql.

For the first kind of loading, there are some problems:
(a) How to add the tag, and how to align the data with ontologies?
(b) I found that when loading RDB data into OrientDB, one row should one record, and ';' ended as the command end tag.
     But if the field data in the RDB contains ';', then the console.bat cannot understand the command, and will generate the fields not matched with values error. How to handle this kind of problem?

Is there any sites to describe the related problems?
Thanks very much.


For the second kind of loading:
 I'm trying to load nt files into OrientDB, and I tried the functions described in "https://github.com/tinkerpop/gremlin/wiki/SPARQL-vs.-Gremlin ", I found that the classes used are not supported for lib jars problem in Gremlin console.
 So I tried to load the files in Java code, and I download blueprints-graph-sail-2.2.0.jar, blueprints-sail-graph-2.2.0.jar and openrdf-sesame-2.6.10-onejar.jar to obtain the classes I needed.
The code is like below:
        PropertyConfigurator.configure("config/log4j.properties");
        OrientGraph graph = new OrientGraph("remote:localhost/tripletest",
                "admin", "admin");
        System.out.println(graph);
        Sail sail = new GraphSail<OrientGraph>(graph);
        System.out.println(sail);
       
        SailGraph sailGraph = new SailGraph(sail);
        try {
            sailGraph.loadRDF(new FileInputStream("./20130301_location.nt"),
                    "http://data.samsung.com/ontology/dining/", "n-triples",
                    null);
        } catch (FileNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } finally {
            sailGraph.shutdown();
            graph.shutdown();
        }

There is no error in the code, but when I run it. Exception:"Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class com.tinkerpop.blueprints.impls.sail.SailGraph" is occurred.
Since there is not any related answers for this kind of exception, is there anybody could figure out what the problem is?
The orientdb release: 1.3.0, and all the blueprints related jars are 2.2.0.
I'm really confused with that. Thanks for any reply or suggestions.

--
 
---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Luca Garulli]

Hi,

the best is importing the dataset using Blueprints: https://github.com/tinkerpop/blueprints/wiki (under import section).

Lvc@

On 7 March 2013 01:58, 周真 <zhoul...@gmail.com> wrote:

Hi,
I have read about the Sail Implementation page, and I think that page is based on RDF data backend.
(a) My question is that if I import data from RDB to OrientDB, and one table one class.  Meanwhile, the table structure is not aligned with Ontology.
     All my data are stored in MySQL/PostgreSQL. And the relationships defined in Ontology is not just the relations between tables, the situation is more complicated.
     Then, how to solve the problem I described in (a);
(b) The character ";" is in some fields, like reviews. And they are not in "" in the RDB data. But when I dump them from RDB using mysqldump command, they are between double quotes in the values command.
And for (a) and (b), I think (a) is more important. Even if I import all the data, how to allign and define all the relationships with ontologies is a very critical issue.

For question (2), I have solved the problem. For missing some openrdf and sesame related libraries.

Hope for your reply. Thanks very much.

[周真]

Hi Lvc,
Sorry, maybe I did't make myself clear. And saving your presence, if there are some other ways to contact you. For in my area, google is not allowed, and I cannot make a post easily.
With some instant message tools, the communication can be more convenient and clear. Thanks very much.

Meanwhile, for the problem (2), the code can run now.
The same code as below, but when I try to load some big n-triples file (>80M, maybe 10G), some errors happen as below:
2013-03-08 09:10:37,029 WARN  sail.GraphSailConnection - Rolling back transaction due to connection close
java.lang.Throwable
    at org.openrdf.sail.helpers.SailConnectionBase.close(SailConnectionBase.java:161)
    at com.tinkerpop.blueprints.impls.sail.SailGraph.loadRDF(SailGraph.java:369)
    at TripleLoadTest.main(TripleLoadTest.java:43)
Exception in thread "main" java.lang.RuntimeException: Database is closed
    at com.tinkerpop.blueprints.impls.sail.SailGraph.shutdown(SailGraph.java:414)
    at TripleLoadTest.main(TripleLoadTest.java:57)
Caused by: java.lang.IllegalStateException: Database is closed
    at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.openOrCreate(OrientBaseGraph.java:371)
    at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getContext(OrientBaseGraph.java:364)
    at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getRawGraph(OrientBaseGraph.java:347)
    at com.tinkerpop.blueprints.impls.orient.OrientGraph.stopTransaction(OrientGraph.java:74)
    at com.tinkerpop.blueprints.oupls.sail.GraphSailConnection.closeInternal(GraphSailConnection.java:78)
    at org.openrdf.sail.helpers.SailConnectionBase.close(SailConnectionBase.java:172)
    at com.tinkerpop.blueprints.impls.sail.SailGraph.closeAllConnections(SailGraph.java:402)
    at com.tinkerpop.blueprints.impls.sail.SailGraph.shutdown(SailGraph.java:411)
    ... 1 more

And if I change the OrientGraph to OrientBatchGraph, out of java heap stack happens.
What about this problem. Some configures should be set? Or some links can solve this problem.

Hope for your reply. Thanks  very much.

[Luca Garulli]

Hi,

about the first exception I've just pulled a request where before to stop the transaction assures the db is open and a transaction is running: https://github.com/tinkerpop/blueprints/pull/368

About the OrientBatchGraph problem you've to tune the settings based on your memory. Take a look from https://github.com/tinkerpop/blueprints/wiki/Batch-Implementation.

Lvc@

[Marko Rodriguez]

Hi,

I just merged this pull request into master and deployed to SonaType SNAPSHOT repo. This will come out with TinkerPop 2.3.0 planned for ~1 week.

Enjoy,

Marko.

http://markorodriguez.com

[Luca Garulli]

Thanks!

Lvc@

[Justin Harris]

Version: 1.7-rc2

plocal storage

I'm looking for a good way to insert a lot of RDF data in to OrientDB.

I tried doing something similar what was described in the original message.  I started at first with just trying to insert one line:

<http://example.com/from> <http://example.com/rel> <http://example.com/to> .

But I'm getting the error:

Exception in thread "main" java.lang.RuntimeException: com.orientechnologies.orient.core.exception.OSchemaException: Found invalid class name. Character '%' cannot be used in class name.

at com.tinkerpop.blueprints.impls.sail.SailGraph.loadRDF(SailGraph.java:349)

at com.package.loadRdf(Class.java:102)

at com.package.main(Class.java:317)

Caused by: com.orientechnologies.orient.core.exception.OSchemaException: Found invalid class name. Character '%' cannot be used in class name.

at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClassInternal(OSchemaShared.java:269)

at com.orientechnologies.orient.core.metadata.schema.OSchemaProxy.createClassInternal(OSchemaProxy.java:120)

at com.orientechnologies.orient.core.sql.OCommandExecutorSQLCreateClass.execute(OCommandExecutorSQLCreateClass.java:144)

at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:57)

at com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:96)

at com.orientechnologies.orient.core.storage.OStorageEmbedded.command(OStorageEmbedded.java:85)

at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)

at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:242)

at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:153)

at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:145)

at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:133)

at com.orientechnologies.orient.core.metadata.schema.OSchemaProxy.createClass(OSchemaProxy.java:77)

at com.tinkerpop.blueprints.impls.orient.OrientElement$1.call(OrientElement.java:292)

at com.tinkerpop.blueprints.impls.orient.OrientElement$1.call(OrientElement.java:288)

at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.executeOutsideTx(OrientBaseGraph.java:1195)

at com.tinkerpop.blueprints.impls.orient.OrientElement.checkForClassInSchema(OrientElement.java:287)

at com.tinkerpop.blueprints.impls.orient.OrientEdge.getClassName(OrientEdge.java:403)

at com.tinkerpop.blueprints.impls.orient.OrientEdge.createDocument(OrientEdge.java:389)

at com.tinkerpop.blueprints.impls.orient.OrientEdge.convertToDocument(OrientEdge.java:336)

at com.tinkerpop.blueprints.impls.orient.OrientEdge.setProperty(OrientEdge.java:185)

at com.tinkerpop.blueprints.oupls.sail.IndexingMatcher.indexStatement(IndexingMatcher.java:149)

at com.tinkerpop.blueprints.oupls.sail.GraphSailConnection.addStatementInternal(GraphSailConnection.java:272)

at com.tinkerpop.blueprints.oupls.sail.GraphSailConnection.addStatementInternal(GraphSailConnection.java:219)

at org.openrdf.sail.helpers.SailConnectionBase.addStatement(SailConnectionBase.java:471)

at com.tinkerpop.blueprints.impls.sail.SailGraph$SailAdder.handleStatement(SailGraph.java:585)

at org.openrdf.rio.ntriples.NTriplesParser.parseTriple(NTriplesParser.java:319)

at org.openrdf.rio.ntriples.NTriplesParser.parse(NTriplesParser.java:193)

at org.openrdf.rio.ntriples.NTriplesParser.parse(NTriplesParser.java:132)

at com.tinkerpop.blueprints.impls.sail.SailGraph.loadRDF(SailGraph.java:342)

        ... 2 more

I've seen this before, Orient URL encodes edge names and that causes problems when you try to set a property on an edge.  In the past, I got around this by making sure that my edge names were okay.  However, if I don't give valid URI to SailGraph.loadRDF, then it complains.

Any suggestions for how to proceed?

I would actually rather insert the RDF into OrientDB using my own RDF parser because I want to do certain pre-processing but it was really slow (after inserting 3 million relationships, it's at about 10 relations/second).  It's probably slow because I created an index on the vertex label property, as to make sure that it didn't insert duplicate vertices.  FYI the index type is UNIQUE and I'm using CASE_INSENSITIVE_COLLATE.  I use that index to get an existing vertex before making a new one.

Any suggestions on the best way to get RDF into OrientDB?

BTW for anyone else looking to do this too, I had to add the two maven dependencies:

   

 <dependency>
      <groupId>com.tinkerpop.blueprints</groupId>
      <artifactId>blueprints-sail-graph</artifactId>
      <version>2.5.0</version>
      <optional>true</optional>
    </dependency>
    <dependency>
      <groupId>com.tinkerpop.blueprints</groupId>
      <artifactId>blueprints-graph-sail</artifactId>
      <version>2.5.0</version>
      <optional>true</optional>
    </dependency>

[Justin Harris]

I think I found some things to help me out.  First, do:

...
orientGraph.setUseClassForEdgeLabel(false);
new GraphSail<>(orientGraph);
...

but now my edges get inserted but the labels are ugly URL encodings and the in's and out's don't show in the UI when I do:

SELECT FROM V;

So my next plan is to mess around with the Sail's parser and/or valueFactory.  It looks like the Rio.createParser methods change some global parsers.  I'll try it tomorrow, does anyone have experience with these things?

Thanks.

[Justin Harris]

So I made some changes to allow loadRDF to take in a custom ValueFactory so that I can have shorter strings, for example:

"http://example.com/rel" -> "ex:rel"

but now I still get similar errors now when adding edges.  It works if the label has a period, so now I use: "ex:.rel".

Why does OrientDB URL encode edge names if it's going to complain about them later?  Is there a bug report for this?

[Max]

Hi!

I was trying to use SailLoader class to load a large amount of Data into OrientDb that is a ,nt file. I got similar errors now as you did,

 I am using tinkeprop graph sail version 2.5.0.

here are the error messages:

WARNING: Requested command 'Committing the active transaction to create the new type 'http%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23type' as subclass of 'E'. The transaction will be reopen right after that. To avoid this behavior create the classes outside the transaction' must be executed outside active transaction: the transaction will be committed and reopen right after it. To avoid this behavior execute it outside a transaction

com.orientechnologies.orient.core.exception.OSchemaException: Found invalid class name. Character '%' cannot be used in class name.

at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClassInternal(OSchemaShared.java:340)

at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:314)

at com.orientechnologies.orient.core.metadata.schema.OSchemaProxy.createClass(OSchemaProxy.java:97)

at com.orientechnologies.orient.core.sql.OCommandExecutorSQLCreateClass.execute(OCommandExecutorSQLCreateClass.java:141)

at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:60)

at com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:94)

at com.orientechnologies.orient.core.storage.OStorageEmbedded.command(OStorageEmbedded.java:83)

at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)

at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1181)

at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:340)

at com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169)

at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45)

Sep 15, 2014 12:08:04 AM com.tinkerpop.blueprints.oupls.sail.SailLoader load

INFO: loaded 0 statements in 66ms

Exception in thread "main" java.lang.AssertionError: expected:<29> but was:<0>

at org.junit.Assert.fail(Assert.java:74)

at org.junit.Assert.failNotEquals(Assert.java:448)

at org.junit.Assert.assertEquals(Assert.java:102)

at org.junit.Assert.assertEquals(Assert.java:323)

at org.junit.Assert.assertEquals(Assert.java:319)

at com.master.thesis.LoadData.testAll(LoadData.java:42)

at com.master.thesis.first.main(first.java:16)

Can you please share what you did to overcome this problem finally? It would be such a great help. I have been stuck on this since yesterday and I completely stumped!

[Luca Garulli]

What's the problem? Can you report the exception and they way you imported data?

Lvc@

On 15 September 2014 01:04, Max <sykam...@gmail.com> wrote:

Hi!

I was trying to use SailLoader class to load a large amount of Data into OrientDb. I got the exact same errors now as you did!

 I am using tinkeprop graph sail version 2.5.0.

--

---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Max]

Hi 

Sorrry for the late response. I got it to work. There were some syntax problems and I also had to set: 

alter database custom useClassForEdgeLabel=false

so thats edge labels from orientDb are not used! From then on it worked.

Thanks!!

[Luca Garulli]

Hi Max,

Glad to hear it. I'm sure further users will find it useful.

Lvc@