Some unclear behaviours of database importing and exporting

[Salvatore Piccione]

Hello,

I would like to share with you the results of some test I performed on the OrientDB import - export capabilities. I'm using OrientDB Graph Ed 1.0rc5.

IMPORTING THE WHOLE DATABASE DUMP
I notice that when importing a database, the version number increases even if the record properties are the same. You can verify this by executing this export, then this import that imports the dump previously created and finally the export again. In the enclosed zip file there are both dump files:

• exp_db1.json, the dump created with the first export
• exp_db2.json, the dump created after the import of exp_db1.json

As you can see, the version number of the records increases even if I imported exactly the same data that are already in the DB. (1) Is this the expected behaviour or the version number should be increased only if the data are actually changed?
Furthermore, in case the database is deleted before the import, the version number of all records is resetted to zero. (2) This seems to be right because I'm importing data in an empty database and I have no data relating to previous versions so it wouldn't make sense to preserve the version history of the imported records.

IMPORTING THE DATABASE RECORDS
In case I execute this export enabling only the records export (by commenting line #67 and uncommenting line #69), I get the following exception when I try to import the records dump:

Error on database import happened just before line 143, column 9
com.orientechnologies.orient.core.exception.OSecurityAccessException: Can't update internal record #0:1
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:376)
    at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:330)
    at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:83)
    at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1333)
    at com.orientechnologies.orient.client.remote.OStorageRemote.updateRecord(OStorageRemote.java:391)
    at com.orientechnologies.orient.client.remote.OStorageRemoteThread.updateRecord(OStorageRemoteThread.java:148)
    at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.save(ODatabaseRaw.java:218)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:566)
    at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:64)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:209)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:204)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:40)
    at com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.save(ODatabaseRecordWrapperAbstract.java:215)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:118)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:34)
    at com.orientechnologies.orient.core.record.ORecordAbstract.save(ORecordAbstract.java:273)
    at com.orientechnologies.orient.core.record.ORecordSchemaAwareAbstract.save(ORecordSchemaAwareAbstract.java:56)
    at com.orientechnologies.orient.core.record.ORecordSchemaAwareAbstract.save(ORecordSchemaAwareAbstract.java:32)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importRecord(ODatabaseImport.java:583)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importRecords(ODatabaseImport.java:526)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importDatabase(ODatabaseImport.java:117)
    at org.example.graph.orientdb.GraphDatabaseImporter.main(GraphDatabaseImporter.java:89)
Exception in thread "main" com.orientechnologies.orient.core.db.tool.ODatabaseExportException: Error on importing database 'tinkerpop' from file: streaming
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importDatabase(ODatabaseImport.java:133)
    at org.example.graph.orientdb.GraphDatabaseImporter.main(GraphDatabaseImporter.java:89)
Caused by: com.orientechnologies.orient.core.exception.OSecurityAccessException: Can't update internal record #0:1
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:376)
    at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:330)
    at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:83)
    at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1333)
    at com.orientechnologies.orient.client.remote.OStorageRemote.updateRecord(OStorageRemote.java:391)
    at com.orientechnologies.orient.client.remote.OStorageRemoteThread.updateRecord(OStorageRemoteThread.java:148)
    at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.save(ODatabaseRaw.java:218)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:566)
    at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:64)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:209)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:204)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:40)
    at com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.save(ODatabaseRecordWrapperAbstract.java:215)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:118)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:34)
    at com.orientechnologies.orient.core.record.ORecordAbstract.save(ORecordAbstract.java:273)
    at com.orientechnologies.orient.core.record.ORecordSchemaAwareAbstract.save(ORecordSchemaAwareAbstract.java:56)
    at com.orientechnologies.orient.core.record.ORecordSchemaAwareAbstract.save(ORecordSchemaAwareAbstract.java:32)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importRecord(ODatabaseImport.java:583)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importRecords(ODatabaseImport.java:526)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importDatabase(ODatabaseImport.java:117)
    ... 1 more

(3) I guess this can be resolved by limiting the import of the records to non restricted clusters...but which are these clusters?

[Luca Garulli]

On 13 September 2011 13:11, Salvatore Piccione <salvo...@gmail.com> wrote:

Hello,

Hi Salvatore

 

IMPORTING THE WHOLE DATABASE DUMP
I notice that when importing a database, the version number increases even if the record properties are the same. You

This is the normal behaviour because the version++ is executed inside the OStorage.update() method.

 

IMPORTING THE DATABASE RECORDS
In case I execute this export enabling only the records export (by commenting line #67 and uncommenting line #69), I get the following exception when I try to import the records dump:

This exception is from the remote server that doesn't allow the update of 0:1 record since it's internal. Probably the best would be to just jump the record 0:1 on import. I've fixed it in SVN r3830.

 

(3) I guess this can be resolved by limiting the import of the records to non restricted clusters...but which are these clusters?

Only the record 0:1 is internal because contains the schema.

Lvc@

[Luca Garulli]

About to update really changed records there are also these opened issues:

- http://code.google.com/p/orient/issues/detail?id=518

- http://code.google.com/p/orient/issues/detail?id=516

- http://code.google.com/p/orient/issues/detail?id=517

Lvc@

On 14 September 2011 12:36, Salvatore Piccione <salvo...@gmail.com> wrote:

Thanks for your replies Luca!

Below a comment to one of your replies.

IMPORTING THE WHOLE DATABASE DUMP
I notice that when importing a database, the version number increases even if the record properties are the same. You

This is the normal behaviour because the version++ is executed inside the OStorage.update() method.

 

Maybe this behaviour could be improved when ISSUE 108 will be solved so only actual changes in the record content increments the version number.

Cheers,

Salvatore

[Salvatore Piccione]

Thanks for your replies Luca!

Below a comment to one of your replies.

IMPORTING THE WHOLE DATABASE DUMP
I notice that when importing a database, the version number increases even if the record properties are the same. You

This is the normal behaviour because the version++ is executed inside the OStorage.update() method.

 

Maybe this behaviour could be improved when ISSUE 180 will be solved so only actual changes in the record content increments the version number.

Cheers,

Salvatore

[Salvatore Piccione]

Thanks agains for these useful links: I've starred all of them!

2011/9/14 Luca Garulli <l.ga...@gmail.com>