Arango restore for sharded collections, how to restore without error 1466

[Mike Atambo]

We created a db with sharded collections, as instructed for Pregel:

db._create("main", {shardKeys:['_key'],numberOfShards: 8}) 

db._create( "others", { distributeShardsLike:"main" ,numberOfShards: 8}) 

...

db._createEdgeCollection( "links", {  distributeShardsLike:"main" ,   shardKeys:['vertex'],numberOfShards: 8 })

...

But when we  export and import the data to another cluster we always get this:

2017-08-25T09:47:04Z [9839] ERROR must not specify _key for this collection: got error from server: HTTP 400 (Bad Request): ArangoError 1466: must not specify _key for this collection: must not specify _key for this collection

The code in the error  is listed here:  https://docs.arangodb.com/3.2/Manual/Appendix/ErrorCodes.html#ERROR_CLUSTER_MUST_NOT_SPECIFY_KEY,

so surely there is something we are doing wrong, and there is a right way to  dump and restore. How are we to do this?

How should we restore data from "Pregel"  ready  databases? 

Backup command we are using:

arangodump   --output-directory "landman.$(date +%Y-%m-%d:%H:%M:%S)"  --include-system-collections true 

Restore command we are using:

arangorestore  --create-collection true  --import-data true  --include-system-collections true --recycle-ids true   --input-directory  $1

[Simon G]

Ok you ran into a little bit of an unsolved issue here:
 Arangodb does not alow the import into the "others" collection if a document has the _key attribute, because you specified "vertex" as sharding attribute.
ArangoDB needs to guarantee the uniqueness of the _key attribute, but in a cluster we can only support this by generating a new _key attribute
(Otherwise imports would be much slower, because we would need to check each shard ).

I will add a new option to arangoimp to allow slectively ignoring keys, until then the only workaround is to either remove the "_key" from
the exported data files or to rename it.

[Mike Atambo]

Thanks Simon, 

Looking forward to the new option, 

but why for arangoimp, i thought it would be also for arangorestore?  or whats the difference? 

Mike. 

--
You received this message because you are subscribed to a topic in the Google Groups "ArangoDB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/arangodb/V5ozuatuIvE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to arangodb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Simon G]

Thats not quite the same tool, arangorestore works with a different codepath than arangoimp

[Mike Atambo]

On Fri, Aug 25, 2017 at 3:12 PM Simon G <simon.g...@gmail.com> wrote:

I will add a new option to arangoimp to allow slectively ignoring keys, until then the only workaround is to either remove the "_key" from
the exported data files or to rename it.

 

Simon, would you  mind expounding one this alternative for removing the _key? 

i  just tried to remove the _key from  all the collections,  and from all but the "main"

and  the import seems to work without error, but the db collections are empty. 

How to do this? 

Mike.