arangorestore (or arangoimport) too slow

47 views
Skip to first unread message

Alexander Lanin

unread,
Sep 29, 2021, 12:46:39 PM9/29/21
to ArangoDB

Hi!
Arangodb 3.7.11 Community edition
Collection size:
 - 132,132,714 records
 - 8.7Gb like .json.gz (after arngodump)
Host: 16CPU/32RAM/SATA_RAID0
```
>hdparm -tT /dev/md0p1
/dev/md0p1:
 Timing cached reads:   14780 MB in  1.99 seconds = 7417.56 MB/sec
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 Timing buffered disk reads: 2946 MB in  3.00 seconds = 981.66 MB/sec
```
This command took too long:
```
arangorestore \
--batch-size 40000000 \
--force-same-database true \
--input-directory /big \
--threads 3 \
--server.authentication false \
--server.database $DBNAME \
--server.endpoint http+tcp://arango:8529 \
--collection $COLNAME \
--overwrite false
```
`INFO [a66e1] {restore} Processed 1 collection(s) in 78709.097923 s, read 109679969672 byte(s) from datafiles, sent 2742 data batch(es) of 109679969671 byte(s) total size`

Is it normal?
I want to be able to restore my database from the backup (arangodump or arangoexport).
But 21 hour for the one collection. It is impossible to use.

One of the possible problems is the 21 index for this collection.

Do any "special" parameters or tricks to backup and restore such database faster?
Reply all
Reply to author
Forward
0 new messages