How to optimize my write operations of levelDB in my case

[程洋]

I tries to use `kythe` tools developed by google on AOSP project. it turns out there are lots of data to write into levelDB. And i encounter really big problem on performance.

Here is my use case

1. I need to write database every week.

2. each time i have 3 TB data to write, averagely 500 bytes per key-value entry

3. Unfortunately, maybe 80% of the data are duplicated.

4. after all 60TB data is written, there is only read operations after. until the next 60 TB write in the future.

5. i have machine with 512GB memory and 64 cores.

6. my disk reading benchmark is ~150MB/s,  so reading all 3TB data takes about 6 hours.  however it takes about 20 hours to finish all write. so bottle neck is on levelDB

Currently i adjust WriteBufferSize to 2GB. Though it still need 24 hours to write.

Is there any chance i can optimize write performance in my case?