MongoDB, Toku and linkbench

[MARK CALLAGHAN]

I hope you consider using linkbench for MongoDB because MongoDB + TokuDB struggle with it compared to the other MongoDB engines. One of my tests loads with maxid1=2M which fits in the 8G of RAM on my test server (Intel NUC, core i3, 8G RAM, Samsung 850 EVO SSD). Most engines sustain 4000+ inserts/second, Mongo+Toku 3.0.9 got ~3000 inserts/second. Mongo+Toku for 3.2.4 has dropped to about ~1000/second when the load is 80% done and is currently doing ~200/second so the average rate is dropping fast.

I used the same command line for 3.0.9 and 3.2.4 but the perf problem is much worse in 3.2.4. Am I missing a config option?

Command line and mongo.conf:
https://gist.github.com/mdcallag/575d9b853e652d2e4768f040e9beb99a

PMP stack traces:
https://gist.github.com/mdcallag/1fac002d643a4ce49dc77fa0325f23e5

The log describes the binary as:
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten] MongoDB starting : pid=12930 port=27017 dbpath=/data/m/mo master=1 64-bit host=nuc3
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten] db version v3.2.4-1.0rc2
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten] git version: ff8d1c68e1b8d4d0686f21fafe85ba79d8043376
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten] allocator: tcmallocs: 0
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten] modules: noneuth Session
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten] build environment: pid=12930 port=27017 dbpath=/data/m/mo master=1 64-bit host=nuc3
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten]     distarch: x86_640rc2
2016-04-25T16:36:20.240-0700 I CONTROL  [initandlisten]     target_arch: x86_64b8d4d0686f21fafe85ba79d8043376

--

Mark Callaghan
mdca...@gmail.com

[MARK CALLAGHAN]

The same problem reproduces after ~12M inserts with my insert benchmark client. The insert rate drops from 15,000/second to ~200/second. The bottleneck is CPU and the PMP traces are similar to those shared earlier in this thread.

Client code from https://github.com/mdcallag/mytools/tree/master/bench/ibench

Client command line is:
python iibench.py --db_name=ib --rows_per_report=100000 --db_host=127.0.0.1 \

  --mongo --mongo_w=1 --max_rows=250000000 --table_name=pi1 --setup \

  --num_secondary_indexes=3 --data_length_min=10 --data_length_max=20 \

  --rows_per_commit=100 --inserts_per_second=0 --query_threads=0

mongod command line is:
/usr/bin/mongod --config mongo.conf --storageEngine PerconaFT \

--PerconaFTCollectionCompression zlib \

--PerconaFTCollectionReadPageSize 16384 \

--PerconaFTEngineCacheSize $(( 1024 * 1024 * 1024 * 1 )) \

 --PerconaFTIndexCompression zlib \

--PerconaFTIndexReadPageSize 16384 --master

mongo.conf is:
processManagement:
  fork: true
systemLog:
  destination: file
  path: /home/mdcallag/b/toku.mo/log
  logAppend: true
storage:
  syncPeriodSecs: 60
  dbPath: /data/m/mo
  journal:
    enabled: true
  mmapv1:
    journal:
      commitIntervalMs: 100

operationProfiling.slowOpThresholdMs: 2000
replication.oplogSizeMB: 2000

[Vadim Tkachenko]

Mark,

Thanks for the report. This indeed is a problem, we should look how to fix this.

> --
> You received this message because you are subscribed to the Google Groups
> "Percona Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to percona-discuss...@googlegroups.com.
> To post to this group, send email to percona-d...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Vadim Tkachenko, CTO, Percona
Phone +1-919-249-7483, Skype: vadimtk153