[HLFS] I/O Performance
6 views
Skip to first unread message
harryxiyou
May 13, 2013, 4:37:07 AM5/13/13
to clo...@googlegroups.com, Kang Hua, Chen Lijun
Hi all,
最近使用NBD时发现有bug，具体如下：
I use nbd-2.9.15 and Linux kernel version is 3.5.0. The errors are
like following.
nbd-server:
** Message: connect from 127.0.0.1, assigned file is (null)
** Message: Authorized client
** Message: Starting to serve
Error: Failed/5:Resource temporarily unavailable
Exiting.
** Message: Child exited with 1
dmesg outputs:
[245826.049068] block nbd1: Receive control failed (result -32)
[245826.049129] block nbd1: shutting down socket
[245826.049179] block nbd1: queue cleared
HLFS report error messages like following.
[...]
17 20130512 14:16:43.414 DEBUG hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/utils/storage_helper.c][init
_storage_handler][28]enter func init_storage_handler
18 20130512 14:16:43.414 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/utils/storage_helper.c][init
_storage_handler][39]parse_from_uri happened error
19 20130512 14:16:43.415 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/storage/init_hlfs.c][__init_
hlfs][114][uri:testfs] can not accessable
20 20130512 14:16:43.415 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/storage/init_hlfs.c][init_hl
fs][227]init raw hlfs ctrl failed
21 20130512 14:16:43.415 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/nbd-2.9.15/nbd-server.c][negotia
te][1280]Failed to init storage control block
目前以上bug已经修复，随后把修复后的nbd放到我的分支上，也许还存在漏洞，但是
我已经可以正常使用。刚才在standalone方式下测了下HLFS的性能具体如下：
local模式
==========
$ sudo fio -filename=/dev/nbd4 -direct=1 -rw=randwrite -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-write
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[14.3% done] [0K/3142K /s] [0 /2 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[19.0% done] [0K/6285K /s] [0 /5 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[23.8% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[28.6% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[33.3% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[38.1% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[42.9% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[47.6% done] [0K/7332K /s] [0 /6 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[52.4% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[57.1% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[61.9% done] [0K/4190K /s] [0 /3 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[66.7% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[71.4% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[76.2% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[81.0% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[85.7% done] [0K/6285K /s] [0 /5 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[90.5% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[95.2% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[100.0% done] [0K/8380K /s] [0 /7 ioJobs: 57 (f=57):
[______www_wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[13.7% done] [0K/8380K /s] [0 /7 iopJobs: 50 (f=50):
[______www____ww_wwww_wwwwwwwwwwwwwwwwwwwww__wwwwwwwwwwwwwwwwwwww]
[7.5% done] [0K/7332K /s] [0 /6 iopsJobs: 42 (f=42):
[______ww_____ww__www__wwwwwww_w_wwwwwwwwww___www_wwwwwww_wwwwwww]
[5.0% done] [0K/8380K /s] [0 /7 iopsJobs: 35 (f=35):
[______ww_____ww__www__wwwwwww_w_www_wwwwww___www___ww_ww_ww___ww]
[3.9% done] [0K/7332K /s] [0 /6 iopsJobs: 26 (f=26):
[______w______ww__www____www_w_w__ww_wwwww____www____w__w_w____ww]
[3.0% done] [0K/9427K /s] [0 /8 iopsJobs: 21 (f=21):
[______w______ww__www_____w__w____w__wwww_____www____w__w_w____ww]
[2.8% done] [0K/5237K /s] [0 /4 iopsJobs: 16 (f=16):
[______w______ww__www________w____w____w______w_w____w__w_w____ww]
[2.5% done] [0K/5237K /s] [0 /4 iopsJobs: 10 (f=10):
[_________________www________w____w____w______w_w____w__w________]
[2.3% done] [0K/6285K /s] [0 /5 iopsJobs: 4 (f=4):
[_________________ww__________________________w_________w________]
[2.1% done] [0K/6285K /s] [0 /5 iops] [eta 23m:25s]
test-rand-write: (groupid=0, jobs=64): err= 0: pid=25265
write: io=243712KB, bw=8317.6KB/s, iops=8 , runt= 29301msec
clat (msec): min=31 , max=1770 , avg=770.31, stdev=325.68
lat (msec): min=32 , max=1770 , avg=770.49, stdev=325.67
bw (KB/s) : min= 34, max= 5616, per=11.89%, avg=989.00, stdev=852.00
cpu : usr=0.00%, sys=0.01%, ctx=2291, majf=2, minf=1833
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/238/0, short=0/0/0
lat (msec): 50=8.40%, 100=2.94%, 500=2.94%, 750=15.13%, 1000=60.08%
lat (msec): 2000=10.50%
Run status group 0 (all jobs):
WRITE: io=243712KB, aggrb=8317KB/s, minb=8517KB/s, maxb=8517KB/s,
mint=29301msec, maxt=29301msec
Disk stats (read/write):
nbd4: ios=0/2138, merge=0/0, ticks=0/1558972, in_queue=1561912, util=99.67%
$ sudo fio -filename=/dev/nbd4 -direct=1 -rw=randread -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[14.3% done] [24093K/0K /s] [22 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[19.0% done] [6285K/0K /s] [5 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[23.8% done] [15712K/0K /s] [14 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[28.6% done] [19903K/0K /s] [18 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[33.3% done] [28283K/0K /s] [26 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[38.1% done] [19903K/0K /s] [18 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[42.9% done] [19903K/0K /s] [18 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[47.6% done] [25140K/0K /s] [23 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[52.4% done] [27235K/0K /s] [25 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[57.1% done] [26188K/0K /s] [24 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[61.9% done] [8380K/0K /s] [7 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[66.7% done] [17807K/0K /s] [16 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[71.4% done] [18855K/0K /s] [17 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[76.2% done] [23045K/0K /s] [21 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[81.0% done] [24093K/0K /s] [22 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[85.7% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[90.5% done] [20950K/0K /s] [19 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[95.2% done] [25140K/0K /s] [23 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[100.0% done] [27235K/0K /s] [25 /0 Jobs: 54 (f=54):
[________rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr__]
[10.0% done] [13617K/0K /s] [12 /0 iJobs: 32 (f=32):
[______________________________rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr__]
[3.4% done] [23045K/0K /s] [21 /0 ioJobs: 25 (f=25):
[_____________________________________rrrrrrrrrrrrrrrrrrrrrrrrr__]
[2.9% done] [7332K/0K /s] [6 /0 iops] [eta 13m:31s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=25358
read : io=456704KB, bw=19192KB/s, iops=18 , runt= 23797msec
clat (msec): min=102 , max=4214 , avg=3174.37, stdev=831.38
lat (msec): min=102 , max=4214 , avg=3174.37, stdev=831.38
bw (KB/s) : min= 242, max= 1935, per=1.88%, avg=361.12, stdev=203.80
cpu : usr=0.00%, sys=0.00%, ctx=882, majf=0, minf=18283
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=446/0/0, short=0/0/0
lat (msec): 250=0.67%, 500=0.90%, 750=1.12%, 1000=1.35%, 2000=4.93%
lat (msec): >=2000=91.03%
Run status group 0 (all jobs):
READ: io=456704KB, aggrb=19191KB/s, minb=19652KB/s, maxb=19652KB/s,
mint=23797msec, maxt=23797msec
Disk stats (read/write):
nbd4: ios=3976/0, merge=0/0, ticks=3083088/0, in_queue=3101528, util=99.61%
hdfs模式
========
$ sudo fio -filename=/dev/nbd5 -direct=1 -rw=randwrite -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-write
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[14.3% done] [0K/15712K /s] [0 /14 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[19.0% done] [0K/16760K /s] [0 /15 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[23.8% done] [0K/1047K /s] [0 /0 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[28.6% done] [0K/0K /s] [0 /0 iops] Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[33.3% done] [0K/13617K /s] [0 /12 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[38.1% done] [0K/30378K /s] [0 /28 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[42.9% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[47.6% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[52.4% done] [0K/11522K /s] [0 /10 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[57.1% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[61.9% done] [0K/13617K /s] [0 /12 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[66.7% done] [0K/18855K /s] [0 /17 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[71.4% done] [0K/17790K /s] [0 /16 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[76.2% done] [0K/15712K /s] [0 /14 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[81.0% done] [0K/6285K /s] [0 /5 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[85.7% done] [0K/13617K /s] [0 /12 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[90.5% done] [0K/11522K /s] [0 /10 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[95.2% done] [0K/12557K /s] [0 /11 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[100.0% done] [0K/23045K /s] [0 /21 Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[2200.0% done] [0K/2095K /s] [0 /1 iJobs: 44 (f=44):
[__________________wwwwwwwwwwwwwwwwwwwwww_wwww_wwwwwwwwwwwwwwwwww]
[5.2% done] [0K/20950K /s] [0 /19 ioJobs: 31 (f=31):
[_______________________________wwwwwwwww_wwww_wwwwwwwwwwwwwwwwww]
[3.3% done] [0K/13617K /s] [0 /12 ioJobs: 19 (f=19):
[__________________________________________w_w__wwwwwwwwwwwwwwwww]
[2.5% done] [0K/12557K /s] [0 /11 iops] [eta 16m:15s]
test-rand-write: (groupid=0, jobs=64): err= 0: pid=25656
write: io=346112KB, bw=13993KB/s, iops=13 , runt= 24734msec
clat (msec): min=30 , max=3219 , avg=475.87, stdev=492.35
lat (msec): min=30 , max=3219 , avg=476.07, stdev=492.35
bw (KB/s) : min= 41, max= 7806, per=11.92%, avg=1668.13, stdev=1682.47
cpu : usr=0.00%, sys=0.01%, ctx=3308, majf=1, minf=1834
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/338/0, short=0/0/0
lat (msec): 50=7.40%, 100=7.69%, 250=15.68%, 500=42.01%, 750=9.17%
lat (msec): 1000=5.62%, 2000=10.65%, >=2000=1.78%
Run status group 0 (all jobs):
WRITE: io=346112KB, aggrb=13993KB/s, minb=14329KB/s, maxb=14329KB/s,
mint=24734msec, maxt=24734msec
Disk stats (read/write):
nbd5: ios=0/2988, merge=0/0, ticks=0/1366864, in_queue=1370284, util=99.63%
$ sudo fio -filename=/dev/nbd5 -direct=1 -rw=randread -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[14.3% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[19.0% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[23.8% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[28.6% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[33.3% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[38.1% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[42.9% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[47.6% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[52.4% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[57.1% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[61.9% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[66.7% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[71.4% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[76.2% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[81.0% done] [4185K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[85.7% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[90.5% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[95.2% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[100.0% done] [4190K/0K /s] [3 /0 ioJobs: 60 (f=60):
[rrrrrrrrrrrrrrrrrr_r_r_r_rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[21.8% done] [4190K/0K /s] [3 /0 iopJobs: 56 (f=56):
[rr_rrrrrrrrrrrrr_r_r_r_r_r_rr_rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[12.5% done] [4190K/0K /s] [3 /0 iopJobs: 53 (f=53):
[____rrrrrrrrrrrr_r_r_r_r_r_rr_rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[9.6% done] [3142K/0K /s] [2 /0 iopsJobs: 42 (f=42):
[_________r_r_r_r_r_r_r_r_r_rr_rrrrrrrrrrrrrrrrrrrr___rrrrrrrrrrr]
[5.0% done] [11522K/0K /s] [10 /0 ioJobs: 36 (f=36):
[_________r_r_r_______r_r_r_rr_rrrrrrrrrrrrrrrrrrrr___rr___rrrrrr]
[4.0% done] [6285K/0K /s] [5 /0 iopsJobs: 28 (f=28):
[_________r_r_r______________r_rrrrrrrrrrrrrrrrrrrr___rr_______rr]
[3.2% done] [8380K/0K /s] [7 /0 iopsJobs: 15 (f=15):
[_____________r_______________________rrrrrrrrrrrr____rr_________]
[2.4% done] [13617K/0K /s] [12 /0 ioJobs: 10 (f=10):
[________________________________________rrrrrrrrr_____r_________]
[2.2% done] [5237K/0K /s] [4 /0 iops] [eta 21m:25s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=25874
read : io=123904KB, bw=4351.2KB/s, iops=4 , runt= 28476msec
clat (msec): min=492 , max=21578 , avg=13272.26, stdev=5444.60
lat (msec): min=492 , max=21578 , avg=13272.26, stdev=5444.60
bw (KB/s) : min= 47, max= 1089, per=2.58%, avg=112.23, stdev=133.49
cpu : usr=0.00%, sys=0.00%, ctx=228, majf=0, minf=18283
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=121/0/0, short=0/0/0
lat (msec): 500=0.83%, 1000=0.83%, 2000=1.65%, >=2000=96.69%
Run status group 0 (all jobs):
READ: io=123904KB, aggrb=4351KB/s, minb=4455KB/s, maxb=4455KB/s,
mint=28476msec, maxt=28476msec
Disk stats (read/write):
nbd5: ios=1064/0, merge=0/0, ticks=3653304/0, in_queue=3701932, util=99.71%
--
Thanks
Harry Wei
最近使用NBD时发现有bug，具体如下：
I use nbd-2.9.15 and Linux kernel version is 3.5.0. The errors are
like following.
nbd-server:
** Message: connect from 127.0.0.1, assigned file is (null)
** Message: Authorized client
** Message: Starting to serve
Error: Failed/5:Resource temporarily unavailable
Exiting.
** Message: Child exited with 1
dmesg outputs:
[245826.049068] block nbd1: Receive control failed (result -32)
[245826.049129] block nbd1: shutting down socket
[245826.049179] block nbd1: queue cleared
HLFS report error messages like following.
[...]
17 20130512 14:16:43.414 DEBUG hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/utils/storage_helper.c][init
_storage_handler][28]enter func init_storage_handler
18 20130512 14:16:43.414 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/utils/storage_helper.c][init
_storage_handler][39]parse_from_uri happened error
19 20130512 14:16:43.415 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/storage/init_hlfs.c][__init_
hlfs][114][uri:testfs] can not accessable
20 20130512 14:16:43.415 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/src/storage/init_hlfs.c][init_hl
fs][227]init raw hlfs ctrl failed
21 20130512 14:16:43.415 ERROR hlfslog-
[0x723200][/home/jiawei/workshop1/hlfs_trunk/nbd-2.9.15/nbd-server.c][negotia
te][1280]Failed to init storage control block
目前以上bug已经修复，随后把修复后的nbd放到我的分支上，也许还存在漏洞，但是
我已经可以正常使用。刚才在standalone方式下测了下HLFS的性能具体如下：
local模式
==========
$ sudo fio -filename=/dev/nbd4 -direct=1 -rw=randwrite -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-write
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[14.3% done] [0K/3142K /s] [0 /2 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[19.0% done] [0K/6285K /s] [0 /5 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[23.8% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[28.6% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[33.3% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[38.1% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[42.9% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[47.6% done] [0K/7332K /s] [0 /6 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[52.4% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[57.1% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[61.9% done] [0K/4190K /s] [0 /3 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[66.7% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[71.4% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[76.2% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[81.0% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[85.7% done] [0K/6285K /s] [0 /5 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[90.5% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[95.2% done] [0K/8380K /s] [0 /7 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[100.0% done] [0K/8380K /s] [0 /7 ioJobs: 57 (f=57):
[______www_wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[13.7% done] [0K/8380K /s] [0 /7 iopJobs: 50 (f=50):
[______www____ww_wwww_wwwwwwwwwwwwwwwwwwwww__wwwwwwwwwwwwwwwwwwww]
[7.5% done] [0K/7332K /s] [0 /6 iopsJobs: 42 (f=42):
[______ww_____ww__www__wwwwwww_w_wwwwwwwwww___www_wwwwwww_wwwwwww]
[5.0% done] [0K/8380K /s] [0 /7 iopsJobs: 35 (f=35):
[______ww_____ww__www__wwwwwww_w_www_wwwwww___www___ww_ww_ww___ww]
[3.9% done] [0K/7332K /s] [0 /6 iopsJobs: 26 (f=26):
[______w______ww__www____www_w_w__ww_wwwww____www____w__w_w____ww]
[3.0% done] [0K/9427K /s] [0 /8 iopsJobs: 21 (f=21):
[______w______ww__www_____w__w____w__wwww_____www____w__w_w____ww]
[2.8% done] [0K/5237K /s] [0 /4 iopsJobs: 16 (f=16):
[______w______ww__www________w____w____w______w_w____w__w_w____ww]
[2.5% done] [0K/5237K /s] [0 /4 iopsJobs: 10 (f=10):
[_________________www________w____w____w______w_w____w__w________]
[2.3% done] [0K/6285K /s] [0 /5 iopsJobs: 4 (f=4):
[_________________ww__________________________w_________w________]
[2.1% done] [0K/6285K /s] [0 /5 iops] [eta 23m:25s]
test-rand-write: (groupid=0, jobs=64): err= 0: pid=25265
write: io=243712KB, bw=8317.6KB/s, iops=8 , runt= 29301msec
clat (msec): min=31 , max=1770 , avg=770.31, stdev=325.68
lat (msec): min=32 , max=1770 , avg=770.49, stdev=325.67
bw (KB/s) : min= 34, max= 5616, per=11.89%, avg=989.00, stdev=852.00
cpu : usr=0.00%, sys=0.01%, ctx=2291, majf=2, minf=1833
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/238/0, short=0/0/0
lat (msec): 50=8.40%, 100=2.94%, 500=2.94%, 750=15.13%, 1000=60.08%
lat (msec): 2000=10.50%
Run status group 0 (all jobs):
WRITE: io=243712KB, aggrb=8317KB/s, minb=8517KB/s, maxb=8517KB/s,
mint=29301msec, maxt=29301msec
Disk stats (read/write):
nbd4: ios=0/2138, merge=0/0, ticks=0/1558972, in_queue=1561912, util=99.67%
$ sudo fio -filename=/dev/nbd4 -direct=1 -rw=randread -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[14.3% done] [24093K/0K /s] [22 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[19.0% done] [6285K/0K /s] [5 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[23.8% done] [15712K/0K /s] [14 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[28.6% done] [19903K/0K /s] [18 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[33.3% done] [28283K/0K /s] [26 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[38.1% done] [19903K/0K /s] [18 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[42.9% done] [19903K/0K /s] [18 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[47.6% done] [25140K/0K /s] [23 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[52.4% done] [27235K/0K /s] [25 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[57.1% done] [26188K/0K /s] [24 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[61.9% done] [8380K/0K /s] [7 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[66.7% done] [17807K/0K /s] [16 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[71.4% done] [18855K/0K /s] [17 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[76.2% done] [23045K/0K /s] [21 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[81.0% done] [24093K/0K /s] [22 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[85.7% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[90.5% done] [20950K/0K /s] [19 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[95.2% done] [25140K/0K /s] [23 /0 iJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[100.0% done] [27235K/0K /s] [25 /0 Jobs: 54 (f=54):
[________rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr__]
[10.0% done] [13617K/0K /s] [12 /0 iJobs: 32 (f=32):
[______________________________rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr__]
[3.4% done] [23045K/0K /s] [21 /0 ioJobs: 25 (f=25):
[_____________________________________rrrrrrrrrrrrrrrrrrrrrrrrr__]
[2.9% done] [7332K/0K /s] [6 /0 iops] [eta 13m:31s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=25358
read : io=456704KB, bw=19192KB/s, iops=18 , runt= 23797msec
clat (msec): min=102 , max=4214 , avg=3174.37, stdev=831.38
lat (msec): min=102 , max=4214 , avg=3174.37, stdev=831.38
bw (KB/s) : min= 242, max= 1935, per=1.88%, avg=361.12, stdev=203.80
cpu : usr=0.00%, sys=0.00%, ctx=882, majf=0, minf=18283
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=446/0/0, short=0/0/0
lat (msec): 250=0.67%, 500=0.90%, 750=1.12%, 1000=1.35%, 2000=4.93%
lat (msec): >=2000=91.03%
Run status group 0 (all jobs):
READ: io=456704KB, aggrb=19191KB/s, minb=19652KB/s, maxb=19652KB/s,
mint=23797msec, maxt=23797msec
Disk stats (read/write):
nbd4: ios=3976/0, merge=0/0, ticks=3083088/0, in_queue=3101528, util=99.61%
hdfs模式
========
$ sudo fio -filename=/dev/nbd5 -direct=1 -rw=randwrite -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-write
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-write: (g=0): rw=randwrite, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[14.3% done] [0K/15712K /s] [0 /14 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[19.0% done] [0K/16760K /s] [0 /15 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[23.8% done] [0K/1047K /s] [0 /0 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[28.6% done] [0K/0K /s] [0 /0 iops] Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[33.3% done] [0K/13617K /s] [0 /12 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[38.1% done] [0K/30378K /s] [0 /28 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[42.9% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[47.6% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[52.4% done] [0K/11522K /s] [0 /10 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[57.1% done] [0K/9427K /s] [0 /8 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[61.9% done] [0K/13617K /s] [0 /12 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[66.7% done] [0K/18855K /s] [0 /17 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[71.4% done] [0K/17790K /s] [0 /16 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[76.2% done] [0K/15712K /s] [0 /14 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[81.0% done] [0K/6285K /s] [0 /5 iopJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[85.7% done] [0K/13617K /s] [0 /12 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[90.5% done] [0K/11522K /s] [0 /10 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[95.2% done] [0K/12557K /s] [0 /11 iJobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[100.0% done] [0K/23045K /s] [0 /21 Jobs: 64 (f=64):
[wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww]
[2200.0% done] [0K/2095K /s] [0 /1 iJobs: 44 (f=44):
[__________________wwwwwwwwwwwwwwwwwwwwww_wwww_wwwwwwwwwwwwwwwwww]
[5.2% done] [0K/20950K /s] [0 /19 ioJobs: 31 (f=31):
[_______________________________wwwwwwwww_wwww_wwwwwwwwwwwwwwwwww]
[3.3% done] [0K/13617K /s] [0 /12 ioJobs: 19 (f=19):
[__________________________________________w_w__wwwwwwwwwwwwwwwww]
[2.5% done] [0K/12557K /s] [0 /11 iops] [eta 16m:15s]
test-rand-write: (groupid=0, jobs=64): err= 0: pid=25656
write: io=346112KB, bw=13993KB/s, iops=13 , runt= 24734msec
clat (msec): min=30 , max=3219 , avg=475.87, stdev=492.35
lat (msec): min=30 , max=3219 , avg=476.07, stdev=492.35
bw (KB/s) : min= 41, max= 7806, per=11.92%, avg=1668.13, stdev=1682.47
cpu : usr=0.00%, sys=0.01%, ctx=3308, majf=1, minf=1834
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=0/338/0, short=0/0/0
lat (msec): 50=7.40%, 100=7.69%, 250=15.68%, 500=42.01%, 750=9.17%
lat (msec): 1000=5.62%, 2000=10.65%, >=2000=1.78%
Run status group 0 (all jobs):
WRITE: io=346112KB, aggrb=13993KB/s, minb=14329KB/s, maxb=14329KB/s,
mint=24734msec, maxt=24734msec
Disk stats (read/write):
nbd5: ios=0/2988, merge=0/0, ticks=0/1366864, in_queue=1370284, util=99.63%
$ sudo fio -filename=/dev/nbd5 -direct=1 -rw=randread -bs=1M -size=1G
-numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
...
test-rand-read: (g=0): rw=randread, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 1.59
Starting 64 processes
Jobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[14.3% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[19.0% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[23.8% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[28.6% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[33.3% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[38.1% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[42.9% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[47.6% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[52.4% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[57.1% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[61.9% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[66.7% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[71.4% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[76.2% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[81.0% done] [4185K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[85.7% done] [2095K/0K /s] [1 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[90.5% done] [3142K/0K /s] [2 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[95.2% done] [4190K/0K /s] [3 /0 iopJobs: 64 (f=64):
[rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[100.0% done] [4190K/0K /s] [3 /0 ioJobs: 60 (f=60):
[rrrrrrrrrrrrrrrrrr_r_r_r_rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[21.8% done] [4190K/0K /s] [3 /0 iopJobs: 56 (f=56):
[rr_rrrrrrrrrrrrr_r_r_r_r_r_rr_rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[12.5% done] [4190K/0K /s] [3 /0 iopJobs: 53 (f=53):
[____rrrrrrrrrrrr_r_r_r_r_r_rr_rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr]
[9.6% done] [3142K/0K /s] [2 /0 iopsJobs: 42 (f=42):
[_________r_r_r_r_r_r_r_r_r_rr_rrrrrrrrrrrrrrrrrrrr___rrrrrrrrrrr]
[5.0% done] [11522K/0K /s] [10 /0 ioJobs: 36 (f=36):
[_________r_r_r_______r_r_r_rr_rrrrrrrrrrrrrrrrrrrr___rr___rrrrrr]
[4.0% done] [6285K/0K /s] [5 /0 iopsJobs: 28 (f=28):
[_________r_r_r______________r_rrrrrrrrrrrrrrrrrrrr___rr_______rr]
[3.2% done] [8380K/0K /s] [7 /0 iopsJobs: 15 (f=15):
[_____________r_______________________rrrrrrrrrrrr____rr_________]
[2.4% done] [13617K/0K /s] [12 /0 ioJobs: 10 (f=10):
[________________________________________rrrrrrrrr_____r_________]
[2.2% done] [5237K/0K /s] [4 /0 iops] [eta 21m:25s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=25874
read : io=123904KB, bw=4351.2KB/s, iops=4 , runt= 28476msec
clat (msec): min=492 , max=21578 , avg=13272.26, stdev=5444.60
lat (msec): min=492 , max=21578 , avg=13272.26, stdev=5444.60
bw (KB/s) : min= 47, max= 1089, per=2.58%, avg=112.23, stdev=133.49
cpu : usr=0.00%, sys=0.00%, ctx=228, majf=0, minf=18283
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w/d: total=121/0/0, short=0/0/0
lat (msec): 500=0.83%, 1000=0.83%, 2000=1.65%, >=2000=96.69%
Run status group 0 (all jobs):
READ: io=123904KB, aggrb=4351KB/s, minb=4455KB/s, maxb=4455KB/s,
mint=28476msec, maxt=28476msec
Disk stats (read/write):
nbd5: ios=1064/0, merge=0/0, ticks=3653304/0, in_queue=3701932, util=99.71%
--
Thanks
Harry Wei
harryxiyou
May 13, 2013, 5:18:15 AM5/13/13
to clo...@googlegroups.com, Kang Hua, Chen Lijun
Ren Zhen
May 13, 2013, 8:57:43 AM5/13/13
to clo...@googlegroups.com
嗨，贾威
你应该简单的描述一下bug,怎么fix的 (:-\
虽然上次听过你介绍fio工具的使用，还是不太能看懂。
你做个简单的分析总结好点。
>
>
>
>
> --
> Thanks
> Harry Wei
>
> --
> 您收到此邮件是因为您订阅了 Google 网上论坛的“cloudxy”论坛。
> 要退订此论坛并停止接收此论坛的电子邮件，请发送电子邮件到 cloudxy+u...@googlegroups.com。
> 要查看更多选项，请访问 https://groups.google.com/groups/opt_out。
>
>
--
Sincerely,
Ren Zhen
> 目前以上bug已经修复，随后把修复后的nbd放到我的分支上，也许还存在漏洞，但是
> 我已经可以正常使用。刚才在standalone方式下测了下HLFS的性能具体如下：
写成两封邮件好些。
> 我已经可以正常使用。刚才在standalone方式下测了下HLFS的性能具体如下：
你做个简单的分析总结好点。
>
>
>
>
> --
> Thanks
> Harry Wei
>
> --
> 您收到此邮件是因为您订阅了 Google 网上论坛的“cloudxy”论坛。
> 要退订此论坛并停止接收此论坛的电子邮件，请发送电子邮件到 cloudxy+u...@googlegroups.com。
> 要查看更多选项，请访问 https://groups.google.com/groups/opt_out。
>
>
--
Sincerely,
Ren Zhen
Reply all
Reply to author
Forward
0 new messages