SATA throughput?

[Thomas Kaiser]

Hello everyone,

I wonder what throughput do you get when accessing SATA drives attached to the cubietruck or cubieboard2? I'm especially interested in sequential write speeds. What I've found so far on the net is this:

1) cubieboard (Intel X25-M SSD, "hdparm -t", http://saturn.ffzg.hr/rot13/index.cgi?cubieboard )
write: n/a / read: 153 MB/s

2) cubieboard2 (Intel 313 SSD, "hdparm -t", http://irclog.whitequark.org/linux-sunxi/2013-09-21 )
write: n/a / read: 69 MB/s

3) cubietruck (SAMSUNG SSD 830, partial wrong use of dd with raw device, http://tinyurl.com/pdzxl8v )
write: 148 MB/s, read: 106 MB/s

4) cubietruck (WD WD40EZRX HD, "iozone/bonnie++", my own tests: http://tinyurl.com/or6pzzu ):
write: 38 MB/s / read: 145 MB/s

The 148 MB/s writes to the Samsung SSD are obviously misleading since using dd without 'oflag=direct', a test size of just 957 MB on a linux system with 2 GB RAM might test system buffers/caches but neither the disk nor the SATA implementation of the board in question (BTW: The test results seem to be copy&paste from http://82.66.125.195/public/wp/?p=66 or vice versa).

Since I found not a single write performance result for a Cubieboard/Cubietruck over SATA I still have no clue whether my results can be considered typical or not (the HD I used is many times faster -- tested it on other systems as well).

Anyone here sharing his experiences or better results?

Thx,

Thomas

BTW: Since I didn't found iozone ARM binaries for Ubuntu 12.04 I created one myself:
http://kaiser-edv.de/downloads/iozone3_397-2ubuntu1_armhf.deb (md5sum: a94c8ee7a5efc1bb297b7220cc558f7c)
http://kaiser-edv.de/downloads/iozone3_397-2ubuntu1_armel.deb (md5sum: d13ae0963716da55597f577fd2f17980)

[Thomas Kaiser]

I forgot to mention this test:

5) cubieboard (SanDisk Extreme 120GB SSD, bonnie++, http://tinyurl.com/ld4d5za )

    write: 41 MB/s, read: 104 MB/s

Seems like the cubie*'s SATA implementation is limited to approx. 40 MB/sec sequential writes?

[Florian Heigl]

Hi Thomas,

I've seen performance similar to what you reported here, and I could also see the drop with concurrent network IO you reported.

For me it's still sufficient, but I tried to do some tuning and didn't find any that would write perf above 40MBytes.

This is proper sync write to an old OCZ vertex2 which can manage up to 180MB/s write.

root@cubie1:/mnt/src# dd if=/dev/zero of=blah2 bs=1024k count=1024 conv=fdatasync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB) copied, 28.3741 s, 37.8 MB/s

Read perf is much nicer:

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB) copied, 6.76209 s, 159 MB/s

2MB readahead gives about the maximum throughput

root@cubie1:/mnt/src# dd if=/dev/sda of=/dev/null bs=1024k count=1024

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB) copied, 5.81373 s, 185 MB/s

I dropped caches between runs.

So, the read performance is really great, while writing sucks.

Hard to tell what's causing this, hopefully just a driver issue. :>

Florian

Ps.: I didn't find the other thread any more. 

I'd say it's rather pointless to point someone at iozone etc. if he already hit a dead-end bottleneck with such a light benchmark as DD)

[Roman Mamedov]

I think you are expecting way too much from an $50 4-watt ARM board,
"root@cubie1", even the single-core version of it? 37.8 MB/sec when writing to
an actual filesystem with all the housekeeping that's involved, is a pretty
amazing result for such device, doesn't necessarily indicate a "driver issue".
In any case, try writing to a raw block device and see if it's any faster.

--
With respect,
Roman

[Florian Heigl]

Hi Roman,

thanks for your mail.

On 11.12.2013, at 00:18, Roman Mamedov <r...@romanrm.net> wrote:

> On Tue, 10 Dec 2013 15:10:51 -0800 (PST)
> Florian Heigl <floria...@gmail.com> wrote:
>
>> Hi Thomas,
>>
>> I've seen performance similar to what you reported here, and I could also
>> see the drop with concurrent network IO you reported.
>> For me it's still sufficient, but I tried to do some tuning and didn't find
>> any that would write perf above 40MBytes.

>> 2MB readahead gives about the maximum throughput
>> root@cubie1:/mnt/src# dd if=/dev/sda of=/dev/null bs=1024k count=1024
>> 1024+0 records in
>> 1024+0 records out
>> 1073741824 bytes (1.1 GB) copied, 5.81373 s, 185 MB/s
>>
>> I dropped caches between runs.
>> So, the read performance is really great, while writing sucks.
>>
>> Hard to tell what's causing this, hopefully just a driver issue.
>
> I think you are expecting way too much from an $50 4-watt ARM board,
> "root@cubie1", even the single-core version of it? 37.8 MB/sec when writing to
> an actual filesystem with all the housekeeping that's involved, is a pretty
> amazing result for such device, doesn't necessarily indicate a "driver issue".
> In any case, try writing to a raw block device and see if it's any faster.

No it’s not faster writing to a raw device.
The CPU load is below maxing out a one of the cores.

The cubietruck simple gives a lot more options,
so of course we’ll find some things that it can’t handle.

So far I’ve never seen a system that manages to hit over 4 times read vs. write throughput.
I’m not saying it’ll be fixable, especially if it’s something in how it works IO-wise.

Still it won’t hurt to accept “this is odd” instead of “this is how it is"
Incidentially: My $100 NAS has less than 50% of the CPU power (1core, armv5, 1.2GHz), 1/4 the RAM and sustains a 100MB/s write and read rate.
Thanks to the NAS I also don’t care about the performance really.

Greetings,
Florian

[Thomas Kaiser]

Hi,

thanks for sharing your results. In the meantime I found another reference on the net regarding A10 SATA write speeds (measured correctly with dd using oflag=direct):

     https://groups.google.com/forum/#!msg/linux-sunxi/fZbKdS9m6bc/-57nxQUrsJwJ 

Regarding iozone I posted the links to the .debs right here in this thread at the beginning:

http://kaiser-edv.de/downloads/iozone3_397-2ubuntu1_armhf.deb (md5sum: a94c8ee7a5efc1bb297b7220cc558f7c)

http://kaiser-edv.de/downloads/iozone3_397-2ubuntu1_armel.deb (md5sum: d13ae0963716da55597f577fd2f17980)

Using bonnie++/iozone is always better than a limited dd test since it might give you hints where the bottleneck is or what scenarios to better avoid with a certain combination of hardware/drivers.

Cheers,

Thomas

Florian Heigl wrote:

I've seen performance similar to what you reported here […]

[Thomas Kaiser]

Hi,

I've also a "WandBoard" with SATA and Gbit Ethernet. It's based on an i.MX6 from Freescale (like the Nitrogen6X or SABRE from Boundary Devices), the SATA throughput is better or let's say more balanced (80-90 MB/sec write, +100 MB/sec read). But the Ethernet throughput is horrible without further adjustments as outlined here:

http://boundarydevices.com/i-mx6-ethernet/

I believe people (like me ;-) read something about SATA and Gbit Ethernet and think 'hey that would build a cool NAS device too'. Unfortunately some devices have poor write rates when it comes to SATA other while accessing the board through the network and some combine both.

At the moment I believe only the Marvell Kirkwood based ARM boards (88F6281, 88F6282) perform good as a NAS (since both SATA and Gbit Ethernet allow high throughput simultaneously) even with less CPU power and way less RAM compared to cubieboard2/cubietruck. No wonder that many NAS vendors chose these SoMs.

But I still ask me whether the low SATA write speeds A10/A20 based devices seem to share are chipset or driver related?

BTW: At the moment I'm totally happy with my 'CubieNAS' used as a backup device for Macs. The backup process isn't that fast (12-15 MB/sec), the restore rates are perfectly fine (40-50 MB/sec) and the low power consumption is awesome.

A big 'thank you!' to all the people who contributed so much good work to let these tiny devices do so many fascinating things! :-)

Best regards,

Thomas

[Michal Suchanek]

On 11 December 2013 13:30, Thomas Kaiser <thomas...@phg-online.de> wrote:
> Hi,
>
> I've also a "WandBoard" with SATA and Gbit Ethernet. It's based on an i.MX6 from Freescale (like the Nitrogen6X or SABRE from Boundary Devices), the SATA throughput is better or let's say more balanced (80-90 MB/sec write, +100 MB/sec read). But the Ethernet throughput is horrible without further adjustments as outlined here:
>
> http://boundarydevices.com/i-mx6-ethernet/
>
> I believe people (like me ;-) read something about SATA and Gbit Ethernet and think 'hey that would build a cool NAS device too'. Unfortunately some devices have poor write rates when it comes to SATA other while accessing the board through the network and some combine both.
>
> At the moment I believe only the Marvell Kirkwood based ARM boards (88F6281, 88F6282) perform good as a NAS (since both SATA and Gbit Ethernet allow high throughput simultaneously) even with less CPU power and way less RAM compared to cubieboard2/cubietruck. No wonder that many NAS vendors chose these SoMs.

It seems it has no graphics accelerator but it has 2xPCIe which people
actually use for extending x86 notebooks with external graphics card -
you get an expresscard slot and a mini PCIe slot for WiFi on many.
With dual Gbit and dual SATA .. maybe somebody interested in doing
some hacks with those?

Thanks

Michal

[Thomas Kaiser]

Hi Michal,

the boards or systems I found were rather expensive (for example GuruPlug/DreamPlug) and lack both multimedia as well as GPIO expansion possibilities as you already mentioned. Maybe the best starting point for experiments are 'real' NAS devices -- they're cheaper and include already an enclosure :-)

http://www.qnap.com/static/products/comparison/All_NAS.php (HOME & SOHO)
http://forum.synology.com/wiki/index.php/What_kind_of_CPU_does_my_NAS_have

But since they come already with loads of (user friendly) software playing around with this stuff seems a bit weird (unless you have very specific needs)

Best,

Thomas

Michal Suchanek wrote:

> [Marvell Kirkwood based ARM boards]