Micro SD card writes very slow

[chameleon452]

Hello,

I cannot get my BBB to use the external micro SD card with decent speeds. I have set up Angstrom on a Sandisk 16GB Class 10. With a USB card reader on Windows I get read speeds of 15-20MB/s and write speeds of 8MB/s. Under Angstrom, hdparm -t returns

/dev/mmcblk0: (SD card)

 Timing buffered disk reads:  18 MB in  3.29 seconds =   5.47 MB/sec

/dev/mmcblk1: (internal EMMC)

 Timing buffered disk reads:  50 MB in  3.06 seconds =  16.34 MB/sec

I did not find out how to benchmark the write speed without losing data, but from copying a few files it is <1 MB/sec for the micro SD card. Also, it appears that Linux uses write caching on the SD card, and when it decides to flush the buffer it can stall for several seconds due to the slow write speed. What is going on?

[Roy Bellingan]

> I did not find out how to benchmark the write speed without losing
> data, but from copying a few files it is <1 MB/sec for the micro SD
> card. Also, it appears that Linux uses write caching on the SD card,
> and when it decides to flush the buffer it can stall for several
> seconds due to the slow write speed. What is going on?
>

You can pipe some data with dd

http://romanrm.ru/en/dd-benchmark

[William Hermans]

Remember, eMMC access is 8bit, uSD card access is 4bit. This is a hardware limitation as far as I know.

--
For more options, visit http://beagleboard.org/discuss
--- You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jens S.]

I too was wondering about the slow access to the microSD card, so I did some testing, and got some surprising results:

I used dd to measure speed, using:

dd bs=1M count=512 if=/dev/zero of=test conv=fdatasync;rm test

...and:

dd bs=1M count=512 if=/dev/mmcblk0 of=/dev/null 

...or some slight variation as needed.

Internal eMMC:
Read: 18 MB/s
Write: 3.1 MB/s

Sandisk Ultra MicroSDHC 32GB Class 10 / UHS Class 1 (SDSDQUA-032G-U46A):
Read: 5.6 MB/s
Write: 3.1 MB/s

Testing done with newest Angstrom demo image. I got more or less the same results with Debian.

The exact same card in a Raspberry Pi B (512MB) using the SD-adapter supplied with the Sandisk MicroSD card:

Read: 21.0 MB/s

Write: 9.1 MB/s

(And for good measure I tested the same card in a USB3 reader on a PC and got 39.3/11.0 read/write)

Looking at the schematic for the BBB and the RPi, it seems that the SD slots on both are hooked up to the cpu more or less identically.

It may be a hardware limitation, but it doesn't seem to be 8 vs. 4 bit.

One would think that the BBB should be superior to, or at least on par with the RPi in that regard. Some misconfiguration of the io registers maybe???

[Jens S.]

Did some reading, and I am pretty sure that there is something fishy with the uSD slot.

To get a clearer picture of what is going on, I wanted to find out if the eMMC is performing at spec.

The datasheet for the Micron eMMC part says that it should be capable of 30MB/s read and 6.6MB/s write. It also states that the part supports DDR mode, which the AM335x parts does not. I'm not sure how non-DDR access affects the speed, but we are getting about half, and we also need to consider real-world overhead from the file-system etc., so that's probably ok, I don't know.

The eMMC part is connected to MMC1 on the MPU using the full 8 bit, while the uSD slot is connected to MMC0 at 4 bits.

I then found the Technical Reference Manual for the MPU (document spruh73i.pdf from Texas Instruments) to find out what we should expect from the MMC0 port at 4 bits.

On page 3869 it states:

Clock support

– 96-MHz functional clock source input

– up to 384Mbit/sec (48MByte/sec) in MMC mode 8-bit data transfer

– up to 192Mbit/sec (24MByte/sec) in High-Speed SD mode 4-bit data transfer
– up to 24Mbit/sec (3MByte/sec) in Default SD mode 1-bit data transfer

That means it should be possible to reach far better speeds (up to 24 MB/s) from a fast uSD card than we are getting now.  

MMC0 and MMC1 ports are independently configured, so I believe it is entirely possible that some register needs to be configured to reach maximum speed.

Now I just need to learn how to read and write those registers. :-)

[William Hermans]

The uSD card is 4bit versus the eMMC's 8bit datawidth . ..

--

For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.

[William Hermans]

Ok, sorry I stated the obvious going by what you wrote there ( sorry did not read the whole post first). Anyway, going by what I've seen said on these groups, I would not expect too much work in the area of "fixing" this.

1. you can boot from the eMMC.

2. you can netboot. ( possible to get 11.9MB/s reads, and 11.1MB/s writes )

3. you can boot from a USB device.( for me ~20MB/s writes, ~10.6MB/s reads )

Plenty of room there for different purposes, and at least two to avoid wearing out the eMMC( or uSD ). So you could for instance setup a development environment on the BBB, and compile your life away - Just be warned, it will be painfully slow. Example compiling nodejs on the BBB took me around 2 hours. In a virtual machine( Debian ) on a windows host it took 5minutes. <- One CPU 2GB ram.

Anyhow uSD is not going to be long term reliable, this is why I think you wont see much improvement( if any ) in this area.

[Gerald Coley]

So, does the Linux have any say in all this? These are raw numbers, No OS timer ticks.  No interruptions.No screen updates.

I suggest you write bare metal code and rerun your tests on all of this.

Gerald

[William Hermans]

Robert Nelson's Custom debian kernel as of 3.8.13-bone21 == 600-800K/s reads / writes. To uSD that is, which is a samsung 16GB class 10 uSD card.

[Robert Nelson]

On Tue, Sep 10, 2013 at 11:08 AM, William Hermans <yyr...@gmail.com> wrote:
> Robert Nelson's Custom debian kernel as of 3.8.13-bone21 == 600-800K/s reads
> / writes. To uSD that is, which is a samsung 16GB class 10 uSD card.

It really depends on the media.. (and how much you pay for it..)

SanDisk Ultra 8GB (advertised 30MB/s class 10):
debian@bbb:~$ sudo hdparm -tT /dev/mmcblk0

/dev/mmcblk0:
Timing cached reads: 372 MB in 2.00 seconds = 185.64 MB/sec
Timing buffered disk reads: 16 MB in 3.00 seconds = 5.33 MB/sec

eMMC:
debian@bbb:~$ sudo hdparm -tT /dev/mmcblk1

/dev/mmcblk1:
Timing cached reads: 444 MB in 2.01 seconds = 221.28 MB/sec
Timing buffered disk reads: 42 MB in 3.11 seconds = 13.51 MB/se

On the beaglebone black, it is 'slower' then other mmc implementations
using the same media and a 4bit interface. Probably just the early
nature of the existing mmc code in 3.8 for this hardware... I would
test v3.12, but my Ultra's are all at home..

Regards,

--
Robert Nelson
http://www.rcn-ee.com/

[William Hermans]

Robert, I did not mean that as a slight. I am perfectly happy. especially considering I have not contributed to the kernel at all, and it comes at no charge. Granted, I did not use hdparm, I used dd for ~1GB worth of data.

Anyhow . . .

william@arm:~$ sudo hdparm -tT /dev/mmcblk0

/dev/mmcblk0:

 Timing cached reads:   382 MB in  2.01 seconds = 190.40 MB/sec

 Timing buffered disk reads:  16 MB in  3.09 seconds =   5.18 MB/sec

[Jens S.]

Thanks to all of you for your feedback.

A few notes:

Bus-width is not the (entire) issue since Raspberry Pi can read 4x faster than BBB with the same cards, also using 4-bit access.

The controllers for the eMMC and the uSD seems to be identical except for bus-width and powercontrol. This suggests that Linux is fully capable of handling the controller.

Going bare metal may be more efficient than Linux, but again the RPi can do 4x the performance using Linux. (unfortunately docs on the RPi controller are not available)

I haven't tested every linux distro, but performance seems to be more or less identical between different distros.

I know there are ways around the issue (eg. USB), but I think it would be a shame not to use this fantastic device to it fullest potential. It may be silly of me to try to figure this out, but hey, sometimes the journey is the reward :-)

I'll let you know if I find a solution or a pointer to where the problem might be.

[Gerald Coley]

I was referring to the data you posted as it relates the hardware specification of the components in question and your assertion that those numbers could be met with a high level operating system,. 

That is all I was saying. And yes, bare metal will be faster as that is all it has to do.

Gerald

[William Hermans]

Maybe someone will take a look at Rasbian source code, and make adjustments ? *shrug*

Personally I do not really care. sd is not exactly reliable enough for production.

[Jens S.]

Sorry for not being clear about that. No, I don't expect full 24MB/s using Linux. I was only trying to argue that nothing in the hardware specs suggests that ~5.6MB/s is the limit for read speed.

[Gerald Coley]

Understood. Just to be clear on my end. People read these emails and posts. I don't want them led into the wrong direction if at all possible, Especially those that may just skim them.

Gerald

[Jens S.]

I agree.

Anyway, now I'm getting somewhere... Luckily the datapins on the uSD connector are exposed while the card is inserted, so I hooked up the oscilloscope, and probed the pins while reading and writing to the uSD card:

Pin 1 (DAT2): No activity (high)

Pin 2 (CD/DAT3): No activity (high)

Pin 3 (CMD): Some activity, varying freq. around 3 KHz

Pin 4 (VDD): High

Pin 5 (CLK): Slight variation around 47 MHz

Pin 6 (VSS): Low

Pin 7 (DAT0): Lots of activity, freq. varying wildly, but mostly around 6-7MHz

Pin 8 (DAT1): No activity (high)

No observable difference between reading and writing.

Doesn't that look like 1 bit mode?

[Gerald Coley]

LOL! Not luck. By design.

Yes, that does indeed sound like 1Bit mode.

Gerald

[Jens S.]

Thank the designer (you, I guess?) from me, it was a really satisfying moment when I probed those pins :-)

Unfortunately I have absolutely no idea how to proceed from here. Any pointers?

[André Prado]

It's probably using the SPI mode, you gotta change the device driver for the sdcard i guess

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Att
André

[André Prado]

And it does make sense because the SDCARD organization says that you must pay royalties if you want to use the 4 bits mode

--
Att
André

[Robert Nelson]

On Wed, Sep 11, 2013 at 12:19 PM, André Prado <andrep...@gmail.com> wrote:
> And it does make sense because the SDCARD organization says that you must
> pay royalties if you want to use the 4 bits mode

Until you realize the eMMC populated on the BeagleBone black doesn't
talk over SPI mode... Otherwise it too would be limited to 5Mb/s.

Obviously a driver problem..

[Jens S.]

I believe that the pattern of activity on pin 3 and 7 would be swapped when changing from read to write if you were right. The pattern of activity is the same, suggesting 1 bit SD access.

[Jens S.]

It's working !!!

I took a wild guess, and decompiled /boot/am335x-boneblack.dtb

I found the parts for the three mmc controllers. I figured that mmc@48060000 was the uSD slot and mmc@481d8000 was the eMMC part. The second one had this line: bus-width = <0x8>;

What if we took this hint, and added bus-width = <0x4>; to the first one?. I compiled the dts file and rebooted.

Read speed: 19.9 MB/s

Write speed 8.8 MB/s

Yeah! I just tripled the performance of the uSD card. :-)

This works both when booted from the eMMC and when booted from the uSD card.

This was done on Angstrom "Linux beaglebone 3.8.13 #1 SMP Wed Sep 4 09:09:32 CEST 2013 armv7l GNU/Linux". Haven't tested it on Debian yet.

I don't know if this is the right way to do things or if it is just a dirty hack, and I don't know who to notify of my findings. Please assist.

[William Hermans]

Jens,

Nice find ! Would you mind doing a mini writeup ? Not quite sure exactly what you've done, but I would be willing to test on my end too as well.

[William Hermans]

I found . . .

&mmc2 {

        vmmc-supply = <&vmmcsd_fixed>;

        bus-width = <8>;

        ti,non-removable;

        status = "okay";

        reset = <&rstctl 0 0>;

        reset-names = "eMMC_RSTn-CONSUMER";

};

but nothing else in ./arch/arm/boot/dts/am335x-boneblack.dts. Is this what I am looking for ?

[Jens S.]

Sure, no problem.

(Please make a backup first. This worked for me, there are no guarantees that it will work for you. I haven't yet been able to confirm that this is the right way to solve the problem)

1. decompile the device tree:

dtc -I dtb -O dts -o test.dts /boot/am335x-boneblack.dtb

2. Edit the decompiled file:

nano test.dts

3.Search for the parts with the mmc controllers:

^w
mmc@48060000

4. You will see three mmc controllers:

               mmc@48060000 {
                        compatible = "ti,omap3-hsmmc";
                        ti,hwmods = "mmc1";
                        ti,dual-volt;
                        ti,needs-special-reset;
                        ti,needs-special-hs-handling;
                        dmas = <0x8 0x18 0x8 0x19>;
                        dma-names = "tx", "rx";
                        status = "okay";
                        pinctrl-names = "default";
                        pinctrl-0 = <0x9>;
                        cd-gpios = <0xa 0x6 0x0>;
                        cd-inverted;
                        vmmc-supply = <0xb>;
                        ti,vcc-aux-disable-is-sleep;
                        linux,phandle = <0x28>;
                        phandle = <0x28>;
                };

                mmc@481d8000 {
                        compatible = "ti,omap3-hsmmc";
                        ti,hwmods = "mmc2";
                        ti,needs-special-reset;
                        ti,needs-special-hs-handling;
                        dmas = <0x8 0x2 0x8 0x3>;
                        dma-names = "tx", "rx";
                        status = "disabled";
                        vmmc-supply = <0xb>;
                        bus-width = <0x8>;
                        ti,non-removable;
                        reset = <0xc 0x0 0x0>;
                        reset-names = "eMMC_RSTn-CONSUMER";
                        linux,phandle = <0x29>;
                        phandle = <0x29>;
                };

                mmc@47810000 {
                        compatible = "ti,omap3-hsmmc";
                        ti,hwmods = "mmc3";
                        ti,needs-special-reset;
                        ti,needs-special-hs-handling;
                        status = "disabled";
                        linux,phandle = <0x2a>;
                        phandle = <0x2a>;
                };

The first one is the uSD card ("mmc1"). The second one is the eMMC device and the third one is unused.

5. Add this line to the first one:

                        bus-width = <0x4>;

I don't know if the order of the lines is critical, but I added the line below the vmmc-supply line, just like the eMMC device.

The first device should now look like this:

                mmc@48060000 {
                        compatible = "ti,omap3-hsmmc";
                        ti,hwmods = "mmc1";
                        ti,dual-volt;
                        ti,needs-special-reset;
                        ti,needs-special-hs-handling;
                        dmas = <0x8 0x18 0x8 0x19>;
                        dma-names = "tx", "rx";
                        status = "okay";
                        pinctrl-names = "default";
                        pinctrl-0 = <0x9>;
                        cd-gpios = <0xa 0x6 0x0>;
                        cd-inverted;
                        vmmc-supply = <0xb>;
                        bus-width = <0x4>;
                        ti,vcc-aux-disable-is-sleep;
                        linux,phandle = <0x28>;
                        phandle = <0x28>;
                };

6. Save the file

^x
y
[enter]

7. compile the dts file:

dtc -I dts -O dtb -o /boot/am335x-boneblack.dtb test.dts

8. reboot

reboot

9. Enjoy :-)

YMMV. I got about three times better performance with a Sandisk Ultra card. If you have a slower card, you will not see the same improvement.

[Jens S.]

oh, and the above writeup was done on Angstrom.

[William Hermans]

Awesome thanks Jens. Exact steps ;)

Second post I just made was from a dts file I searched through using nano that came from instruction given by Robert Nelsons Linux on ARM ( debian 3.8.x ). Figured it was not right, but was about to search through a couple of includes . . .

[Robert Nelson]

Nice find..

/dev/mmcblk0:
Timing cached reads: 380 MB in 2.01 seconds = 189.30 MB/sec
Timing buffered disk reads: 56 MB in 3.00 seconds = 18.64 MB/sec

you can also just add the 4bit setting to the device dtb via:

diff --git a/arch/arm/boot/dts/am335x-bone.dts
b/arch/arm/boot/dts/am335x-bone.dts
index d5f43fe..aa1efb1 100644
--- a/arch/arm/boot/dts/am335x-bone.dts
+++ b/arch/arm/boot/dts/am335x-bone.dts
@@ -19,4 +19,5 @@
&mmc1 {
status = "okay";
vmmc-supply = <&ldo3_reg>;
+ bus-width = <0x4>;
};
diff --git a/arch/arm/boot/dts/am335x-boneblack.dts
b/arch/arm/boot/dts/am335x-boneblack.dts
index a6d7b0e..2a3f731 100644
--- a/arch/arm/boot/dts/am335x-boneblack.dts
+++ b/arch/arm/boot/dts/am335x-boneblack.dts
@@ -22,6 +22,7 @@

&mmc1 {
vmmc-supply = <&vmmcsd_fixed>;
+ bus-width = <0x4>;
};

&mmc2 {

[William Hermans]

Confirmed here too . . 

william@arm:~$ lsb_release -a | grep Description

No LSB modules are available.

Description:    Debian GNU/Linux 7.1 (wheezy)

william@arm:~$ uname -a

Linux arm 3.8.13-bone21 #1 SMP Sat Jun 15 19:36:33 MST 2013 armv7l GNU/Linux

william@arm:~$ sudo hdparm -tT /dev/mmcblk1

/dev/mmcblk1:

 Timing cached reads:   456 MB in  2.00 seconds = 227.80 MB/sec

 Timing buffered disk reads:  54 MB in  3.07 seconds =  17.56 MB/sec

However I did this slightly differently.

first I made a backup of the dtb on the beaglebone ( cp'd to ~/ on a reg user ) then on my cross system... 

william@support:~$ nano ~/debian/linux-dev/KERNEL/arch/arm/boot/dts/am335x-boneblack.dts

^W added bus-width <enter> and found . . .

&mmc1 {
  vmmc-supply = <&vmmcsd_fixed>;
 bus-width = <0x4>; //<--- added this
 };

saved out of nano.

william@support:~/debian/linux-dev/KERNEL$ dtc -I dts -O dtb -o ~/am335x-boneblack.dtb ./arch/arm/boot/dts/am335x-boneblack.dts

 

And I boot tftp / nfs on my BBB from this cross system, so all i had to do was move the file into the nfsroot/boot, and bam done.

Thanks again jens awesome find.

[Koen Kooi]

If you wants something added to the kernel, send a pull request against https://github.com/beagleboard/kernel/

For people wanting to get this speed boost, just do:

opkg update ; opkg install kernel-devicetrees ; reboot

That will install the new devicetree(s) and nothing else and reboot into the new, much faster system.

regards,

Koen

[Jens S.]

Thank you Koen. As far as I can tell, you did the pull request already, so no need for me to do it now, right? (sorry if it's a total noob question)

And thanks to those of you brave enough to confirm my finding.

[Koen Kooi]

Op 12 sep. 2013, om 12:54 heeft Jens S. <je...@sandalgaard.dk> het volgende geschreven:

> Thank you Koen. As far as I can tell, you did the pull request already, so no need for me to do it now, right? (sorry if it's a total noob question)

Right :)

And for future reference, you can inspect the mmc parameters using debugfs as well:

root@beaglebone-cam:~# cat /sys/kernel/debug/mmc1/ios
clock: 52000000 Hz
vdd: 7 (1.65 - 1.95 V)
bus mode: 2 (push-pull)
chip select: 0 (don't care)
power mode: 2 (on)
bus width: 3 (8 bits)
timing spec: 1 (mmc high-speed)
signal voltage: 0 (3.30 V)

root@beaglebone-cam:~# cat /sys/kernel/debug/mmc0/ios
clock: 50000000 Hz
vdd: 20 (3.2 ~ 3.3 V)
bus mode: 2 (push-pull)
chip select: 0 (don't care)
power mode: 2 (on)
bus width: 2 (4 bits)
timing spec: 2 (sd high-speed)
signal voltage: 0 (3.30 V)

I wonder if am335x can support the 100 and 200MHz modes in the UHS-I spec, that would be a neat speed boost.

regards,

Koen

>
> And thanks to those of you brave enough to confirm my finding.
>
> Den torsdag den 12. september 2013 11.48.17 UTC+2 skrev Koen Kooi:
> > I don't know if this is the right way to do things or if it is just a dirty hack, and I don't know who to notify of my findings. Please assist.
>
> If you wants something added to the kernel, send a pull request against https://github.com/beagleboard/kernel/
>
>

[Jens S.]

That debugfs command could have saved me from a huge learning experience ;-)

According to the Technical Reference Manual for the MPU, 24 MB/s is the theoretical maximum speed for a 4-bit connection.

Den torsdag den 12. september 2013 12.59.48 UTC+2 skrev Koen Kooi:

And for future reference, you can inspect the mmc parameters using debugfs as well:

[William Hermans]

D'oh ! Silly me, I was editing the wrong file, or rather I was putting it into the wrong place. tftp gotcha !

william@arm:~$ sudo hdparm -tT /dev/mmcblk0

/dev/mmcblk0:

 Timing cached reads:   482 MB in  2.00 seconds = 240.72 MB/sec

 Timing buffered disk reads:  54 MB in  3.05 seconds =  17.68 MB/sec

william@arm:~$ sudo hdparm -tT /dev/mmcblk1

/dev/mmcblk1:

 Timing cached reads:   402 MB in  2.00 seconds = 200.65 MB/sec

 Timing buffered disk reads:  54 MB in  3.08 seconds =  17.56 MB/sec

Got it right now though ! mmcblk0 is the sdcard.

[al...@indigenious.ro]

What would the equivalent package for Debian? There doesn't seem to be any Debian package called "kernel-devicetrees". 

[Robert Nelson]

On Sat, Jun 15, 2019 at 7:18 AM <al...@indigenious.ro> wrote:
>
> What would the equivalent package for Debian? There doesn't seem to be any Debian package called "kernel-devicetrees".

It's built into the linux-image-(uname -r) package..

Regards,

--
Robert Nelson
https://rcn-ee.com/