RE: [EXTERNAL] Re: Industrial Beagle Bone Black damages SD cards.
![Steffensen, Flemming [COMRES/SOL/AAR]'s profile photo](https://lh3.googleusercontent.com/a/default-user=s40-c)
Steffensen, Flemming [COMRES/SOL/AAR]
Thank you for your answer and time, Jason, it’s really appreciated.
Per your request, I’m also adding beagl...@googlegroups.com to the list of recipients.
The failing SD-cards are for the most part the consumer
SanDisk Ultra A1 SDHC 16gb, but we also have a few Kingston industrial 8gb (SDCIT/8GB B0623-004.A00LFTS).
They are bought through official distributors of the respective brands, and not likely to be fake, although we have not performed tests anywhere near as detailed as the one you link to.
Talking with the distributer, I’m told that they internally have 0.08 % failure rate on the consumer SanDisk, and less than 0.00 % of the industrial Kingston.
The datasheet for either card makes no promise in this regard, except 5 and 10 year warranty. I’ve asked for further details, if possible.
We have already planned to make the root filesystem read-only (and later move it to the eMMC). The ‘overlayroot’ suggested by Robert Nelson may come in handy here.
However, will this solve the issue?
Afterall, what we experience is not filesystem corruption, which is what we would have expected, but that the entire card becomes unreadable/unrecognized.
It looks to us that the problem can occur to the card at different points in time:
- During normal usage of the card, where the cards properties are cached in RAM. (Invisible corruption)
- During the system shutdown and restart, where there are no power-fluctuations.
- During the power starvation that occur after a correct shutdown, when the power to the BBB is cut.
- During the power starvation that occur due to accidental external power-loss.
- During the bootloader at startup.
We always discover the problem at 5, following either 4, 3, or 2. But the problem may actually occur at 1.
In regards to 1, are there any means to monitor whether the CID/CSD or other “embedded” SD registers change under normal usage (reading/writing to the filesystem) of the card?
Do you have other ideas on how we can dig deeper than just the error message from dmesg?
Best regards,
--
Flemming Steffensen | Senior Software Engineer | Transportation Solutions
Emerson Commercial & Residential Solutions | Axel Kiers Vej 5A | Højbjerg | DK-8270 | Denmark
T +45 7023 4444 | D +45 8987 3710 | M +45 4094 3793
Flemming....@Emerson.com
Twitter | Facebook | LinkedIn | YouTube | Climate Conversations Blog
This e-mail message is intended by Emerson Commercial & Residential Solutions, for use only by the individual or entity to which it is addressed. This message may contain information that is protected, privileged or confidential. It is not intended for transmission to, or receipt by, anyone other than the named addressee (or a person authorized to receive and deliver it to the named addressee). If you have received this transmission in error, please delete it from your system without copying or forwarding it, and notify the sender of the error by reply e-mail or by calling +45 7023 4444
From: Jason Kridner <jkri...@beagleboard.org>
Sent: 5. april 2019 20:29
To: Steffensen, Flemming [COMRES/SOL/AAR] <Flemming....@Emerson.com>
Cc: Urup, Mark [EXTERNAL/CONTRACTOR] <Mark...@Emerson.com>; Ranum, Morten [COMRES/SOL/AAR] <Morten...@Emerson.com>; Robert Nelson <robert...@beagleboard.org>; Christine Kridner <chr...@beagleboard.org>
Subject: [EXTERNAL] Re: Industrial Beagle Bone Black damages SD cards.
On Fri, Apr 5, 2019 at 5:17 AM Steffensen, Flemming [COMRES/SOL/AAR] <Flemming....@emerson.com> wrote:
Hi Jason,
I’m addressing you directly, as this is very specifically an error that it’s not likely the community can help with.
You could be surprised. In any respects, I like to answer all follow-up questions with the community list (beagl...@googlegroups.com) in copy such that we all benefit from the answers.
We are working on a product that incorporates the Industrial version of the BeagleBone Black: element14 BeagleBone Black Industrial, BBONE-BLACK-IND-4G.
We are using a slightly modified version of you IoT Debian 9.3, to run our application, and to access our custom cape (The mandatory I2C chip, custom power-supply, a few basic IO’s and serial ports).
Everything is working as expected… Our app is running, and logging a few things to a file on the SD card.
On rare occasions we cold-boot, either due to external power-loss, or more normally on a regular shutdown followed by a correct power-off.
This typically works flawless, but on a few occasions, it’s no longer possible to boot from the SD card.
As we have the hard set up as R/W, we understand that we might damage the filesystem, but this appear to not be the case here.
Examining the SD card (several brands, some industrial others not) we find that they are utterly unrecognized by everything. The cards internal registers, such as the CID, CSD and the OCR appear to be blank. At this point the SD card is dead!☹
SD cards are notoriously unreliable, which is a big part of why we ship with on-board flash. The "fake" market seems to be very entrenched (https://www.bunniestudios.com/blog/?p=2297 [bunniestudios.com]).
What are you finding in regards to industrial rated cards?
If you have a supplier, maybe we can approach them together to ask for some failure analysis as well as quality control? The firmware on the cards is meant to handle a lot of conditions. Clearly they didn't handle one properly.
Windows refuses to even see the cards, while Linux produces this sort of errors if the card is inserted:
# dmesg
[10028.659056] mmc0: unrecognised CSD structure version 3
[10028.659069] mmc0: error -22 whilst initialising SD card
We have never experience the problem on the regular BBB (which we used during most of the development time), only the industrial version, so we suspect there’s a difference between the two somewhere.
Do you have any idea what may cause this error, and how we can avoid it?
Over the last 10 years, I've probably "burnt-up" 3 or 4 microSD cards and never really knew what happened to them.
We still have some of the dead SD-cards, but since it contains our software we will not be able to send it to you for debugging, unless some sort of NDA agreement can be made between our companies.
We are looking forward to hear back from you.
Best regards.,
--
Flemming Steffensen | Senior Software Engineer | Transportation Solutions
Emerson Commercial & Residential Solutions | Axel Kiers Vej 5A | Højbjerg | DK-8270 | Denmark
T +45 7023 4444 | D +45 8987 3710 | M +45 4094 3793
Twitter [twitter.com] | Facebook [facebook.com] | LinkedIn [linkedin.com] | YouTube [youtube.com] | Climate Conversations Blog [emersonclimateconversations.com]
This e-mail message is intended by Emerson Commercial & Residential Solutions, for use only by the individual or entity to which it is addressed. This message may contain information that is protected, privileged or confidential. It is not intended for transmission to, or receipt by, anyone other than the named addressee (or a person authorized to receive and deliver it to the named addressee). If you have received this transmission in error, please delete it from your system without copying or forwarding it, and notify the sender of the error by reply e-mail or by calling +45 7023 4444
--
https://beagleboard.org/about [beagleboard.org] - a 501c3 non-profit educating around open hardware computing
Graham
I happened to run across this series of articles yesterday.
Dr. Michael J. Chudobiak
The failing SD-cards are for the most part the consumer SanDisk Ultra A1 SDHC 16gb, but we also have a few Kingston industrial 8gb (SDCIT/8GB B0623-004.A00LFTS).
They are bought through official distributors of the respective brands, and not likely to be fake, although we have not performed tests anywhere near as detailed as the one you link to.
Talking with the distributer, I’m told that they internally have 0.08 % failure rate on the consumer SanDisk, and less than 0.00 % of the industrial Kingston.
The datasheet for either card makes no promise in this regard, except 5 and 10 year warranty. I’ve asked for further details, if possible.
I had the same problems until switching to SwissBit SLC SD cards, which are supposed to have aggressive power-up/power-down protection circuits. No problems since. Very expensive, but cheaper than in-field failures.
- Mike