Maximising SD Card Life 24/7

[mm0fmf]

I've used Pi's since as soon as I could buy them but I've never run one
24/7 before so I'd like to ensure I've done sufficient to maximise SD
Card life.

All this PI does is sit accepting SSH connections and acts as a gateway
to allow tunnelling or SOCKS5 proxying. It also lets me send WOL magic
packets to other local computers to wake them up so I don't need to
leave those computers running 24/7. That's all working fine.

However, I'd like to maximise the SC Card life where possible so I have
mounted a few places onto tmpfs to minimise writes to the card. I have a
couple of cron jobs running that tickle the dynamic DNS provider and
another that reboots the Pi every 24 hours as a crude and simple way of
ensuring not the tmpfs log space doesn't fill.

The following are mounted onto tmpfs:

tmpfs on /tmp type tmpfs (rw,nosuid,noatime,size=102400k)
tmpfs on /var/spool/mqueue type tmpfs
(rw,nosuid,noatime,size=30720k,mode=700,gid=12)
tmpfs on /var/tmp type tmpfs (rw,nosuid,noatime,size=30720k)
tmpfs on /var/log type tmpfs (rw,nosuid,noatime,size=102400k,mode=755)

There is no swap running as all this does is SSH connection handling.

I have plenty of old small (8Gb) SD Cards to replace the one in use but
I don't want to have to be copying a new one more often than needed.

Is there anything else that really needs doing or is this sufficient?

[Jim Jackson]

It sounds like you usage is a fit for running with the root file system
readonly. If you make all the SD card filesystems readonly, then that
gets rid of the problems (except things like bad contacts etc :-( )

How to get to running a readonly root filesystem depends on your setup,
but I'm sure there is advice out there.

[Chris Elvidge]

If you can, put the root filesystem on a USB connected thumbdrive or SSD.


--
Chris Elvidge
England

[Craig Dooley]

BY: mm0fmf(3:770/3)


m> I've used Pi's since as soon as I could buy them but I've never run one
m> 24/7 before so I'd like to ensure I've done sufficient to maximise SD
m> Card life.
m>
Not sure what Pi you have, but if it's a 4, have you thought about going with
USB instead of SD card? I've currently got my BBS running on a RPi 4 with
everything on a USB drive. 128 GB flash drive the size of my thumbnail plugged
into one of the USB 3.0 ports. Works fine, and I don't have to worry about
read/write cycles.

-Craig

[mm0fmf]

I meant to say it's a Pi Zero W but forgot.

[Tauno Voipio]

I'm just curious: What makes a Flash memory on an USB interface
more durable than a Flash memory on a SD card?

--

-TV

[Jim Jackson]

It's flash whether the interface is SD or USB - there is a write limit!

[Craig Dooley]

BY: Tauno Voipio(3:770/3)


TV>
TV> I'm just curious: What makes a Flash memory on an USB interface
TV> more durable than a Flash memory on a SD card?
TV>
I'm not really sure. it's my understanding that SD cards will wear out because
of read/write cycles eventually. I've never heard that said of a USB
flashdrive. But I'm not an expert on either.

-Craig

[mm0fmf]

I don't know.

My day job is writing semiconductor models (think QEMU-like). We've
done loads of NAND, NOR, UFS, eMMC, SdCard/MMC device and controllers,
MobileStorage, NVMe models for our customers with varying levels of
realism. Our customers have never been keen to discuss wear levelling,
it's considered commercially confidential by them and isn't released
even under NDA.

[Ahem A Rivet's Shot]

On Sat, 20 Mar 2021 16:29:04 +0200
Tauno Voipio <tauno....@notused.fi.invalid> wrote:

> I'm just curious: What makes a Flash memory on an USB interface
> more durable than a Flash memory on a SD card?

Nothing in principle, except the quality and type of the flash
memory (there are a lot of types) and the on-board controller. All flash
memory (including SLC SSDs) are subject to write wear due to the mechanism
and write amplification due to large block sizes - how quickly they wear
depends on the flash technology used, the sophistication of the controller
(wear levelling, TRIM etc) and the quality of manufacture. Micro SD cards
tend to be towards the bottom end of everything - NVMe SSDs sold for data
centre use tend to be towards the top.

USB sticks are very variable IME but the good ones are a lot better
than any micro SD card.

--
Steve O'Hara-Smith | Directable Mirror Arrays
C:\>WIN | A better way to focus the sun
The computer obeys and wins. | licences available see
You lose and Bill collects. | http://www.sohara.org/

[Martin Gregorie]

IIRC the on-device controller on any type of SSD is a lot smarter than
the ones fitted to a built-down-to-price semi-disposable SD card, and one
place this shows up is in dealing with card wear, i.e EEPROM ageing
defects. I don't think any SD card can remove a bad memory page from the
active pages map and replace it with one of the spares - something most
SSDs can do - because for starters SD memory pages are relatively large
(4Kb?) and pricing says they don't have any list of spare pages or a
controller that could handle remapping.

I'm told that the better (enterprise grade) SSDs will drop into read-only
mode when they find a prefefined unfixable number of defects. This means
the data content can be retrieved without errors. Lower spec SSDs and all
SD cards will merrily go on writing data well past that point.

--
Martin | martin at
Gregorie | gregorie dot org

[Martin Gregorie]

SSDs wear out too - eventually - just the same as an HDD. The difference
is that SSDs data pages (a) survive many more read/zero/write cycles than
those in SD cards and (b) have a set of spare data pages that can be
remapped to replace failing ones. Cheap as chips SD cards don't do this
either.

[Nikolaj Lazic]

Dana Sat, 20 Mar 2021 11:42:38 +1300, Craig Dooley <nospam.Cr...@f126.n123.z1.fidonet.org> napis'o:

I've got few of USB flash drives that died. Or got read only.

[Folderol]

I haven't got a singe failed one, and my oldest (still in use) is a massive
128M - from the days when that was the biggest you could get!

I particularly like that it has a mechanical write protect switch and slim
but tough case. I can't tell you the make - all the writing has long since worn
away!

--
W J G

[mm0fmf]

On 20/03/2021 11:35, Jim Jackson wrote:
> It sounds like you usage is a fit for running with the root file system
> readonly

Thanks. I found an article from 2016 about making root read only FS for
a Pi. I'll give that a test run tomorrow and see how it goes.

[Computer Nerd Kev]

The "PiCore" version of Tiny Core Linux runs on the Pi with the core
system entirely in RAM. It should be possible to have that boot from
a read-only partition, though it might be easiest to remount it
read-only after booting to avoid modifying the boot scripts.

By remastering PiCore with all your required software "built-in" it
would be possible to run everything in RAM and unmount the SD card
entirely after boot.

--
__ __
#_ < |\| |< _#

[nev young]

I wouldn't worry about it.
I have 10 Pi zeros working 24/7 as CCTV cameras.
Each day they record between 1-5GB of new images recovering space by
removing images that are a few days old.
The SD cards (32GB) wear out every 3 or 4 years.

--
Nev
It causes me a great deal of regret and remorse
that so many people are unable to understand what I write.

[David Taylor]

Andy, it's possible that you're worrying needlessly.

I have 25 RPi cards here, some since the RPi was first released - and
(touch wood) SD card wear-out hasn't been an issue. Many are doing very
little, and those supporting the Pi-Star software (DMR hotspots) do have
the system running mostly read-on (that's built into the Pi-Star
software). I'm not using any programs which drive the system into paging.

For one that's doing APT weather satellite reception and regularly
creating image data I do have a 128 GB USB stick for storing that data,
specifically to reduce wear on the SD card.
The only other thing I do these days is to buy as big an SD card as I
can. That's 32 GB now (I don't know whether the RPi or the PC I use for
imaging would take 64 GB).

I suspect that unless you are repeatedly writing a "lot" of data, or
needing to page, there's no need to worry.

73,
David GM8ARV
--
Cheers,
David
Web: http://www.satsignal.eu

[Mike]

In article <slrns5cnrb.q1l....@mudrac.ffzg.hr>,

Me too. (Me, Two!)

Used on RPI as "larger storage", both went read-only in action.

The first one did not survive a reboot, and became a
Teflon drive. A non-stick. Not a USB memory device at all. It just
went away.

The other was still readable to recover from, on a PC, but was
discarded after!

Currently on 2nd SD card and 3rd USB stick.

There is no magic to USB drives that means "they don't wear out".
--
--------------------------------------+------------------------------------
Mike Brown: mjb[-at-]signal11.org.uk | http://www.signal11.org.uk

[mm0fmf]

Another interesting solution to try. I have been using various flavours
of Debian since 2003 so Raspbian on the Pi just feels the same. But this
again sounds like it's worth doing.

[mm0fmf]

I can live with that lifespan :-) I didn't want to end up having to
write a new one out every 3-4 months.

[druck]

On 20/03/2021 11:22, mm0fmf wrote:
> The following are mounted onto tmpfs:
>
> tmpfs on /tmp type tmpfs (rw,nosuid,noatime,size=102400k)
> tmpfs on /var/spool/mqueue type tmpfs
> (rw,nosuid,noatime,size=30720k,mode=700,gid=12)
> tmpfs on /var/tmp type tmpfs (rw,nosuid,noatime,size=30720k)
> tmpfs on /var/log type tmpfs (rw,nosuid,noatime,size=102400k,mode=755)

The problem with /var/log on /tmp is you have no clue as to what has
gone wrong when there is a problem and you have to reboot. You can set
up syslog to log to another computer, but this wont help if problems are
due to network issues.

---druck

[druck]

On 20/03/2021 14:29, Tauno Voipio wrote:

> I'm just curious: What makes a Flash memory on an USB interface
> more durable than a Flash memory on a SD card?

There are two extremes:-

SD card - cheap flash with basic controller aimed at storing large files
on a FAT32 filing system for use with cameras and misc players. Not good
for being a general purposing filing system for a computer.

SSD - good quality flash with a sophisticated controller, aimed at being
a general purpose read/write filing system for a computer, to replace a
hard disc. Has the best performance and long levity.

USB sticks are somewhere in between. The cheap and low profile ones are
often no better than SD cards when used in a Pi. The more expensive USB
3.1 and larger bar format may have a better flash and controller which
are closer in performance and long levity to a low end SSD.

My experience that an SD card when used in a very active Pi, can last as
little a year, but more often 2-3. A Samsung USB 3.1 bar, is much better
and I've only had one fail after 4 years. The SSDs I'm using including a
couple which had previously been used in a PC haven't had any failures
in over 6 years.

---druck

[Theo]

Tauno Voipio <tauno....@notused.fi.invalid> wrote:
> I'm just curious: What makes a Flash memory on an USB interface
> more durable than a Flash memory on a SD card?

Space. A microSD card is very very space constrained. It typically has a
stack of flash dice (eg 32 dice) bonded to a controller, that all has to fit
in the micro SD form factor, with very tight thickness constraints.
Additionally, the form factor limits its heat dissipation. Typically the
controller doesn't have much local RAM, which is used for caching and
wear levelling.

A USB device isn't limited by the microSD form factor or heat budget, so
there's a lot more potential for using proper controllers sometimes with
dedicated RAM chips. For example the Sandisk Extreme series (at least
the ones I've looked at) contains a full SATA SSD controller, with a SATA to
USB bridge chip on the front. That's a lot more to fit into the package
than just slapping a simple flash controller onto a stack of NAND dice.

Theo

[The Natural Philosopher]

This is very true, but you can always revert to using the SD card if
there is a repeatable issue



--
“The ultimate result of shielding men from the effects of folly is to
fill the world with fools.”

Herbert Spencer

[Computer Nerd Kev]

David Taylor <david-...@blueyonder.co.uk.invalid> wrote:
> The only other thing I do these days is to buy as big an SD card as I
> can. That's 32 GB now (I don't know whether the RPi or the PC I use for
> imaging would take 64 GB).

So far as the Pi goes there shouldn't be any worries up to 256GB,
and higher for the Pi 3 and later.

https://www.raspberrypi.org/documentation/installation/sd-cards.md

[David Taylor]

Thanks!

[John Aldridge]

There is something stange with SD cards...

I've had one machine running pretty much 24/7 for over 5 years without
any problems at all. I've had several others (which haven't had heavy
use or been allowed to get full) fail much more quickly than that, just
a few weeks in one case. The failing cards have not all been the same
brand (Samsung and Sandisk, at least), and weren't bought from Ebay!

The symptom has pretty much always been the same: the SD card becomes
read only, but you don't notice -- sometimes for weeks -- because Linux
just keeps any changed data cached in memory. It's only when you
eventually reboot the machine that the contents of the card revert back
to the last time it was working properly.

Maybe it's something I'm doing to upset them, but I don't know what!

John

In article <s36r6v$93m$1...@dont-email.me>, david-
tay...@blueyonder.co.uk.invalid says...

[Ahem A Rivet's Shot]

On Mon, 22 Mar 2021 19:54:04 -0000
John Aldridge <jp...@cantab.net> wrote:

<about dying SD cards>

> Maybe it's something I'm doing to upset them, but I don't know what!

Bad power would be my first guess.

[John Aldridge]

In article <20210322203636.a7e1...@eircom.net>,
ste...@eircom.net says...

>
> On Mon, 22 Mar 2021 19:54:04 -0000
> John Aldridge <jp...@cantab.net> wrote:
>
> <about dying SD cards>
>
> > Maybe it's something I'm doing to upset them, but I don't know what!
>
> Bad power would be my first guess.

They suffer from the odd power cut, but then the good one has too. The
good one's an RPi2, I guess that may be significant?

--
John

[Martin Gregorie]

On Mon, 22 Mar 2021 23:35:31 +0000, John Aldridge wrote:

>
> They suffer from the odd power cut, but then the good one has too. The
> good one's an RPi2, I guess that may be significant?
>

More like luck. Its no different to yanking the batteries out of a
running PDA/PNA/phone:

- 95%+ of the time you get away with it because the card is either not
being accessed or being read.

- however, if the Pi is writing to the SD card when the power is cut,
you'll get data corruption or physical damage - just data corruption if
the card was being written to, but if wear levelling was in progress
when the power was cut the card can become unusable.

And yes, I have seen the card in a PNA get corrupted when a mate pulled
the card out while it was still shutting down - just as I'd said 'don't
to that!' to him.

[Jan Novak]

Am 21.03.21 um 00:40 schrieb nev young:

> I have 10 Pi zeros working 24/7 as CCTV cameras.

Hi,

can you describe your Hardware and OS / daemon from that project?


Jan

[nev young]

Yes.

As at Mar 2021 they are:
# PiHardware, Camera, SD
1 Pi zero W, LifeCam HD-3000, 32Gb
2 Pi zero W, LifeCam Cinema, 32Gb
3 Pi zero W, LifeCam HD-3000, 32Gb
4 Pi zero, Pi NOIR cam, 32Gb
5 Pi zero W, LifeCam HD-3000, 32Gb
6 Pi zero W, LifeCam Cinema, 32Gb
7 Pi zero W, LifeCam HD-3000, 32Gb
8 Pi zero W, Pi NOIR cam, 32Gb
9 Pi zero W, GEMBIRD 8Gb
10 Pi zero, PiHut ZeroCam 32Gb

All running headless latest Raspbian Lite GNU/Linux 10 (buster) O/S and
motion 4.1.1 capture software. I run apt to update/upgrade once a month.
Each camera runs a cron job to purge old images every night and a cron
job rsyncs the image directories to a pi file server overnight over wifi.

Cameras 4 & 8 run using IR lights.

You may get more from http://www.nevilley.org.uk/cameras/cameras.php
although this needs updating. There have been changes since Dec 2019
mainly re-locating webcams and rebuilding the SD cards with Rasbian
Buster (from stretch) in Aug 2020. Most of the SDs were replaced as I
just burned a batch of new ones and then changed them en-masse. I bought
20 32Gb SanDisk Ultra microSDHD UHS-1 cards off Amazon.