weewx, raspberry pi and sd cards - the lifetime for card

[Jacek Skowroński]

Hi,

How does Your sd card "live" in raspberry pi?

On my example I lost 5 cards already - all of them by 3-4months long.

Is it normal?

Yust now I'm looking to get rid of sd cards and write down data somewhere else.

Any topics? Any hints?

[Andrew Milner]

I just use the sd card to boot from and then use an attached usb hard drive.  Many others have configured a ram disk and use the ram disk as much as possible - with backups to sd only as and when required so that the sd card is not hammered continuously.  SD cards are really designed for data storage not for repeated read/write operations, and certainly not for 24/7 read/write..

[Jacek Skowroński]

any hint/tutorial how to manage usb HDD?

[gjr80]

Hi again,

As usual Google is your friend - search for 'boot raspberry Pi from USB disk' and you will find a plethora of replies, many of which have tutorials or links to tutorials.

Gary

[Jacek Skowroński]

Have You tried and it work? Could You recommend any certain HDD model and usb-hdd adapter model? Does it need external power connector?

[Andrew Milner]

Yes, they work.  I use a powered usb hub because the rpi usb is somewhat flakey - and my fineoffset weather station is also going to the rpi via a powered usb hub.

[Jacek Skowroński]

great, I will update my environment with hdd.

Regards
Jacek

W dniu 2016-10-31 o 12:09, Andrew Milner pisze:

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/atRpRzKrlyk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Keffer]

I've been running an experiment on how long an SD card can survive in the weewx environment. 

So far, using a Sandisk Extreme Plus, it's been running nearly 2 years without a problem.

If you buy a good one, it can last quite a long time.

What kind of card are you using?

-tk

To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.

[Jacek Skowroński]

I had 5 already

toshiba claas 10
samsung evo pro
sandisk extreme
sandisk ultra
Intenso claas 10

in general all get corrupted after 3-4 month

all cards had been 16-32gb

W dniu 2016-10-31 o 13:34, Thomas Keffer pisze:

To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[gjr80]

Just remember, if you are only using your RPi for weewx and say hosting your weather site you don't need a huge HDD - gigabytes not terrabytes is fine. Remember the powered hub and follow the instructions carefully, you may or may not understand them but if you find some that are clearly laid out chnaces are they will work. If not find another set and try again.

Gary

[Daniel Rich]

Jacek,

When you say you are seeing "corruption", what do you mean? In my experience SD card failures typically show as write or read failures, but not data corruption.

Like Thomas, I have PI systems that have been running for over two years with no SD card failures. As long as you don't do something like enable a swap partition and run so much on the PI that you cause swapping, there shouldn't be enough data activity to hit any of the SD write limits for several years.

---
Dan Rich <dr...@employees.org> |   http://www.employees.org/~drich/
                               |  "Step up to red alert!"  "Are you sure, sir?
                               |   It means changing the bulb in the sign..."
                               |          - Red Dwarf (BBC)

To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/atRpRzKrlyk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[Jacek Skowroński]

I mean crash of the rasbperry Pi

and many errors like kernel panic connected with the filesystem

Few times I had tried to get card to the PC with knoppix and do fsck to repair filesystem but all trials make the data corrupted -> I gave up and bougth new card and system was working for some time.

At the begining I thought it's a case of power / stability of power.

For about 6months I have PiUPS and APC UPS, APC is mainly for making the power more electric user friendly and to keep power for 5minutes. PiUPS is for keeping power for 30minutes and then to clearly poweroff. From this time - I have 100% power on, but on Saturday second card failed. I get mail from WU about no data from station, I attached tv to the Pi and I had seen kernel panic because of filesystem.

This is my problem.

W dniu 2016-10-31 o 16:19, Daniel Rich pisze:

[Les Niles]

The SD card on my first weewx installation died after about 8 months.  I didn’t want to add rotating magnetic storage (HDD), for reasons of engineering aesthetics, cost, and power consumption.  Instead, I added a pair of USB flash drives, configured as a linux RAID-1 array and mounted as /var.  Since almost all of the writes — log files, the weewx database, and the rendered web pages — are in /var, the SD card which hosts the rest of the filesystem should last a long time.  The USB flash drives probably won’t last because they’re the same basic technology as SD cards, but when one dies it can be replaced and the RAID array rebuilt, without shutting down weewx and without losing any data.  That’s the plan anyway; so far it’s been running 13 months without any failures.

Any storage device has the potential to fail.  Even an HDD is likely to fail within the timespan I want my weather station operating.  A RAID array largely solves this issue because double failures is much lower.  And once I settled on RAID, the reliability of USB flash for the underlying storage was plenty good enough.

  -Les

[Macha]

Hi,

In the early days with my Pi farm, I lost* a few SD cards.

I then moved on to keeping my / file system on a usb flash drive, this was a bit better, but I still lost* a couple of those.

I then tried putting / on my NFS server. This worked fine, except that when I had too many (4+) RPis using this arrangement, crashes (recoverable) would become annoyingly frequent.

I've also done things like stopping swap space and so on.

My current arrangement (I have around 10 RPis) is a mix. I have a few using / on my NFS, a few "SD card only" and a few where frequent writes (images etc) are stored on the NFS as a mount point. With this, everything ticks along nicely and I haven't lost a card for a few years.

I also use "raspibackup" to image the SD cards once a week.

I have to say that I have a bit of an aversion to using powered USB hubs - it's a personal  preference!

Cheers

* "lost" means completely unreadable/unwriteable

[Glenn McKechnie]

One way to lock up your SDCard is outlined at rorpi - read only raspberry pi + weewx: now with scripts

Making the / filesystem 100% read-only should go some way to ensuring the SDCard will stay around longer. While it mentions using USB sticks and hdd drives as writable alternatives, the responses in this thread regarding those will be more useful to you. (FWIW, I find the RAID 1 USB stick setup intriguing!)

Interesting that you've lost so many cards, and have eliminated your power supply as a problem.

How busy is your system? If you run iotop (apt-get install iotop) what does it show?

Are you using mysql? It's a lot busier when it comes to disk access.

Also, are you only generating the one skin on the pi? If you have multiple skins (testing them out or whatever) the image files tend to add up ( I bricked an 8gb Sandisk Cruzer doing this ), especially at archive intervals less than 5 minutes.

Cheers
 Glenn

rorpi - read only raspberry pi + weewx: now with scripts

[Jacek Skowroński]

iotop shows nothing special from time to time 50KB/S write but total is always 0.

I use sqlite version - and the file weewx.sdb was located on external USB Flash

I have default skin, I send data to 4 weathers services and ftping those to my web server - located in external DC.

So in my point of view it's crazy!!!

One of my friend told me that the problem could be reading data from /dev/ttyUSB0 and putting logs on sd card

W dniu 2016-11-01 o 00:53, Glenn McKechnie pisze:

[Andrew Milner]

..... so why not also put your logs onto the flash?  Don't see why reading usb should impact the sd card though ....

[Glenn McKechnie]

run iotop as  iotop -oaP  This will show running process's only and accumulated totals - run it for a while and you'll get a better feel of what's happening as regards total writes.

sqlite write total will be included under the python /usr/bin/weewx total - sqlite is a friendlier option for the SDCard

/dev/ttyUSB0 should be dealt with in memory, no disk access involved

logging will be visible under iotop as rsyslogd -n (or whatever daemon you use).  Turns out, it's not as big a burden as it's made out to be; at least on the systems here.

I wouldn't have thought ftp would add any burden - the files have been generated already and it's just reading them.

As you say, it is a bit odd.

Cheers
 Glenn

rorpi - read only raspberry pi + weewx: now with scripts

To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.

[Jacek Skowroński]

Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s

Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/s

  PID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND

   87 be/3 root          0.00 B    136.00 K  0.00 %  0.03 % [jbd2/mmcblk0p2-]

14302 be/4 root          0.00 B    704.00 K  0.00 %  0.00 % python /usr/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf

12435 be/4 pi            0.00 B     24.00 K  0.00 %  0.00 % vmstat 60 1440

  495 be/4 root          0.00 B     16.00 K  0.00 %  0.00 % rsyslogd -n

To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[Glenn McKechnie]

I didn't recognize vmstat, a quick search turns up the following...

"vmstat 60 1440 > /path/to/vmstat.`date +%d` This will write to a file every minute for 24 hours then create a new file and start over."

That might be a good candidate to move over to the flash, along with the logs as Andrew suggested.  In fact, it's writing more than the logging daemon so you've just doubled the logging load there.

[jbd2/mmcblk0p2-] is the ext4 journal - I always find it interesting to see just how large that value gets over time. Make the SDcard read-only and that drops to zero.

Keep it running and see what else shows up, anything that pops up there is a possible candidate to move to the USBstick or hdd.

The webserver (the image / html files generated by weewx are included in weewx's iotop total) can also be moved over to the flash /USB stick, hdd, whatever.
The rorpi HowTo moves the webserver into memory (except for NOAA files - It's been pointed out that if there are a lot of them, they are worth keeping on the drive rather than regenerating.) I haven't thought of a valid reason to keep the www files on disk, they are regenerated constantly. Although there will be an empty page after a weewx  restart, but a placeholder could be kept to inform any one unlucky enough to be viewing in that small time window.

Cheers
 Glenn

rorpi - read only raspberry pi + weewx: now with scripts

On 1 November 2016 at 19:39, Jacek Skowroński <yac...@gmail.com> wrote:

Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s

Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/s

  PID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND

   87 be/3 root          0.00 B    136.00 K  0.00 %  0.03 % [jbd2/mmcblk0p2-]

14302 be/4 root          0.00 B    704.00 K  0.00 %  0.00 % python /usr/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf

12435 be/4 pi            0.00 B     24.00 K  0.00 %  0.00 % vmstat 60 1440

  495 be/4 root          0.00 B     16.00 K  0.00 %  0.00 % rsyslogd -n

On Tuesday, November 1, 2016 at 9:28:58 AM UTC+1, Glenn McKechnie wrote:

run iotop as  iotop -oaP  This will show running process's only and accumulated totals - run it for a while and you'll get a better feel of what's happening as regards total writes.

sqlite write total will be included under the python /usr/bin/weewx total - sqlite is a friendlier option for the SDCard

/dev/ttyUSB0 should be dealt with in memory, no disk access involved

logging will be visible under iotop as rsyslogd -n (or whatever daemon you use).  Turns out, it's not as big a burden as it's made out to be; at least on the systems here.

I wouldn't have thought ftp would add any burden - the files have been generated already and it's just reading them.

As you say, it is a bit odd.

 

[...]

[Jacek Skowroński]

vmstat is from 2 days to see when and what is going on.

I need to talk to my friend who moved files from sd to flash. Because he did it for me.

Regards

Jacek

W dniu 2016-11-01 o 10:01, Glenn McKechnie pisze:

[Jacek Skowroński]

after one hour I get

14302 be/4 root          0.00 B     11.70 M  0.00 %  0.01 % python /usr/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf

does it mean that weewx pid write down 11.7? or which file produced so huge amount of data?

Maybe I should put it in the ram or on the flash

[Glenn McKechnie]

Hi Jacek,

On 1 November 2016 at 20:58, Jacek Skowroński <yac...@gmail.com> wrote:

after one hour I get

14302 be/4 root          0.00 B     11.70 M  0.00 %  0.01 % python /usr/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf

does it mean that weewx pid write down 11.7? or which file produced so huge amount of data?

 

Interesting that you have no DISK READ value, I'd have thought there would be something in that field?

Yes, that's what it has written to disk, although 11.7M doesn't sound excessive for weewx. As I noted previously, that value is for everything that weewx is handling; sqlite database writes, webserver pages and images. It also includes writes to tmp, although thats probably a tmpfs so it's removed from the disk already. Anyway, that means that it's not one file in particular that's attracting all that traffic.

Maybe I should put it in the ram or on the flash

That's your call, if you're asking me then yes :-) One size doesn't fit all, but you can choose a method that you're happy with and can handle.
Me?  I'd be attempting to preserve the SDCard for its main purpose, running the OS and keeping the pi alive. The data that weewx generates is moved elsewhere, preferably somewhere or something with a proven track record. In particular something that matches its value ie: the database gets priority storage, the webserver not so much.

 

[Jacek Skowroński]

Could You provide some structure how to divide data between ram and flash?

On flash i'd like to have

DB

Config

Skins

Rest could be in ram

Www files

Logs

Any other tempoprary weewx files

--

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/atRpRzKrlyk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

[Andrew Milner]

Have you read his detailed wiki??  It is all explained/described in there .....

[Daniel Rich]

To be honest, that sounds like a bad Pi, not a bad SD card. The Pi is fairly tolerant of power issues, I have run them off of USB cables connected to other systems, USB wall power adaptors, and even POE adaptors with never a problem like the one you are seeing. 

The only thing that comes immediately to mind is to make sure you don't have swap enabled in your system. In general, you never want to run anything on a Pi that uses enough memory to cause swapping -- both for performance and SD write issues. The kind of writes that weewx does for the sqlite database and logging shouldn't cause enough write traffic to hit the SD write limits for several years (there is a really good article from a couple of years ago on the myth of SD write limitations, I'll see if I can dig it up and post it to the list).

In all my years of using SD cards in various devices (PoGoPlug, Pi, an old firewall I have since retired, assorted tablets, etc.), I have only ever had a single card failure -- and that was from a bad card, not any sort of write limit being reached.

[Jacek Skowroński]

It's my secind pi first was pi 1b the secind is pi 3

To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

[Andrew Milner]

Out of curiosity - do you

a)run software other than debian, weewx, SQLite and maybe a webserver on your pi?

b)do you regularly - eg once a month - keep your rpi sw up-to-date with update/upgrade?

c)do you regularly keep other software you use up-to-date?

d)do you make sure that you do not run experimental sw on your 'production' pi, but only on a backup pi - in case issues are raised during the experiment .....

e)do you run fsck routinely?

f)have you tried repairing the sd card by booting into single user - or touching to force fsck on boot?

g)do you always - 100% of the time - do orderly powerdowns and in the event of any unexpected power failures routinely do full fsck and repairs?

Some of these questions are relevant in that, for example, the effects of a disorderly loss of power may not be seen until some time after the event - when the disk sector(s) in question are next accessed.  Software which crashes can corrupt a disk before crashing - and if that SW is not used (because it crashed) the effect can go unnoticed for some considerable time .... and so on and so forth .....

There's no need to respond to this post - it is more intended to just make sure that you are thinking of these things - since I seem to recall that you're not, for example, on the latest weewx - and seemed to have difficulties working out a simple template change to add soilTemp1 to a webpage, so I am not sure of your level of expertise in computers/Linux/RPi/weewx/etc .....

Just trying to be helpful and give you some 'food for thought' .....  as I said, no need to respond!!

Andrew

[Jacek Skowroński]

answering inline

W dniu 2016-11-01 o 16:10, Andrew Milner pisze:

Out of curiosity - do you

a)run software other than debian, weewx, SQLite and maybe a webserver on your pi?

no, the pi is dedicated for weewx, so this is a core. Of course I have additional small programs like htop, bwm-ng and other just to monitor the pi.

b)do you regularly - eg once a month - keep your rpi sw up-to-date with update/upgrade?

yes, I do

c)do you regularly keep other software you use up-to-date?

no, because there no software that is outside system update.

d)do you make sure that you do not run experimental sw on your 'production' pi, but only on a backup pi - in case issues are raised during the experiment .....

hard to say what is experimental, I did not provide any bigger chages to the rasbian so I think there is nothing special

e)do you run fsck routinely?

no, I do it only in case of error

f)have you tried repairing the sd card by booting into single user - or touching to force fsck on boot?

2 times, I booted lapton with knoppix, put the card in reader and tried to repair, after few hours of fight - theoretically card was repaired but when I inserted it and booted system from it, the weewex was corrupted

g)do you always - 100% of the time - do orderly powerdowns and in the event of any unexpected power failures routinely do full fsck and repairs?

having pi UPS, it's nearly impossible to have unexpected power failure. This setup is also connected to the APC UPS - to make power better for electric devices.

Some of these questions are relevant in that, for example, the effects of a disorderly loss of power may not be seen until some time after the event - when the disk sector(s) in question are next accessed.  Software which crashes can corrupt a disk before crashing - and if that SW is not used (because it crashed) the effect can go unnoticed for some considerable time .... and so on and so forth .....

First crash of card was done thanks to power failrure, so I invested in APC UPS and then Pi UPS

There's no need to respond to this post - it is more intended to just make sure that you are thinking of these things - since I seem to recall that you're not, for example, on the latest weewx - and seemed to have difficulties working out a simple template change to add soilTemp1 to a webpage, so I am not sure of your level of expertise in computers/Linux/RPi/weewx/etc .....

In this case what I can say - I'm little bit lazy but from the other side. I would not open opened door. If somebody has already template with soil temp, what for fight with the problem if I could use somebody else solution? If it's a problem I do not care.

Just trying to be helpful and give you some 'food for thought' .....  as I said, no need to respond!!

Andrew

Thanks for help
regards
Jacek

[Les Niles]

Hmm… have you looked at how much those monitoring programs hammer the storage? I’m not familiar with the ones you mention, but from the man page it looks like bwm-ng in particular could be writing pretty frequently. (Looks like it generates static HTML pages, with a default update rate of 2/sec!)  

BTW, in the interest of full disclosure: when my SD card died, for several months I had been storing all the loop packets to the database, meaning weewx was writing to the card several times per minute.  I’d tried to put those loop packets in an in-memory database but mysql “helpfully” moved it to disk storage and I got lazy and didn’t fix that.  

  -Les

[Jacek Skowroński]

answering inline

W dniu 2016-11-01 o 16:10, Andrew Milner pisze:

Out of curiosity - do you

a)run software other than debian, weewx, SQLite and maybe a webserver on your pi?

no, the pi is dedicated for weewx, so this is a core. Of course I have additional small programs like htop, bwm-ng and other just to monitor the pi.

b)do you regularly - eg once a month - keep your rpi sw up-to-date with update/upgrade?

yes, I do

c)do you regularly keep other software you use up-to-date?

no, because there no software that is outside system update.

d)do you make sure that you do not run experimental sw on your 'production' pi, but only on a backup pi - in case issues are raised during the experiment .....

hard to say what is experimental, I did not provide any bigger chages to the rasbian so I think there is nothing special

e)do you run fsck routinely?

no, I do it only in case of error

f)have you tried repairing the sd card by booting into single user - or touching to force fsck on boot?

2 times, I booted lapton with knoppix, put the card in reader and tried to repair, after few hours of fight - theoretically card was repaired but when I inserted it and booted system from it, the weewex was corrupted

g)do you always - 100% of the time - do orderly powerdowns and in the event of any unexpected power failures routinely do full fsck and repairs?

having pi UPS, it's nearly impossible to have unexpected power failure. This setup is also connected to the APC UPS - to make power better for electric devices.

Some of these questions are relevant in that, for example, the effects of a disorderly loss of power may not be seen until some time after the event - when the disk sector(s) in question are next accessed.  Software which crashes can corrupt a disk before crashing - and if that SW is not used (because it crashed) the effect can go unnoticed for some considerable time .... and so on and so forth .....

First crash of card was done thanks to power failrure, so I invested in APC UPS and then Pi UPS

There's no need to respond to this post - it is more intended to just make sure that you are thinking of these things - since I seem to recall that you're not, for example, on the latest weewx - and seemed to have difficulties working out a simple template change to add soilTemp1 to a webpage, so I am not sure of your level of expertise in computers/Linux/RPi/weewx/etc .....

In this case what I can say - I'm little bit lazy but from the other side. I would not open open

Just trying to be helpful and give you some 'food for thought' .....  as I said, no need to respond!!

Andrew

[Eelco F]

Well, I experienced the same issues running the system on a raspberry pi B. Every 3 Months or so, the sd cards just gave up. Corrupted files and file system, causing freezes. I had the whole system including swap, raspbian and weewx and Apache running on the sd card. No extra disks or sticks.
I ended up doing some calculation: weewx rewrites its database every 5 min, that is 12 times/h, 248 times/day, let's say 7500/month.
Apart from that it rewrites html files, images etc. at the same frequency. Also the operating system performs writes, in the log for example, and swap.
Adding this all up together, I think an sd card just wears out, after a few months.
I replaced the raspberry by a secondhand HP thin client, for a few bucks at eBay. With a regular hdd.
It takes more power of course, but it's a lot faster too. It's running Ubuntu server, and is stable until now.

[Jacek Skowroński]

I am in the middle of chcanging sd to hdd. Of success i will keep rpi if fail i will go exactly same solution old PC like hp or dell

[Jacek Skowroński]

Rpi 3 is fully working with usb hdd so i got rid of sd card

To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

[pon...@gmail.com]

Quick comment as someone who uses a good number of SD cards (a dozen or more at a time in daily use for the last 4-5 years).

I have a number of trailcams that I use for nocturnal wildlife surveillance and swap SDcards on a daily basis on from 2 to 7 cameras. In addition I use them for other things -- a Sony RX100 camera, Raspberry Pis (incl. weewx), Linux boot devices etc.

I have found the quality of some brands to be first class (no failures yet) and some almost guaranteed to fail within a few months. In my experience the best cards I've used are SanDisk SDHC Class 4 8Gb and the worst, by a long way, Transcend HC1 16Gb. In fairness, perhaps, I have to say that I wondered if the Transcends were either fake or just a bad batch; only 1 of the last batch of 10 is still in use after about a year. Some devices seem harder on cards than others. My Samsung Chromebook is so bad that I no longer even try to use SD cards with it.

The advice to use a ram drive is good. I'm still using an original SD card with my first Raspberry Pi 2+ years on. Database updates are logged to disk every 5 mins. Log files are written to my Synology NAS. Web page updates go to ram. I can't really see the need to install a hard disk. I've had outages for power cuts and WH1080 USB port lockups, nothing else.

Bottom line (your mileage may vary): get a premium brand SD card and avoid using it as much as possible. 

[fraban]

Hi all

we solved the problem with raspberries and SD Cards with writings most things to "RAMDISK" /  tmp Files.  So RAM Drive is the solution for the problem. 

And we use 32GB or 16GB Cards Class 10 ( SAMUNG - no "NONAMES" ! ) 

We run all our raspberries - with the following entries in /etc/fstab -  So we reduced the writings on the SD Card to a minimum .  

Since we use these settings our raspberries are OK and run now more then 6, 8 and 12 month without problems 24h a day. 

Regards Frank

pi@raspberrypi ~ $ more /etc/fstab

proc            /proc           proc    defaults          0       0

/dev/mmcblk0p1  /boot           vfat    defaults          0       2

/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1

# a swapfile is not a swap partition, no line here

#   use  dphys-swapfile swap[on|off]  for that

# For Debian Jessie - RAM Drive using weewx

tmpfs           /tmp            tmpfs   defaults,nosuid,mode=0755,nodev,noatime   0       0

tmpfs           /var/log        tmpfs   defaults,nosuid,mode=0755,nodev,noatime   0       0

tmpfs           /var/tmp        tmpfs   defaults,nosuid,mode=0755,nodev,noatime   0       0

tmpfs /var/www/weewx  tmpfs   defaults,nosuid,mode=0755,nodev,noatime         0       0

tmpfs           /var/log/nginx  tmpfs   defaults,nosuid,mode=0755,nodev,noatime         0       0

tmpfs           /var/run        tmpfs   defaults,nosuid,mode=0755,noatime               0       0

[vince]

On Monday, November 7, 2016 at 3:48:11 AM UTC-8, fraban wrote:

we solved the problem with raspberries and SD Cards with writings most things to "RAMDISK" /  tmp Files.  So RAM Drive is the solution for the problem.

 

# For Debian Jessie - RAM Drive using weewx

tmpfs           /tmp            tmpfs   defaults,nosuid,mode=0755,nodev,noatime   0       0

tmpfs           /var/log        tmpfs   defaults,nosuid,mode=0755,nodev,noatime   0       0

tmpfs           /var/tmp        tmpfs   defaults,nosuid,mode=0755,nodev,noatime   0       0

tmpfs           /var/run        tmpfs   defaults,nosuid,mode=0755,noatime               0       0

hmmm - this is really interesting, you can stick syslogs and /tmp and /var/tmp in tmpfs just by editing fstab ?  Cool.

I did this in a vagrant vm and it's making systemd rather unhappy, although the system seems to run ok:

• initial login after bootup is 'very' slow
• systemctl reports us...@MYUID.service failed
• systemctl reports systemd-update-utmp.service failed
• systemctl reports 'Failed to start System Logging Service' every 90 secs or so
• systemctl reports watchdog timeout for systemd.logind
• systemctl reports watchdog timeout for systemd.journald

I did a little more research with some controlled reboots and found that systemd really doesn't like /var/run in tmpfs.

• not putting /var/run into tmpfs made the problems above go away, and 'systemctl' is clean

Does a 'systemctl' command on your system show a couple failed processes ?

tmpfs /var/www/weewx  tmpfs   defaults,nosuid,mode=0755,nodev,noatime         0       0

tmpfs           /var/log/nginx  tmpfs   defaults,nosuid,mode=0755,nodev,noatime         0       0

• for the weewx directory, your NOAA reports will disappear every time you reboot, so wouldn't weewx (very slowly) regenerate them every reboot ?
• I didn't notice where your archive directories are located.  Do you copy the old one back into place via rc.local or something on every reboot ?

Cool idea putting that stuff in tmpfs rather than hard-allocating a ramdisk and maybe not using it...that'll give me more free space on the pi that does my weathercam.  Thanks !

[Frank Bandle]

Hi Vince

• for the weewx directory, your NOAA reports will disappear every time you reboot, so wouldn't weewx (very slowly) regenerate them every reboot ?
• I didn't notice where your archive directories are located.  Do you copy the old one back into place via rc.local or something on every reboot ?

We did not use NOAA reports on these raspberries. The raspberries we use only collect data and send to our servers. 

Archive - the weewx.db is still on the SD Card in standard path  /var/lib/weewx 

we run jessie on the raspberries with no problem running this fstab. 

Maybe /var/run to tmpfs causes problems on other systems - here no problems 

Does a 'systemctl' command on your system show a couple failed processes ?

No :-) 

- frank

-- 
Dipl. Met. Frank Bandle
Sent with Airmail

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/atRpRzKrlyk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[vince]

On Monday, November 7, 2016 at 8:47:51 AM UTC-8, fraban wrote:

Hi Vince

• for the weewx directory, your NOAA reports will disappear every time you reboot, so wouldn't weewx (very slowly) regenerate them every reboot ?

we run jessie on the raspberries with no problem running this fstab. 

Maybe /var/run to tmpfs causes problems on other systems - here no problems

hmmm - definitely has problems under Vagrant using the debian/jessie64 image from the Debian project.

All I did was manually paste your tmpfs lines into fstab and reboot.
Commenting out the /var/run line and rebooting fixes all issues.
Odd.

Thanks again for the tmpfs example. Pretty cool.

[Les Niles]

On my RPi installs of jessie and wheezy, /var/run is just a symlink to /run, and /run is a tmpfs filesystem — it’s created by an init script, not mounted out of fstab, but it is tmpfs.  

  -Les

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[vince]

On Monday, November 7, 2016 at 11:50:31 AM UTC-8, ln77 wrote:

On my RPi installs of jessie and wheezy, /var/run is just a symlink to /run, and /run is a tmpfs filesystem — it’s created by an init script, not mounted out of fstab, but it is tmpfs.  

ummm - yup it is, I forgot to look at that.  Thanks.
And it's already tmpfs, as you mentioned (wheezy 8.6).

[Chris Thompstone]

In case anyone might be interested, my setup:

Odroid u3 (now discontinued, but other better ones now available)
Sqlite
Debian wheezy running on the eMMC card (maybe eMMC better life span than sd?)
using ramlog to store all logs into ram, I think it maybe writing occasionally or before reboots.
the weewx website is stored on tmpfs (ram) so no writes here.
The only writes to storage I make is to the sqlite database every 5 mins.
Obviously automated backups are made elsewhere of the database.

This has run happily for over 2 years so far.
No problems touch wood.

[Maj Hek]

Hi,

I use ramdisk on folder where web pages are made and also storing data on external sql server. So far it has been up and running for 5 months.

BR Magnus

[Jerry Kutche]

I Use only Samsung 32GB Evo + Nothing else ever I have had some in service for years running 24/7