Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Doh! Raid10 in the dark ...
3 views
Skip to first unread message
Adrian Caspersz
Feb 16, 2024, 7:37:23 AM2/16/24
to
I have weekly scheduled backups and have successfully restored, I am in
a good place - now need to grow back some hair.
Lesson learned for a cheapskate Dell R620 home lab environment.
Do not set up a Raid 10 system (stripe + mirror) and forget to keep an
eye on monitoring the raid [1] for errors
- OR specifically,
forget to keep an *eye on the monitoring platform* for functioning
itself [2]
I had two drives grow bad sectors, so when I came to pull and replace
one of them, found the other bad disk was mirroring in the same stripe
group.
Hmmm...An uncorrectable array, that I could have avoided had I known and
replaced the first drive earlier[3].
So downtime[4], check all disks and restore all VMs from backup :(
Ho hmmmm... :)
1 - Especially when using cheapo £9 dubious 10 year old 900GB SAS drives
from eBay.
2 - Out of the box, Proxmox configures mail alerts via SMTP -
unfortunately that outgoing is blocked courtesy of my ISP/Spamhaus, so a
GMail workaround now implemented.
3 - Hmmm, time to look at configuring a "hot spare" hard drive.
4 - Yeah, not critical though. I'm lucky I don't do much infrastructure
work like this for a job!
--
Adrian C
a good place - now need to grow back some hair.
Lesson learned for a cheapskate Dell R620 home lab environment.
Do not set up a Raid 10 system (stripe + mirror) and forget to keep an
eye on monitoring the raid [1] for errors
- OR specifically,
forget to keep an *eye on the monitoring platform* for functioning
itself [2]
I had two drives grow bad sectors, so when I came to pull and replace
one of them, found the other bad disk was mirroring in the same stripe
group.
Hmmm...An uncorrectable array, that I could have avoided had I known and
replaced the first drive earlier[3].
So downtime[4], check all disks and restore all VMs from backup :(
Ho hmmmm... :)
1 - Especially when using cheapo £9 dubious 10 year old 900GB SAS drives
from eBay.
2 - Out of the box, Proxmox configures mail alerts via SMTP -
unfortunately that outgoing is blocked courtesy of my ISP/Spamhaus, so a
GMail workaround now implemented.
3 - Hmmm, time to look at configuring a "hot spare" hard drive.
4 - Yeah, not critical though. I'm lucky I don't do much infrastructure
work like this for a job!
--
Adrian C
Andy Burns
Feb 16, 2024, 8:18:43 AM2/16/24
to
Adrian Caspersz wrote:
> forget to keep an eye on the monitoring platform for functioning
> itself [2]
Don't PERC cards have a fault LED on the chassis?
> forget to keep an eye on the monitoring platform for functioning
> itself [2]
Don't PERC cards have a fault LED on the chassis?
Jeff Gaines
Feb 16, 2024, 8:32:23 AM2/16/24
to
On 16/02/2024 in message <l39380...@mid.individual.net> Adrian
wonder if I should use them as 4 x individual drives - is that JBOD?
--
Jeff Gaines Dorset UK
We chose to do this not because it is easy but because we thought it would
be easy.
Caspersz wrote:
>Do not set up a Raid 10 system (stripe + mirror) and forget to keep an eye
>on monitoring the raid [1] for errors
My NAS has 4 x 2 TB SSD in RAID 10. In fact I only have 1.2 TB of data and
>Do not set up a Raid 10 system (stripe + mirror) and forget to keep an eye
>on monitoring the raid [1] for errors
wonder if I should use them as 4 x individual drives - is that JBOD?
--
Jeff Gaines Dorset UK
We chose to do this not because it is easy but because we thought it would
be easy.
Adrian Caspersz
Feb 16, 2024, 9:10:21 AM2/16/24
to
into degraded state.
Instead both the drives that were mirrored were screaming SMART pre-fail
messages that I could not hear.
Otherwise the RAID & drives were working fine silently correcting bad
reads until I pulled out one of the drives, then PERC announced the bad
blocks closely followed by a Proxmox VM backup hang.
--
Adrian C
0 new messages