Raid 1 Degraded

[John G]

Hi,

A few days ago the lights on the front of my DNS-323 changed to amber.  On the WebUI, I see the raid is degraded.  However, on the Disk->RAID page there isn't anything in the maintenance section.  Both disks are indicated healthy, but I think the issue is on sdb2 in the right slot.  Let me know what logs might help diagnose this.  What can I do to fix this?

Thanks

[John G]

Here is the system log:

System Log

May 29 12:27:12 DNS syslog.info syslogd started: BusyBox v1.20.2
May 29 12:27:12 DNS user.notice kernel: klogd started: BusyBox v1.20.2 (2013-03-17 19:50:30 WET)
May 29 12:27:12 DNS user.notice kernel: Linux version 2.6.35.14 (jcard@silver) (gcc version 4.3.3 (GCC) ) #1 Sun Mar 17 19:51:52 WET 2013
May 29 12:27:12 DNS user.warn kernel: CPU: Feroceon [41069260] revision 0 (ARMv5TEJ), cr=a0053177
May 29 12:27:12 DNS user.warn kernel: CPU: VIVT data cache, VIVT instruction cache
May 29 12:27:12 DNS user.warn kernel: Machine: D-Link DNS-321/323
May 29 12:27:12 DNS user.warn kernel: Clearing invalid memory bank 0KB@0xffffffff
May 29 12:27:12 DNS user.warn kernel: Ignoring unrecognised tag 0x00000000
May 29 12:27:12 DNS user.warn kernel: Ignoring unrecognised tag 0x41000403
May 29 12:27:12 DNS user.warn kernel: Memory policy: ECC disabled, Data cache writeback
May 29 12:27:12 DNS user.notice root: Starting syslogd: OK. Starting klogd: OK.
May 29 12:27:12 DNS user.debug kernel: On node 0 totalpages: 16384
May 29 12:27:12 DNS user.debug kernel: free_area_init_node: node 0, pgdat c0381bd0, node_mem_map c0397000
May 29 12:27:12 DNS user.debug kernel:   Normal zone: 128 pages used for memmap
May 29 12:27:12 DNS user.debug kernel:   Normal zone: 0 pages reserved
May 29 12:27:12 DNS user.debug kernel:   Normal zone: 16256 pages, LIFO batch:3
May 29 12:27:12 DNS user.warn kernel: Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
May 29 12:27:12 DNS user.notice kernel: Kernel command line: root=/dev/ram console=ttyS0,115200 :::DB88FXX81:egiga0:none console=ttyS0,115200 root=/dev/ram0 init=/init
May 29 12:27:12 DNS user.info kernel: PID hash table entries: 256 (order: -2, 1024 bytes)
May 29 12:27:12 DNS user.info kernel: Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
May 29 12:27:12 DNS user.info kernel: Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
May 29 12:27:12 DNS user.info kernel: Memory: 64MB = 64MB total
May 29 12:27:12 DNS user.notice kernel: Memory: 55064k/55064k available, 10472k reserved, 0K highmem
May 29 12:27:12 DNS user.notice kernel: Virtual kernel memory layout:
May 29 12:27:12 DNS user.notice kernel:     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
May 29 12:27:12 DNS user.notice kernel:     fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
May 29 12:27:12 DNS user.notice kernel:     DMA     : 0xffc00000 - 0xffe00000   (   2 MB)
May 29 12:27:12 DNS user.notice kernel:     vmalloc : 0xc4800000 - 0xfd800000   ( 912 MB)
May 29 12:27:12 DNS user.notice kernel:     lowmem  : 0xc0000000 - 0xc4000000   (  64 MB)
May 29 12:27:12 DNS user.notice kernel:     modules : 0xbf000000 - 0xc0000000   (  16 MB)
May 29 12:27:12 DNS user.notice kernel:       .init : 0xc0008000 - 0xc0029000   ( 132 kB)
May 29 12:27:12 DNS user.notice kernel:       .text : 0xc0029000 - 0xc0350000   (3228 kB)
May 29 12:27:12 DNS user.notice kernel:       .data : 0xc036a000 - 0xc03821e0   (  97 kB)
May 29 12:27:12 DNS user.info kernel: NR_IRQS:64
May 29 12:27:12 DNS user.info kernel: Calibrating delay loop... 332.59 BogoMIPS (lpj=1662976)
May 29 12:27:12 DNS user.info kernel: pid_max: default: 4096 minimum: 301
May 29 12:27:12 DNS user.warn kernel: Mount-cache hash table entries: 512
May 29 12:27:12 DNS user.info kernel: CPU: Testing write buffer coherency: ok
May 29 12:27:12 DNS user.info kernel: NET: Registered protocol family 16
May 29 12:27:12 DNS user.info kernel: Orion ID: MV88F5182-A2. TCLK=166666667.
May 29 12:27:12 DNS user.info kernel: DNS-321/323: Identified HW revision B1
May 29 12:27:12 DNS user.warn kernel: DNS-321/323: Found ethernet MAC address: 00:24:01:9d:09:30.
May 29 12:27:12 DNS user.warn kernel: bio: create slab  at 0
May 29 12:27:12 DNS user.notice kernel: SCSI subsystem initialized
May 29 12:27:12 DNS user.debug kernel: libata version 3.00 loaded.
May 29 12:27:12 DNS user.info kernel: usbcore: registered new interface driver usbfs
May 29 12:27:12 DNS user.info kernel: usbcore: registered new interface driver hub
May 29 12:27:12 DNS user.info kernel: usbcore: registered new device driver usb
May 29 12:27:12 DNS user.info kernel: Switching to clocksource orion_clocksource
May 29 12:27:12 DNS user.info kernel: NET: Registered protocol family 2
May 29 12:27:12 DNS user.info kernel: IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
May 29 12:27:12 DNS user.info kernel: TCP established hash table entries: 2048 (order: 2, 16384 bytes)
May 29 12:27:12 DNS user.info kernel: TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
May 29 12:27:12 DNS user.info kernel: TCP: Hash tables configured (established 2048 bind 2048)
May 29 12:27:12 DNS user.info kernel: TCP reno registered
May 29 12:27:12 DNS user.info kernel: NET: Registered protocol family 1
May 29 12:27:12 DNS user.info kernel: RPC: Registered udp transport module.
May 29 12:27:12 DNS user.info kernel: RPC: Registered tcp transport module.
May 29 12:27:12 DNS user.info kernel: RPC: Registered tcp NFSv4.1 backchannel transport module.
May 29 12:27:12 DNS user.debug kernel: PCI: CLS 0 bytes, default 32
May 29 12:27:12 DNS user.info kernel: Trying to unpack rootfs image as initramfs...
May 29 12:27:12 DNS user.info kernel: rootfs image is not initramfs (junk in compressed archive); looks like an initrd
May 29 12:27:12 DNS user.info kernel: Freeing initrd memory: 6236K
May 29 12:27:12 DNS user.notice kernel: VFS: Disk quotas dquot_6.5.2
May 29 12:27:12 DNS user.warn kernel: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
May 29 12:27:12 DNS user.info kernel: squashfs: version 4.0 (2009/01/31) Phillip Lougher
May 29 12:27:12 DNS user.info kernel: Installing knfsd (copyright (C) 1996 [redacted]).
May 29 12:27:12 DNS user.info kernel: aufs 2-standalone.tree-35-20100809
May 29 12:27:12 DNS user.info kernel: msgmni has been set to 119
May 29 12:27:12 DNS user.info kernel: io scheduler noop registered
May 29 12:27:12 DNS user.info kernel: io scheduler deadline registered
May 29 12:27:12 DNS user.info kernel: io scheduler cfq registered (default)
May 29 12:27:12 DNS user.info kernel: Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
May 29 12:27:12 DNS user.info kernel: serial8250.0: ttyS0 at MMIO 0xf1012000 (irq = 3) is a 16550A
May 29 12:27:12 DNS user.info kernel: console [ttyS0] enabled
May 29 12:27:12 DNS user.info kernel: brd: module loaded
May 29 12:27:12 DNS user.info kernel: loop: module loaded
May 29 12:27:12 DNS user.info kernel: sata_mv sata_mv.0: version 1.28
May 29 12:27:12 DNS user.info kernel: sata_mv sata_mv.0: slots 32 ports 2
May 29 12:27:12 DNS user.info kernel: scsi0 : sata_mv
May 29 12:27:12 DNS user.info kernel: scsi1 : sata_mv
May 29 12:27:12 DNS user.info kernel: ata1: SATA max UDMA/133 irq 29
May 29 12:27:12 DNS user.info kernel: ata2: SATA max UDMA/133 irq 29
May 29 12:27:12 DNS user.notice kernel: physmap platform flash device: 00800000 at f4000000
May 29 12:27:12 DNS user.info kernel: physmap-flash.0: Found 1 x16 devices at 0x0 in 8-bit bank
May 29 12:27:12 DNS user.info kernel: Amd/Fujitsu Extended Query Table at 0x0040
May 29 12:27:12 DNS user.notice kernel: number of CFI chips: 1
May 29 12:27:12 DNS user.notice kernel: cmdlinepart partition parsing not available
May 29 12:27:12 DNS user.notice kernel: RedBoot partition parsing not available
May 29 12:27:12 DNS user.notice kernel: Using physmap partition information
May 29 12:27:12 DNS user.notice kernel: Creating 5 MTD partitions on "physmap-flash.0":
May 29 12:27:12 DNS user.notice kernel: 0x000000000000-0x000000010000 : "MTD1"
May 29 12:27:12 DNS user.notice kernel: 0x000000010000-0x000000020000 : "MTD2"
May 29 12:27:12 DNS user.notice kernel: 0x000000020000-0x0000001a0000 : "Linux Kernel"
May 29 12:27:12 DNS user.notice kernel: 0x0000001a0000-0x0000007d0000 : "File System"
May 29 12:27:12 DNS user.notice kernel: 0x0000007d0000-0x000000800000 : "u-boot"
May 29 12:27:12 DNS user.notice kernel: MV-643xx 10/100/1000 ethernet driver version 1.4
May 29 12:27:12 DNS user.info kernel: mv643xx_eth smi: probed
May 29 12:27:12 DNS user.notice kernel: net eth0: port 0 with MAC address 00:24:01:9d:09:30
May 29 12:27:12 DNS user.info kernel: ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
May 29 12:27:12 DNS user.info kernel: orion-ehci orion-ehci.0: Marvell Orion EHCI
May 29 12:27:12 DNS user.info kernel: orion-ehci orion-ehci.0: new USB bus registered, assigned bus number 1
May 29 12:27:12 DNS user.info kernel: orion-ehci orion-ehci.0: irq 17, io mem 0xf1050000
May 29 12:27:12 DNS user.info kernel: orion-ehci orion-ehci.0: USB 2.0 started, EHCI 1.00
May 29 12:27:12 DNS user.info kernel: hub 1-0:1.0: USB hub found
May 29 12:27:12 DNS user.info kernel: hub 1-0:1.0: 1 port detected
May 29 12:27:12 DNS user.info kernel: Initializing USB Mass Storage driver...
May 29 12:27:12 DNS user.info kernel: usbcore: registered new interface driver usb-storage
May 29 12:27:12 DNS user.info kernel: USB Mass Storage support registered.
May 29 12:27:12 DNS user.info kernel: input: gpio-keys as /class/input/input0
May 29 12:27:12 DNS user.info kernel: i2c /dev entries driver
May 29 12:27:12 DNS user.info kernel: rtc-m41t80 0-0068: chip found, driver version 0.05
May 29 12:27:12 DNS user.info kernel: rtc-m41t80 0-0068: rtc core: registered m41t80 as rtc0
May 29 12:27:12 DNS user.info kernel: lm75 0-0048: hwmon1: sensor 'lm75'
May 29 12:27:12 DNS user.info kernel: md: linear personality registered for level -1
May 29 12:27:12 DNS user.info kernel: md: raid1 personality registered for level 1
May 29 12:27:12 DNS user.debug kernel: Registered led device: power:blue
May 29 12:27:12 DNS user.debug kernel: Registered led device: right:amber
May 29 12:27:12 DNS user.debug kernel: Registered led device: left:amber
May 29 12:27:12 DNS user.info kernel: usbcore: registered new interface driver usbhid
May 29 12:27:12 DNS user.info kernel: usbhid: USB HID core driver
May 29 12:27:12 DNS user.info kernel: TCP cubic registered
May 29 12:27:12 DNS user.info kernel: NET: Registered protocol family 17
May 29 12:27:12 DNS user.info kernel: rtc-m41t80 0-0068: setting system clock to 2018-05-29 19:26:54 UTC (1527622014)
May 29 12:27:12 DNS user.info kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
May 29 12:27:12 DNS user.info kernel: ata1.00: ATA-9: WDC WD30EZRX-00SPEB0, 80.00A80, max UDMA/133
May 29 12:27:12 DNS user.info kernel: ata1.00: 5860533168 sectors, multi 0: LBA48 NCQ (depth 31/32)
May 29 12:27:12 DNS user.info kernel: ata1.00: configured for UDMA/133
May 29 12:27:12 DNS user.notice kernel: scsi 0:0:0:0: Direct-Access     ATA      WDC WD30EZRX-00S 80.0 PQ: 0 ANSI: 5
May 29 12:27:12 DNS user.notice kernel: sd 0:0:0:0: Attached scsi generic sg0 type 0
May 29 12:27:12 DNS user.notice kernel: sd 0:0:0:0: [sda] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
May 29 12:27:12 DNS user.notice kernel: sd 0:0:0:0: [sda] 4096-byte physical blocks
May 29 12:27:12 DNS user.notice kernel: sd 0:0:0:0: [sda] Write Protect is off
May 29 12:27:12 DNS user.debug kernel: sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
May 29 12:27:12 DNS user.notice root: Starting sslcert: OK.
May 29 12:27:12 DNS user.notice kernel: sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
May 29 12:27:12 DNS user.info kernel:  sda: sda1 sda2
May 29 12:27:12 DNS user.notice kernel: sd 0:0:0:0: [sda] Attached SCSI disk
May 29 12:27:12 DNS user.info kernel: ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
May 29 12:27:12 DNS user.info kernel: ata2.00: ATA-9: WDC WD30EZRX-00D8PB0, 80.00A80, max UDMA/133
May 29 12:27:12 DNS user.info kernel: ata2.00: 5860533168 sectors, multi 0: LBA48 NCQ (depth 31/32)
May 29 12:27:12 DNS user.info kernel: ata2.00: configured for UDMA/133
May 29 12:27:12 DNS user.notice kernel: scsi 1:0:0:0: Direct-Access     ATA      WDC WD30EZRX-00D 80.0 PQ: 0 ANSI: 5
May 29 12:27:12 DNS user.notice kernel: sd 1:0:0:0: Attached scsi generic sg1 type 0
May 29 12:27:12 DNS user.notice kernel: sd 1:0:0:0: [sdb] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
May 29 12:27:12 DNS user.notice kernel: sd 1:0:0:0: [sdb] 4096-byte physical blocks
May 29 12:27:12 DNS user.notice kernel: sd 1:0:0:0: [sdb] Write Protect is off
May 29 12:27:12 DNS user.debug kernel: sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
May 29 12:27:12 DNS user.notice kernel: sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
May 29 12:27:12 DNS user.info kernel:  sdb: sdb1 sdb2
May 29 12:27:12 DNS user.notice kernel: sd 1:0:0:0: [sdb] Attached SCSI disk
May 29 12:27:12 DNS user.info kernel: md: Skipping autodetection of RAID arrays. (raid=autodetect will force)
May 29 12:27:12 DNS user.notice kernel: RAMDISK: squashfs filesystem found at block 0
May 29 12:27:12 DNS user.notice kernel: RAMDISK: Loading 6233KiB [1 disk] into ram disk... |^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H
May 29 12:27:12 DNS user.warn kernel: VFS: Mounted root (squashfs filesystem) readonly on device 1:0.
May 29 12:27:12 DNS user.info kernel: Freeing init memory: 132K
May 29 12:27:12 DNS user.info kernel: Adding 524284k swap on /dev/sda1.  Priority:1 extents:1 across:524284k 
May 29 12:27:12 DNS user.info kernel: md: bind
May 29 12:27:12 DNS user.info kernel: Adding 524284k swap on /dev/sdb1.  Priority:1 extents:1 across:524284k 
May 29 12:27:12 DNS user.info kernel: md: bind
May 29 12:27:12 DNS user.warn kernel: md: kicking non-fresh sda2 from array!
May 29 12:27:12 DNS user.info kernel: md: unbind
May 29 12:27:12 DNS user.info kernel: md: export_rdev(sda2)
May 29 12:27:12 DNS user.info kernel: md/raid1:md0: active with 1 out of 2 mirrors
May 29 12:27:12 DNS user.info kernel: md0: bitmap initialized from disk: read 2/2 pages, set 44445 bits
May 29 12:27:12 DNS user.info kernel: created bitmap (22 pages) for device md0
May 29 12:27:12 DNS user.info kernel: md0: detected capacity change from 0 to 3000053874688
May 29 12:27:12 DNS user.info kernel:  md0: unknown partition table
May 29 12:27:12 DNS user.notice hot: Finish fscking md0: fsck 1.41.14 (22-Dec-2010) /dev/md0: clean, 289448/183115776 files, 643250722/732435028 blocks
May 29 12:27:12 DNS user.notice root: Starting urandom: OK.
May 29 12:27:13 DNS daemon.info sysctrl: Starting
May 29 12:27:13 DNS daemon.info sysctrl: reading /etc/sysctrl.conf
May 29 12:27:13 DNS daemon.info sysctrl: cant open /etc/sysctrl.conf: No such file or directory Using defaults
May 29 12:27:13 DNS daemon.info sysctrl: args.lo_fan=2000
May 29 12:27:13 DNS daemon.info sysctrl: args.hi_fan=5000
May 29 12:27:13 DNS daemon.info sysctrl: args.lo_temp=40
May 29 12:27:13 DNS daemon.info sysctrl: args.hi_temp=50
May 29 12:27:13 DNS daemon.info sysctrl: args.mail=1
May 29 12:27:13 DNS daemon.info sysctrl: args.recovery=1
May 29 12:27:13 DNS daemon.info sysctrl: args.fan_off_temp=38
May 29 12:27:13 DNS user.notice root: Starting sysctrl: OK.
May 29 12:27:13 DNS daemon.info sysctrl: args.max_fan_speed=5500
May 29 12:27:13 DNS daemon.info sysctrl: args.crit_temp=54
May 29 12:27:13 DNS daemon.info sysctrl: args.warn_temp=52
May 29 12:27:13 DNS daemon.info sysctrl: args.crit_temp_command="/sbin/poweroff"
May 29 12:27:13 DNS daemon.info sysctrl: args.warn_temp_command="(null)"
May 29 12:27:13 DNS daemon.info sysctrl: args.front_button_command1="(null)"
May 29 12:27:13 DNS daemon.info sysctrl: args.front_button_command2="(null)"
May 29 12:27:13 DNS daemon.info sysctrl: args.back_button_command="(null)"
May 29 12:27:13 DNS daemon.info sysctrl: md0: state=clean level=raid1 degraded=1 action=idle
May 29 12:27:13 DNS daemon.info sysctrl: right_dev disk (sda) wakeup
May 29 12:27:13 DNS daemon.info sysctrl: left_dev disk (sdb) wakeup
May 29 12:27:13 DNS user.info kernel: usbcore: registered new interface driver usblp
May 29 12:27:14 DNS user.info kernel: EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
May 29 12:27:14 DNS user.notice hot: Users directory found in md0
May 29 12:27:14 DNS user.notice root: Starting inetd: OK.
May 29 12:27:14 DNS user.notice hot: Public directory found in md0
May 29 12:27:16 DNS user.info kernel: eth0: link up, 1000 Mb/s, full duplex, flow control disabled
May 29 12:27:21 DNS user.notice root: Starting nmbd: OK. Starting smbd: OK.
May 29 12:27:21 DNS user.notice root: Starting ffp: No ffp instalation found.
May 29 12:27:24 DNS user.notice root: user stopped.
May 29 12:27:24 DNS user.notice root: ffp stopped
May 29 12:27:24 DNS user.notice root: vsftpd stopped
May 29 12:27:24 DNS user.notice root: dropbear stopped
May 29 12:27:24 DNS user.notice root: smbd running
May 29 12:27:24 DNS user.notice root: rpc.statd stopped
May 29 12:27:24 DNS user.notice root: nfsd stopped
May 29 12:27:24 DNS user.notice root: inadyn stopped
May 29 12:27:24 DNS user.notice root: ntpd stopped
May 29 12:27:25 DNS user.notice root: dnsmasq stopped
May 29 12:27:25 DNS user.notice root: stunnel stopped
May 29 12:27:25 DNS user.notice root: inetd running
May 29 12:27:25 DNS user.notice root: portmap stopped
May 29 12:27:25 DNS user.notice root: cleanup stopped
May 29 12:27:25 DNS user.notice root: backup stopped
May 29 12:27:25 DNS user.notice root: mdadm stopped
May 29 12:27:25 DNS user.notice root: smartd stopped
May 29 12:27:25 DNS user.notice root: atd stopped
May 29 12:27:25 DNS user.notice root: crond stopped
May 29 12:27:25 DNS user.notice root: sysctrl running
May 29 12:27:25 DNS user.notice root: urandom stopped
May 29 12:27:25 DNS user.notice root: sslcert stopped.
May 29 12:27:25 DNS user.notice root: syslogd running
May 29 12:27:26 DNS user.notice root: Starting user: OK.
May 29 12:27:26 DNS user.notice root: ****** Boot Log start ******
May 29 12:27:26 DNS user.notice root: May 29 19:27:00 syslogd started: BusyBox v1.20.2
May 29 12:27:26 DNS user.notice root: May 29 19:27:00 root: Board: B1
May 29 12:27:26 DNS user.notice root: May 29 19:27:00 root: Firmware: Alt-F-0.1RC3, kernel 2.6.35.14
May 29 12:27:26 DNS user.notice root: May 29 19:27:02 root: fsck 1.41.14 (22-Dec-2010) fsck.minix: BusyBox v1.20.2 (2013-03-17 19:50:30 WET) /dev/mtdblock0 is clean, check is skipped
May 29 12:27:26 DNS user.notice root: May 29 12:27:02 root: FIXME: Waiting 3 sec for kernel enumeration to finish...
May 29 12:27:26 DNS user.notice root: May 29 12:27:05 root: FIXME: stop waiting.
May 29 12:27:26 DNS user.notice root: May 29 12:27:11 hot: Start fscking md0
May 29 12:27:26 DNS user.notice root: May 29 12:27:12 root: IP from flash-defaults
May 29 12:27:26 DNS user.notice root: May 29 12:27:12 syslogd exiting
May 29 12:27:26 DNS user.notice root: ****** Boot Log end ******
May 29 12:27:26 DNS daemon.info init: starting pid 863, tty '/dev/ttyS0': '/sbin/getty -n -l /bin/sh -L ttyS0 115200 vt100'
May 29 12:27:43 DNS daemon.info sysctrl: temp=40.3	 fan=2137

[João Cardoso]

On Tuesday, 29 May 2018 20:24:41 UTC+1, John G wrote:

Hi,

A few days ago the lights on the front of my DNS-323 changed to amber.  On the WebUI, I see the raid is degraded.  However, on the Disk->RAID page there isn't anything in the maintenance section.

There is an (harmless) extraneous message (fdisk: device...) that avoids the RAID page to be correctly displayed.

From the command line you can see the RAID1 current detailed state:

mdadm --detail /dev/md0

 

and each RAID component disk state:

mdadm --examine /dev/sda2 /dev/sdb2 # probably sda2 and sdb2, examine the previous command last lines output

You should probably upgrade to Alt-F-1.0, but the recovery procedure will always be manual (and covered in the wiki)

You also have the filesystem mounted in read-only (RO) mode, which means that it was found inconsistent (covered also several times in this forum)

  Both disks are indicated healthy,

From a S.M.A.R.T. perspective, but that is only a "summary executive" message word, use Disk->Utilities, Health, Show Status

[John G]

Thanks for the reply.

I will upgrade to 1.0.

Here is the output from the commands:

# mdadm --detail /dev/md0

/dev/md0:

Version : 1.2

Creation Time : Tue Jun 10 23:54:09 2014

Raid Level : raid1

Array Size : 2929740112 (2794.02 GiB 3000.05 GB)

Used Dev Size : 2929740112 (2794.02 GiB 3000.05 GB)

Raid Devices : 2

Total Devices : 1

Persistence : Superblock is persistent

Intent Bitmap : Internal

Update Time : Tue May 29 17:40:08 2018

State : active, degraded

Active Devices : 1

Working Devices : 1

Failed Devices : 0

Spare Devices : 0

Name : ALTF:0

UUID : 520925fb:026bb47a:eadeb52c:1e400b5a

Events : 33371

Number   Major   Minor   RaidDevice State

0       8       18        0      active sync   /dev/sdb2

1       0        0        1      removed

#

# mdadm --examine /dev/sda2 /dev/sdb2 #

/dev/sda2:

Magic : a92b4efc

Version : 1.2

Feature Map : 0x1

Array UUID : 520925fb:026bb47a:eadeb52c:1e400b5a

Name : ALTF:0

Creation Time : Tue Jun 10 23:54:09 2014

Raid Level : raid1

Raid Devices : 2

Avail Dev Size : 5859480496 (2794.02 GiB 3000.05 GB)

Array Size : 5859480224 (2794.02 GiB 3000.05 GB)

Used Dev Size : 5859480224 (2794.02 GiB 3000.05 GB)

Data Offset : 2048 sectors

Super Offset : 8 sectors

State : active

Device UUID : 8a29502d:d5f04f17:7a111db8:bb5845a2

Internal Bitmap : 8 sectors from superblock

Update Time : Fri May 18 12:36:58 2018

Checksum : f4365477 - correct

Events : 6328

Device Role : Active device 1

Array State : AA ('A' == active, '.' == missing)

/dev/sdb2:

Magic : a92b4efc

Version : 1.2

Feature Map : 0x1

Array UUID : 520925fb:026bb47a:eadeb52c:1e400b5a

Name : ALTF:0

Creation Time : Tue Jun 10 23:54:09 2014

Raid Level : raid1

Raid Devices : 2

Avail Dev Size : 5859480496 (2794.02 GiB 3000.05 GB)

Array Size : 5859480224 (2794.02 GiB 3000.05 GB)

Used Dev Size : 5859480224 (2794.02 GiB 3000.05 GB)

Data Offset : 2048 sectors

Super Offset : 8 sectors

State : clean

Device UUID : ffe84a4a:93392413:443ff46f:c35de090

Internal Bitmap : 8 sectors from superblock

Update Time : Wed May 30 10:13:11 2018

Checksum : b384a96e - correct

Events : 33373

Device Role : Active device 0

Array State : A. ('A' == active, '.' == missing) and here are the statuses === START OF INFORMATION SECTION === Model Family: Western Digital Caviar Green (AF) Device Model: WDC WD30EZRX-00SPEB0 Serial Number: WD-WCC4E1279692 LU WWN Device Id: 5 0014ee 25f5353c0 Firmware Version: 80.00A80 User Capacity: 3,000,592,982,016 bytes [3.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5400 rpm Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2 (minor revision not indicated) SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Wed May 30 10:19:49 2018 PDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 176 173 021 Pre-fail Always - 8200 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 39 5 Reallocated_Sector_Ct 0x0033 195 195 140 Pre-fail Always - 167 7 Seek_Error_Rate 0x002e 200 199 000 Old_age Always - 0 9 Power_On_Hours 0x0032 070 070 000 Old_age Always - 22033 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 39 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 35 193 Load_Cycle_Count 0x0032 188 188 000 Old_age Always - 37876 194 Temperature_Celsius 0x0022 119 107 000 Old_age Always - 33 196 Reallocated_Event_Count 0x0032 033 033 000 Old_age Always - 167 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 11 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 5 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 46 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 2287 ------ === START OF INFORMATION SECTION === Model Family: Western Digital Caviar Green (AF) Device Model: WDC WD30EZRX-00D8PB0 Serial Number: WD-WCC4N1285836 LU WWN Device Id: 5 0014ee 20a40d410 Firmware Version: 80.00A80 User Capacity: 3,000,592,982,016 bytes [3.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5400 rpm Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2 (minor revision not indicated) SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Wed May 30 10:20:37 2018 PDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 178 177 021 Pre-fail Always - 6058 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 39 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 069 069 000 Old_age Always - 23151 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 39 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 30 193 Load_Cycle_Count 0x0032 188 188 000 Old_age Always - 37747 194 Temperature_Celsius 0x0022 119 108 000 Old_age Always - 31 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 22878 ------------- I'll look into the RO mounting. I thought it was RW.

[João Cardoso]

It was sda that was removed from the RAID:

May 29 12:27:12 DNS user.warn kernel: md: kicking non-fresh sda2 from array!

The event that caused it to be initially removed from the ARRAY (and not being "fresh" now) is not in the log, it occurred before the current boot

/dev/md0: 

Version : 1.2

Number   Major   Minor   RaidDevice State

       0       8       18        0      active sync   /dev/sdb2

       1       0        0        1      removed

/dev/sda2:

Update Time : Fri May 18 12:36:58 2018

/dev/sdb2:

Update Time : Wed May 30 10:13:11 2018

Notice that you have a 1.2 version RAID1. Some instructions on the wiki applies only to 0.9 or 1.0 versions.

Probably you only need to "add" sda2 to the existing RAID (eventually after "clearing" it RAID info first) and wait for the resync to complete, but be sure that the disk is OK, make long SMART tests on it first (Disk->utilities, Health).

It looks like you don't have automatic SMART self tests active and boot enabled (Services->System, smart), perhaps because you don't have setup an e-mail account?

[John Guenther]

Upgrade to 1.0 has provided a lot of help, but RAID issue remains.  Looks like bad disk.

--
You received this message because you are subscribed to a topic in the Google Groups "Alt-F" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/alt-f/u4cCB89i43Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to alt-f+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/alt-f.
For more options, visit https://groups.google.com/d/optout.

[John G]

The new status page is showing a lot more info and errors:

 

[João Cardoso]

Yes, it looks like the sda disk has failed.

You now don't have redundancy, and as both disks are from the same brand and model and probably bought at the same time on the same shop, they are probably from the same manufacturing batch, so chances that the second disk might also fail is greater. Make sure your backup is up to date (or backup your data first).

That said, the disks are not that old, only aged, 22000 power on hours, and the failed disk SMART log is not that bad (although it shows errors).

Sometime pending read errors are remapped on write, so you might want to write/erase the whole disk on a PC and staging it for some days while performing full SMART tests on it, to see if the errors stop growing and stabilize:

  5 Reallocated_Sector_Ct   0x0033   195   195   140    Pre-fail  Always       -       167

The current reallocated value (195) is still away (lower is worst) from the fail threshold (140) -- 167 is the raw value, higher is worse

196 Reallocated_Event_Count 0x0032   033   033   000    Old_age   Always       -       167
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       11

These pending sectors can only be corrected by a write to the same sector/LBA. As it an expert job to determine which, the brute force approach is to write to the full disk.

198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       5

200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       46

you might want to read https://raid.wiki.kernel.org/index.php/Monitoring_your_system

After having a sane RAID setup e-mail and activate mdadm (Services->System, mdadm) monitoring and SMART tests

[John Guenther]

Thank you very much for all your work and support!

--

You received this message because you are subscribed to a topic in the Google Groups "Alt-F" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/alt-f/u4cCB89i43Y/unsubscribe.

To unsubscribe from this group and all its topics, send an email to alt-f+un...@googlegroups.com.