Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Upgrading from 8.3 to 9.1 zfs pool can't attach

4 views
Skip to first unread message

Leslie Jensen

unread,
May 22, 2013, 6:53:25 AM5/22/13
to

I've upgraded a machine with freebsd-update from 8.3 to 9.1.

After the first restart I edited /etc/fstab in single user mode because
the names on the disks had changed. But the zfs pool I have seem to have
a problem and I'm sure on how to recover it.


May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0.
May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1.
May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4.
May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5.


camcontrol devlist
<ST3320620AS 3.AAE> at scbus0 target 0 lun 0 (ada0,pass0)
<ST3320620AS 3.AAE> at scbus1 target 0 lun 0 (ada1,pass1)
<WDC WD360GD-00FNA0 35.06K35> at scbus2 target 0 lun 0 (ada2,pass2)
<WDC WD360GD-00FNA0 35.06K35> at scbus2 target 1 lun 0 (ada3,pass3)
<ST3320620AS 3.AAE> at scbus3 target 0 lun 0 (ada4,pass4)
<ST3320620AS 3.AAE> at scbus3 target 1 lun 0 (ada5,pass5)

ada2 and ada3 are configured as raid/r0 holding the system.

ada0, ada1, ada4 and ada5 should be a zfs pool with /home


cat /etc/fstab

# Device Mountpoint FStype Options Dump Pass#
/dev/raid/r0s1b none swap sw 0 0
/dev/raid/r0s1a / ufs rw 1 1
/dev/raid/r0s1e /tmp ufs rw 2 2
/dev/raid/r0s1f /usr ufs rw 2 2
/dev/raid/r0s1d /var ufs rw 2 2


I'm not sure but shouldn't the mount point for /home also appear in fstab?

Maybe I messed up during the manual editing of files during the update
process?


Can I recover or shall I destroy the pool and rebuild it?

Thanks

/Leslie



_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-...@freebsd.org"

Ronald Klop

unread,
May 22, 2013, 11:02:09 AM5/22/13
to
Mounts from zfs do not have to be in fstab. You can set a mountpoint on a
zfs volume which kind of mounts itself.
Can you send more information about the 'Unable to attach to ada[0-4]'
messages? Maybe a complete dmesg (/var/run/dmesg.boot).
And how are these disks configured/partitioned? Output from 'gpart list
ada0' or 'gpart show ada0' and for the other disks.
Are you using stuff like glabel?

Ronald.

Freddie Cash

unread,
May 22, 2013, 11:33:51 AM5/22/13
to
What's the output of "zpool import" in single-user mode?

What are the names of your disk devices? "grep ad /var/log/messages" for
example.

Are you using whole disks, disk partitions, labels, etc to access your
disks? Are you sure your disks haven't been renumbered at all?
> ______________________________**_________________
> freeb...@freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-fs<http://lists.freebsd.org/mailman/listinfo/freebsd-fs>
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@**freebsd.org<freebsd-fs-...@freebsd.org>
> "
>



--
Freddie Cash
fjw...@gmail.com

Leslie Jensen

unread,
May 22, 2013, 1:45:12 PM5/22/13
to
> _________________________________________________
> freeb...@freebsd.org <mailto:freeb...@freebsd.org> mailing list
> http://lists.freebsd.org/__mailman/listinfo/freebsd-fs
> <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>
> To unsubscribe, send any mail to
> "freebsd-fs-unsubscribe@__freebsd.org
> <mailto:freebsd-fs-...@freebsd.org>"
>
>
>
>
> --
> Freddie Cash
> fjw...@gmail.com <mailto:fjw...@gmail.com>


zpool import in single user mode gives:

zfs filesystem version 5
zfs storage pool version 28
ZFS WARNING: Unable to attach to ada0.
ZFS WARNING: Unable to attach to ada1.
ZFS WARNING: Unable to attach to ada4.
ZFS WARNING: Unable to attach to ada5.



grep ada /var/log/messages
--------------------------------------------

May 22 19:34:03 bagnerbsd kernel: ada0 at siisch0 bus 0 scbus0 target 0
lun 0
May 22 19:34:03 bagnerbsd kernel: ada0: <ST3320620AS 3.AAE> ATA-7 SATA
1.x device
May 22 19:34:03 bagnerbsd kernel: ada0: 150.000MB/s transfers (SATA 1.x,
UDMA6, PIO 8192bytes)
May 22 19:34:03 bagnerbsd kernel: ada0: Command Queueing enabled
May 22 19:34:03 bagnerbsd kernel: ada0: 305245MB (625142448 512 byte
sectors: 16H 63S/T 16383C)
May 22 19:34:03 bagnerbsd kernel: ada0: Previously was known as ad4
May 22 19:34:03 bagnerbsd kernel: ada1 at siisch1 bus 0 scbus1 target 0
lun 0
May 22 19:34:03 bagnerbsd kernel: ada1: <ST3320620AS 3.AAE> ATA-7 SATA
1.x device
May 22 19:34:03 bagnerbsd kernel: ada1: 150.000MB/s transfers (SATA 1.x,
UDMA6, PIO 8192bytes)
May 22 19:34:03 bagnerbsd kernel: ada1: Command Queueing enabled
May 22 19:34:03 bagnerbsd kernel: ada1: 305245MB (625142448 512 byte
sectors: 16H 63S/T 16383C)
May 22 19:34:03 bagnerbsd kernel: ada1: Previously was known as ad6
May 22 19:34:03 bagnerbsd kernel: ada2 at ata0 bus 0 scbus2 target 0 lun 0
May 22 19:34:03 bagnerbsd kernel: ada2: <WDC WD360GD-00FNA0 35.06K35>
ATA-6 SATA 1.x device
May 22 19:34:03 bagnerbsd kernel: ada2: 150.000MB/s transfers (SATA 1.x,
UDMA5, PIO 8192bytes)
May 22 19:34:03 bagnerbsd kernel: ada2: 35304MB (72303840 512 byte
sectors: 16H 63S/T 16383C)
May 22 19:34:03 bagnerbsd kernel: ada2: Previously was known as ad0
May 22 19:34:03 bagnerbsd kernel: ada3 at ata0 bus 0 scbus2 target 1 lun 0
May 22 19:34:03 bagnerbsd kernel: ada3: <WDC WD360GD-00FNA0 35.06K35>
ATA-6 SATA 1.x device
May 22 19:34:03 bagnerbsd kernel: ada3: 150.000MB/s transfers (SATA 1.x,
UDMA5, PIO 8192bytes)
May 22 19:34:03 bagnerbsd kernel: ada3: 35304MB (72303840 512 byte
sectors: 16H 63S/T 16383C)
May 22 19:34:03 bagnerbsd kernel: ada3: Previously was known as ad1
May 22 19:34:03 bagnerbsd kernel: ada4 at ata1 bus 0 scbus3 target 0 lun 0
May 22 19:34:03 bagnerbsd kernel: ada4: <ST3320620AS 3.AAE> ATA-7 SATA
1.x device
May 22 19:34:03 bagnerbsd kernel: ada4: 150.000MB/s transfers (SATA 1.x,
UDMA5, PIO 8192bytes)
May 22 19:34:03 bagnerbsd kernel: ada4: 305245MB (625142448 512 byte
sectors: 16H 63S/T 16383C)
May 22 19:34:03 bagnerbsd kernel: ada4: Previously was known as ad2
May 22 19:34:03 bagnerbsd kernel: ada5 at ata1 bus 0 scbus3 target 1 lun 0
May 22 19:34:03 bagnerbsd kernel: ada5: <ST3320620AS 3.AAE> ATA-7 SATA
1.x device
May 22 19:34:03 bagnerbsd kernel: ada5: 150.000MB/s transfers (SATA 1.x,
UDMA5, PIO 8192bytes)
May 22 19:34:03 bagnerbsd kernel: ada5: 305245MB (625142448 512 byte
sectors: 16H 63S/T 16383C)
May 22 19:34:03 bagnerbsd kernel: ada5: Previously was known as ad3
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada2
state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk
Volume0:0-ada2 state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada3
state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk
Volume0:1-ada3 state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada0
state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
Volume0:1-ada0 state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada1
state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
Volume0:3-ada1 state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada4
state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk
SiI Raid5 Set:0-ada4 state changed from NONE to STALE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada5
state changed from NONE to ACTIVE.
May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk
SiI Raid5 Set:1-ada5 state changed from NONE to STALE.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5
----------------------------------------------

I'm using whole disks. The pool was created back in 2010 and I have
forgot the details on how I did but I think I just created the out of
the four disks.

The disks might have been renumbered I'm not sure. It would be logical
because the disks ada2 and ada3 had to bee changed in fstab as I wrote
before.

Thanks

/Leslie

Leslie Jensen

unread,
May 22, 2013, 1:54:34 PM5/22/13
to


2013-05-22 17:02, Ronald Klop skrev:
> Mounts from zfs do not have to be in fstab. You can set a mountpoint on
> a zfs volume which kind of mounts itself.
> Can you send more information about the 'Unable to attach to ada[0-4]'
> messages? Maybe a complete dmesg (/var/run/dmesg.boot).
> And how are these disks configured/partitioned? Output from 'gpart list
> ada0' or 'gpart show ada0' and for the other disks.
> Are you using stuff like glabel?
>
> Ronald.


cat /var/run/dmesg.boot
-----------------------------------------------
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013
ro...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
link_elf_obj: symbol ata_controlcmd undefined
KLD file atapicam.ko - could not finalize loading
CPU: Intel(R) Pentium(R) 4 CPU 3.20GHz (3192.08-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0xf43 Family = f Model = 4 Stepping = 3

Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x649d<SSE3,DTES64,MON,DS_CPL,EST,CNXT-ID,CX16,xTPR>
AMD Features=0x20100800<SYSCALL,NX,LM>
TSC: P-state invariant
real memory = 4294967296 (4096 MB)
avail memory = 4084785152 (3895 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <IntelR AWRDACPI>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads
cpu0 (BSP): APIC ID: 0
cpu1 (AP/HT): APIC ID: 1
ioapic0: Changing APIC ID to 4
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <IntelR AWRDACPI> on motherboard
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, bfde0000 (3) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x73 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xbe00-0xbeff mem
0xd0000000-0xdfffffff,0xfd9e0000-0xfd9effff irq 16 at device 0.0 on pci1
hdac0: <ATI RV610 HDA Controller> mem 0xfd9fc000-0xfd9fffff irq 17 at
device 0.1 on pci1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
siis0: <SiI3132 SATA controller> port 0xaf00-0xaf7f mem
0xfd6ff000-0xfd6ff07f,0xfd6f8000-0xfd6fbfff irq 16 at device 0.0 on pci2
siisch0: <SIIS channel> at channel 0 on siis0
siisch1: <SIIS channel> at channel 1 on siis0
pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
pci3: <ACPI PCI bus> on pcib3
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x004101> mem 0xfddf0000-0xfddfffff irq 17 at device 0.0 on pci3
bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM5750 1000BASE-T media interface> PHY 1 on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Ethernet address: 00:50:8d:7c:c2:60
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci4: <ACPI PCI bus> on pcib4
bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x004101> mem 0xfdbf0000-0xfdbfffff irq 16 at device 0.0 on pci4
bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E
miibus1: <MII bus> on bge1
brgphy1: <BCM5750 1000BASE-T media interface> PHY 1 on miibus1
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Ethernet address: 00:50:8d:7c:c2:61
uhci0: <Intel 82801G (ICH7) USB controller USB-A> port 0xff00-0xff1f irq
23 at device 29.0 on pci0
usbus0 on uhci0
uhci1: <Intel 82801G (ICH7) USB controller USB-B> port 0xfe00-0xfe1f irq
19 at device 29.1 on pci0
usbus1 on uhci1
uhci2: <Intel 82801G (ICH7) USB controller USB-C> port 0xfd00-0xfd1f irq
18 at device 29.2 on pci0
usbus2 on uhci2
uhci3: <Intel 82801G (ICH7) USB controller USB-D> port 0xfc00-0xfc1f irq
16 at device 29.3 on pci0
usbus3 on uhci3
ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem
0xfdfff000-0xfdfff3ff irq 23 at device 29.7 on pci0
usbus4: EHCI version 1.0
usbus4 on ehci0
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci5: <ACPI PCI bus> on pcib5
fwohci0: <Texas Instruments TSB82AA2> mem
0xfd8ff000-0xfd8ff7ff,0xfd8f8000-0xfd8fbfff irq 17 at device 2.0 on pci5
fwohci0: OHCI version 1.10 (ROM=1)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 00:50:8d:00:00:72:25:9e
fwohci0: invalid speed 7 (fixed to 3).
fwohci0: Phy 1394a available S800, 3 ports.
fwohci0: Link S800, max_rec 4096 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 02:50:8d:72:25:9e
fwe0: Ethernet address: 02:50:8d:72:25:9e
fwip0: <IP over FireWire> on firewire0
fwip0: Firewire address: 00:50:8d:00:00:72:25:9e @ 0xfffe00000000, S800,
maxrec 4096
dcons_crom0: <dcons configuration ROM> on firewire0
dcons_crom0: bus_addr 0x2a20000
fwohci0: Initiate bus reset
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1,
CYCLEMASTER mode
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH7 SATA300 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfb00-0xfb0f mem
0xfdffe000-0xfdffe3ff at device 31.2 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz0: _CRT value is absurd, ignored (255.0C)
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcffff,0xd0000-0xd3fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
ctl: CAM Target Layer loaded
acpi_perf0: <ACPI CPU Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 102d0000102d
device_attach: est1 attach returned 6
p4tcc1: <CPU Frequency Thermal Control> on cpu1
(noperiph:siisch0:0:-1:-1): rescan already queued
(noperiph:siisch1:0:-1:-1): rescan already queued
firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me)
firewire0: bus manager 0
fwohci0: phy int
Timecounters tick every 1.000 msec
hdacc0: <ATI R6xx HDA CODEC> at cad 0 on hdac0
hdaa0: <ATI R6xx Audio Function Group> at nid 1 on hdacc0
pcm0: <ATI R6xx (HDMI)> at nid 3 on hdaa0
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
acpi_tz0: _CRT value is absurd, ignored (255.0C)
acpi_tz0: failed to set new freq, disabling passive cooling
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
(aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00
00 40 00 00 00 00 03 00
(aprobe2:ata0:0:0:0): CAM status: ATA Status Error
(aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00
(aprobe2:ata0:0:0:0): Retrying command
(aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00
00 40 00 00 00 00 03 00
(aprobe2:ata0:0:0:0): CAM status: ATA Status Error
(aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00
(aprobe2:ata0:0:0:0): Error 5, Retries exhausted
(aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00
00 40 00 00 00 00 03 00
(aprobe0:ata0:0:1:0): CAM status: ATA Status Error
(aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00
(aprobe0:ata0:0:1:0): Retrying command
(aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00
00 40 00 00 00 00 03 00
(aprobe0:ata0:0:1:0): CAM status: ATA Status Error
(aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00
(aprobe0:ata0:0:1:0): Error 5, Retries exhausted
acpi_tz0: _CRT value is absurd, ignored (255.0C)
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ada0 at siisch0 bus 0 scbus0 target 0 lun 0
ada0: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at siisch1 bus 0 scbus1 target 0 lun 0
ada1: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6
ada2 at ata0 bus 0 scbus2 target 0 lun 0
ada2: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad0
ada3 at ata0 bus 0 scbus2 target 1 lun 0
ada3: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
ada3: Previously was known as ad1
ada4 at ata1 bus 0 scbus3 target 0 lun 0
ada4: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
ada4: Previously was known as ad2
ada5 at ata1 bus 0 scbus3 target 1 lun 0
ada5: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
ada5: Previously was known as ad3
SMP: AP CPU #1 Launched!
Timecounter "TSC-low" frequency 12469046 Hz quality 1000
GEOM_RAID: Intel-165267ff: Array Intel-165267ff created.
GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created.
GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from
NONE to ACTIVE.
GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from
NONE to ACTIVE.
GEOM_RAID: Intel-841a1a00: Array started.
GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from STARTING to
OPTIMAL.
GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created.
GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created.
Root mount waiting for: GRAID-SiI GRAID-Intel
ugen1.2: <Dell> at usbus1
ukbd0: <Dell Dell QuietKey Keyboard, class 0/0, rev 1.10/1.50, addr 2>
on usbus1
kbd2 at ukbd0
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
Root mount waiting for: GRAID-SiI GRAID-Intel
GEOM_RAID: Intel-165267ff: Force array start due to timeout.
GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from
NONE to ACTIVE.
GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from
NONE to ACTIVE.
GEOM_RAID: Intel-165267ff: Array started.
GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from STARTING to
BROKEN.
GEOM_RAID: SiI-100311082330: Force array start due to timeout.
GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE.
GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed
from NONE to STALE.
GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE.
GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed
from NONE to STALE.
GEOM_RAID: SiI-100311082330: Array started.
GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from
STARTING to BROKEN.
Trying to mount root from ufs:/dev/raid/r0s1a [rw]...
ZFS filesystem version 5
ZFS storage pool version 28
ZFS WARNING: Unable to attach to ada0.
ZFS WARNING: Unable to attach to ada1.
ZFS WARNING: Unable to attach to ada4.
ZFS WARNING: Unable to attach to ada5.
ZFS WARNING: Unable to attach to ada0.
ZFS WARNING: Unable to attach to ada1.
ZFS WARNING: Unable to attach to ada4.
ZFS WARNING: Unable to attach to ada5.
ZFS WARNING: Unable to attach to ada0.
ZFS WARNING: Unable to attach to ada1.
ZFS WARNING: Unable to attach to ada4.
ZFS WARNING: Unable to attach to ada5.
----------------------------------------------------



gpart list
------------------------------------------------

Geom name: raid/r0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 72298495
first: 63
entries: 4
scheme: MBR
Providers:
1. Name: raid/r0s1
Mediasize: 37013727744 (34G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 32256
Mode: r5w5e9
attrib: active
rawtype: 165
length: 37013727744
offset: 32256
type: freebsd
index: 1
end: 72292499
start: 63
Consumers:
1. Name: raid/r0
Mediasize: 37016829952 (34G)
Sectorsize: 512
Mode: r5w5e14

Geom name: raid/r0s1
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 72292436
first: 0
entries: 8
scheme: BSD
Providers:
1. Name: raid/r0s1a
Mediasize: 1073741824 (1.0G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 32256
Mode: r1w1e1
rawtype: 7
length: 1073741824
offset: 0
type: freebsd-ufs
index: 1
end: 2097151
start: 0
2. Name: raid/r0s1b
Mediasize: 4294967296 (4.0G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1073774080
Mode: r1w1e0
rawtype: 1
length: 4294967296
offset: 1073741824
type: freebsd-swap
index: 2
end: 10485759
start: 2097152
3. Name: raid/r0s1d
Mediasize: 5368709120 (5.0G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1073774080
Mode: r1w1e1
rawtype: 7
length: 5368709120
offset: 5368709120
type: freebsd-ufs
index: 4
end: 20971519
start: 10485760
4. Name: raid/r0s1e
Mediasize: 2147483648 (2.0G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 2147515904
Mode: r1w1e1
rawtype: 7
length: 2147483648
offset: 10737418240
type: freebsd-ufs
index: 5
end: 25165823
start: 20971520
5. Name: raid/r0s1f
Mediasize: 24128825856 (22G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 32256
Mode: r1w1e1
rawtype: 7
length: 24128825856
offset: 12884901888
type: freebsd-ufs
index: 6
end: 72292436
start: 25165824
Consumers:
1. Name: raid/r0s1
Mediasize: 37013727744 (34G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 32256
Mode: r5w5e9

----------------------------------------

gpart list ada0
gpart: No such geom: ada0.

gpart list ada1
gpart: No such geom: ada1.

gpart list ada4
gpart: No such geom: ada4.

gpart list ada5
gpart: No such geom: ada5.

gpart list ad0
gpart: No such geom: ad0.

gpart list ad1
gpart: No such geom: ad1.

gpart list ad4
gpart: No such geom: ad4.

gpart list ad5
gpart: No such geom: ad5

--------------------------------------


gpart show (This is the system raid)

-------------------------------------

=> 63 72298433 raid/r0 MBR (34G)
63 72292437 1 freebsd [active] (34G)
72292500 5996 - free - (3M)

=> 0 72292437 raid/r0s1 BSD (34G)
0 2097152 1 freebsd-ufs (1.0G)
2097152 8388608 2 freebsd-swap (4.0G)
10485760 10485760 4 freebsd-ufs (5.0G)
20971520 4194304 5 freebsd-ufs (2.0G)
25165824 47126613 6 freebsd-ufs (22G)

--------------------------------------------

gpart show ada0
gpart: No such geom: ada0

gpart show ada1
gpart: No such geom: ada1

gpart show ada4
gpart: No such geom: ada4

gpart show ada5
gpart: No such geom: ada5



I'm not sure about glabel, Sorry.

Freddie Cash

unread,
May 22, 2013, 1:54:07 PM5/22/13
to
On Wed, May 22, 2013 at 10:45 AM, Leslie Jensen <les...@eskk.nu> wrote:

> zpool import in single user mode gives:
>
> zfs filesystem version 5
> zfs storage pool version 28
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
>
>
>
> grep ada /var/log/messages
> ------------------------------**--------------
>
> May 22 19:34:03 bagnerbsd kernel: ada0 at siisch0 bus 0 scbus0 target 0
> lun 0
> May 22 19:34:03 bagnerbsd kernel: ada0: <ST3320620AS 3.AAE> ATA-7 SATA 1.x
> device
> May 22 19:34:03 bagnerbsd kernel: ada0: 150.000MB/s transfers (SATA 1.x,
> UDMA6, PIO 8192bytes)
> May 22 19:34:03 bagnerbsd kernel: ada0: Command Queueing enabled
> May 22 19:34:03 bagnerbsd kernel: ada0: 305245MB (625142448 512 byte
> sectors: 16H 63S/T 16383C)
> May 22 19:34:03 bagnerbsd kernel: ada0: Previously was known as ad4
> May 22 19:34:03 bagnerbsd kernel: ada1 at siisch1 bus 0 scbus1 target 0
> lun 0
> May 22 19:34:03 bagnerbsd kernel: ada1: <ST3320620AS 3.AAE> ATA-7 SATA 1.x
> device
> May 22 19:34:03 bagnerbsd kernel: ada1: 150.000MB/s transfers (SATA 1.x,
> UDMA6, PIO 8192bytes)
> May 22 19:34:03 bagnerbsd kernel: ada1: Command Queueing enabled
> May 22 19:34:03 bagnerbsd kernel: ada1: 305245MB (625142448 512 byte
> sectors: 16H 63S/T 16383C)
> May 22 19:34:03 bagnerbsd kernel: ada1: Previously was known as ad6
> May 22 19:34:03 bagnerbsd kernel: ada2 at ata0 bus 0 scbus2 target 0 lun 0
> May 22 19:34:03 bagnerbsd kernel: ada2: <WDC WD360GD-00FNA0 35.06K35>
> ATA-6 SATA 1.x device
> May 22 19:34:03 bagnerbsd kernel: ada2: 150.000MB/s transfers (SATA 1.x,
> UDMA5, PIO 8192bytes)
> May 22 19:34:03 bagnerbsd kernel: ada2: 35304MB (72303840 512 byte
> sectors: 16H 63S/T 16383C)
> May 22 19:34:03 bagnerbsd kernel: ada2: Previously was known as ad0
> May 22 19:34:03 bagnerbsd kernel: ada3 at ata0 bus 0 scbus2 target 1 lun 0
> May 22 19:34:03 bagnerbsd kernel: ada3: <WDC WD360GD-00FNA0 35.06K35>
> ATA-6 SATA 1.x device
> May 22 19:34:03 bagnerbsd kernel: ada3: 150.000MB/s transfers (SATA 1.x,
> UDMA5, PIO 8192bytes)
> May 22 19:34:03 bagnerbsd kernel: ada3: 35304MB (72303840 512 byte
> sectors: 16H 63S/T 16383C)
> May 22 19:34:03 bagnerbsd kernel: ada3: Previously was known as ad1
> May 22 19:34:03 bagnerbsd kernel: ada4 at ata1 bus 0 scbus3 target 0 lun 0
> May 22 19:34:03 bagnerbsd kernel: ada4: <ST3320620AS 3.AAE> ATA-7 SATA 1.x
> device
> May 22 19:34:03 bagnerbsd kernel: ada4: 150.000MB/s transfers (SATA 1.x,
> UDMA5, PIO 8192bytes)
> May 22 19:34:03 bagnerbsd kernel: ada4: 305245MB (625142448 512 byte
> sectors: 16H 63S/T 16383C)
> May 22 19:34:03 bagnerbsd kernel: ada4: Previously was known as ad2
> May 22 19:34:03 bagnerbsd kernel: ada5 at ata1 bus 0 scbus3 target 1 lun 0
> May 22 19:34:03 bagnerbsd kernel: ada5: <ST3320620AS 3.AAE> ATA-7 SATA 1.x
> device
> May 22 19:34:03 bagnerbsd kernel: ada5: 150.000MB/s transfers (SATA 1.x,
> UDMA5, PIO 8192bytes)
> May 22 19:34:03 bagnerbsd kernel: ada5: 305245MB (625142448 512 byte
> sectors: 16H 63S/T 16383C)
> May 22 19:34:03 bagnerbsd kernel: ada5: Previously was known as ad3
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada2
> state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk
> Volume0:0-ada2 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada3
> state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk
> Volume0:1-ada3 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada0
> state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
> Volume0:1-ada0 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada1
> state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
> Volume0:3-ada1 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada4
> state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI
> Raid5 Set:0-ada4 state changed from NONE to STALE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada5
> state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI
> Raid5 Set:1-ada5 state changed from NONE to STALE.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5
> ------------------------------**----------------
>
> I'm using whole disks. The pool was created back in 2010 and I have forgot
> the details on how I did but I think I just created the out of the four
> disks.
>
> The disks might have been renumbered I'm not sure. It would be logical
> because the disks ada2 and ada3 had to bee changed in fstab as I wrote
> before.
>
> Thanks
>

What happens if you rename /boot/zfs/zpool.cache, and run "zpool import -d
/dev <poolname>"?

--
Freddie Cash
fjw...@gmail.com

Andriy Gapon

unread,
May 22, 2013, 2:04:22 PM5/22/13
to
on 22/05/2013 20:45 Leslie Jensen said the following:
>
>
> 2013-05-22 17:33, Freddie Cash skrev:
[snip]
>> On Wed, May 22, 2013 at 3:53 AM, Leslie Jensen <les...@eskk.nu
>> <mailto:les...@eskk.nu>> wrote:
[snip]
>> ada2 and ada3 are configured as raid/r0 holding the system.
>>
>> ada0, ada1, ada4 and ada5 should be a zfs pool with /home

If you look at the messages below you can see that ada0, ada1, ada4 and ada5 are
also claimed by the raid driver.


> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada0 state
> changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
> Volume0:1-ada0 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada1 state
> changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
> Volume0:3-ada1 state changed from NONE to ACTIVE.

> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada4 state
> changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5
> Set:0-ada4 state changed from NONE to STALE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada5 state
> changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5
> Set:1-ada5 state changed from NONE to STALE.

Apparently they used to be in raid configurations and the metadata was never
properly cleared.

--
Andriy Gapon

Freddie Cash

unread,
May 22, 2013, 2:06:23 PM5/22/13
to
Do you have RAID arrays on both the Intel and the Sil controllers? Or just
the Intel? I think you may be getting bit by the "graid now in kernel, old
metadata being read, graid taking control of disks it shouldn't be" issue.

You need to manually clear the RAID metadata from all your non-RAID (ie,
used for ZFS) disks.

See the graid man page for details. And the -stable mailing list archives.

Once you've cleared the RAID metadata from the ZFS disks, ZFS will be able
to pick them up correctly.

--
Freddie Cash
fjw...@gmail.com

Leslie Jensen

unread,
May 22, 2013, 2:06:54 PM5/22/13
to


2013-05-22 19:54, Freddie Cash skrev:
> On Wed, May 22, 2013 at 10:45 AM, Leslie Jensen <les...@eskk.nu
> <mailto:les...@eskk.nu>> wrote:
>
> zpool import in single user mode gives:
>
> zfs filesystem version 5
> zfs storage pool version 28
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
>
>
>
> grep ada /var/log/messages
> ------------------------------__--------------
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk
> ada2 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk
> Volume0:0-ada2 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk
> ada3 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk
> Volume0:1-ada3 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk
> ada0 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
> Volume0:1-ada0 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk
> ada1 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk
> Volume0:3-ada1 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk
> ada4 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330:
> Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk
> ada5 state changed from NONE to ACTIVE.
> May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330:
> Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4.
> May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5
> ------------------------------__----------------
>
> I'm using whole disks. The pool was created back in 2010 and I have
> forgot the details on how I did but I think I just created the out
> of the four disks.
>
> The disks might have been renumbered I'm not sure. It would be
> logical because the disks ada2 and ada3 had to bee changed in fstab
> as I wrote before.
>
> Thanks
>
>
> What happens if you rename /boot/zfs/zpool.cache, and run "zpool import
> -d /dev <poolname>"?
>
> --
> Freddie Cash
> fjw...@gmail.com <mailto:fjw...@gmail.com>


This is what happens

zpool import -d /dev storage
cannot import 'storage': a pool with that name is already created/imported,
and no additional pools with that name were found


/Leslie

Leslie Jensen

unread,
May 22, 2013, 2:19:59 PM5/22/13
to


2013-05-22 20:06, Freddie Cash skrev:
> Do you have RAID arrays on both the Intel and the Sil controllers? Or
> just the Intel? I think you may be getting bit by the "graid now in
> kernel, old metadata being read, graid taking control of disks it
> shouldn't be" issue.
>
> You need to manually clear the RAID metadata from all your non-RAID (ie,
> used for ZFS) disks.
>
> See the graid man page for details. And the -stable mailing list archives.
>
> Once you've cleared the RAID metadata from the ZFS disks, ZFS will be
> able to pick them up correctly.
>
> --
> Freddie Cash
> fjw...@gmail.com <mailto:fjw...@gmail.com>

I'll look into it.

It should only be ada2 and ada2 that are in a raid configuration, if I
remember correctly ;-)

Jeremy Chadwick

unread,
May 22, 2013, 2:30:42 PM5/22/13
to
On Wed, May 22, 2013 at 07:54:34PM +0200, Leslie Jensen wrote:
>
>
> 2013-05-22 17:02, Ronald Klop skrev:
> >On Wed, 22 May 2013 12:53:25 +0200, Leslie Jensen <les...@eskk.nu> wrote:
> >
> >>
> >>I've upgraded a machine with freebsd-update from 8.3 to 9.1.
> >>
> >>After the first restart I edited /etc/fstab in single user mode
> >>because the names on the disks had changed. But the zfs pool I have
> >>seem to have a problem and I'm sure on how to recover it.
> >>
> >>
> >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0.
> >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1.
> >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4.
> >>May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5.
> >>
> >>
> >>camcontrol devlist
> >><ST3320620AS 3.AAE> at scbus0 target 0 lun 0 (ada0,pass0)
> >><ST3320620AS 3.AAE> at scbus1 target 0 lun 0 (ada1,pass1)
> >><WDC WD360GD-00FNA0 35.06K35> at scbus2 target 0 lun 0 (ada2,pass2)
> >><WDC WD360GD-00FNA0 35.06K35> at scbus2 target 1 lun 0 (ada3,pass3)
> >><ST3320620AS 3.AAE> at scbus3 target 0 lun 0 (ada4,pass4)
> >><ST3320620AS 3.AAE> at scbus3 target 1 lun 0 (ada5,pass5)
> >>
> >>ada2 and ada3 are configured as raid/r0 holding the system.
> >>
> >>ada0, ada1, ada4 and ada5 should be a zfs pool with /home
> >>
> >>
> ada0 at siisch0 bus 0 scbus0 target 0 lun 0
> ada0: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
> ada0: Command Queueing enabled
> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada0: Previously was known as ad4
> ada1 at siisch1 bus 0 scbus1 target 0 lun 0
> ada1: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
> ada1: Command Queueing enabled
> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada1: Previously was known as ad6
> ada2 at ata0 bus 0 scbus2 target 0 lun 0
> ada2: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
> ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada2: Previously was known as ad0
> ada3 at ata0 bus 0 scbus2 target 1 lun 0
> ada3: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
> ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada3: Previously was known as ad1
> ada4 at ata1 bus 0 scbus3 target 0 lun 0
> ada4: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada4: Previously was known as ad2
> ada5 at ata1 bus 0 scbus3 target 1 lun 0
> ada5: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada5: Previously was known as ad3
> SMP: AP CPU #1 Launched!
> Timecounter "TSC-low" frequency 12469046 Hz quality 1000
> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created.
> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created.
> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Array started.
> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from
> STARTING to BROKEN.
> GEOM_RAID: SiI-100311082330: Force array start due to timeout.
> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state
> changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state
> changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Array started.
> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from
> STARTING to BROKEN.
> Trying to mount root from ufs:/dev/raid/r0s1a [rw]...
> ZFS filesystem version 5
> ZFS storage pool version 28
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
I think I see what the issue is here. This is going to be a long Email.

You have 6 disks:

ada0 -- attached to siisch0 (Silicon Image 3132)
ada1 -- attached to siisch1 (Silicon Image 3132)
ada2 -- attached to ata0 (Intel ICH7, non-RAID) as master
ada3 -- attached to ata0 (Intel ICH7, non-RAID) as slave
ada4 -- attached to ata1 (Intel ICH7, non-RAID) as master
ada5 -- attached to ata1 (Intel ICH7, non-RAID) as slave

However, **BEFORE** ZFS starts, we can clearly see this (take the time
to read it slowly and carefully)

> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created.
> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created.
> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Array started.
> GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from STARTING to OPTIMAL.
> GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created.
> GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created.
> Root mount waiting for: GRAID-SiI GRAID-Intel
> ---
> --- this message repeats a very long time due to a problem
> --- I'm about to try and explain
> ---
> GEOM_RAID: Intel-165267ff: Force array start due to timeout.
> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Array started.
> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from STARTING to BROKEN.
> GEOM_RAID: SiI-100311082330: Force array start due to timeout.
> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Array started.
> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from STARTING to BROKEN.
> Trying to mount root from ufs:/dev/raid/r0s1a [rw]...

This shows ***THREE*** graid(8) arrays being probed during disk tasting
for RAID metadata.

Remember two things:

1) graid(8) metadata tasting happens ***BEFORE*** ZFS metadata tasting.
2) The metadata is stored *on the disks themselves*.

- Intel-165267ff
-- disk ada0
-- disk ada1
- Intel-841a1a00
-- disk ada2
-- disk ada3
- SiI-100311082330 (used to be part of a RAID-5 set)
-- disk ada4
-- disk ada5

All of these disks become "locked" by graid because of the discovery of
metadata on them.

Plain and simple:

1) ada0 and ada1 were previously used on an Intel controller that had
BIOS-level RAID capability, and (presumably you) never cleared the
metadata from the disks before making use of ZFS,

2) ada4 and ada5 were previously used on a Silicon Image controller that
had RAID capability, and (presumably you) never cleared the metadata
from the disks before making use of ZFS.

Your next question is going to be "how to do I do that?"

There are many ways to do this, and the one I'm going to tell you is the
one that will work reliably without having to ""fight"" with GEOM or
graid(8). So hear me loud and clear.

1. Boot a FreeBSD installation CD/DVD/USB stick
2. Escape to the loader prompt
3. Type: set kern.geom.raid.enable="0"
4. Type: boot
5. Go to a "Fixit" shell (the one that has all the useful utilities)
6. Zero the first and last 8MBytes or so (I'm not going to be accurate)
of all the disks whose RAID controller metadata you want to remove.

Apologies for the caps, but I need to make this clear:

BE AWARE THIS WILL DESTROY PARTITION TABLES AND OVERWRITE/NUKE ANY OTHER
KIND OF METADATA (INCLUDING ZFS), SO IF YOU HAVE DATA ON THESE DRIVES
THAT YOU CARE ABOUT (SOMEHOW?), YOU HAD BETTER GET THE DATA OFF FIRST.

FOR ZFS YOU WILL ALSO NEED TO RE-CREATE THE POOL AFTER THIS (AND IF THEY
ARE 4K SECTOR DISKS YOU WILL NEED TO USE THE GNOP METHOD TO ENSURE
PROPER ALIGNMENT; SEE IVAN VORAS' BLOG).

The procedure to do that, for your disks (given their capacities) is
this:

> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)

sysctl kern.geom.debugflags=16

zpool import -a
-- I am hoping your pool get imported successfully at this point.
-- This might not even be needed depending upon what the kernel
-- does during its earlier boot.

zpool destroy {poolname}
-- Yes, you really do want to do this. This is how you will get
-- ZFS to "unlock" the disks that you're about to nuke metadata
-- on. If you have multiple pools, you need to do this for
-- all of them.

dd if=/dev/zero of=/dev/ada0 bs=65536 count=128
dd if=/dev/zero of=/dev/ada0 bs=65536 count=128 oseek=4880000
dd if=/dev/zero of=/dev/ada1 bs=65536 count=128
dd if=/dev/zero of=/dev/ada1 bs=65536 count=128 oseek=4880000
dd if=/dev/zero of=/dev/ada4 bs=65536 count=128
dd if=/dev/zero of=/dev/ada4 bs=65536 count=128 oseek=4880000
dd if=/dev/zero of=/dev/ada5 bs=65536 count=128
dd if=/dev/zero of=/dev/ada5 bs=65536 count=128 oseek=4880000

On the commands with "oseek", you will probably receive an error that
says something about "short end of device" or what not; that is
perfectly fine.

After this, I strongly recommend rebooting the system ("reboot")
and booting off of your standard hard disks. Your system should
start in multi-user, but obviously there will be no ZFS pools.

At this point, recreate your ZFS pools as you did originally, and you
should be good to go going forward.

--
| Jeremy Chadwick j...@koitsu.org |
| UNIX Systems Administrator http://jdc.koitsu.org/ |
| Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |

Leslie Jensen

unread,
May 22, 2013, 4:02:14 PM5/22/13
to
Thank you very much Jeremy.

Yes you are right, the machine was a Windows home Server and I changed
it to BSD :-)

In that process I rearranged the disks because they where set up in a
strange way, and YES they where already in a RAID configuration when I
reused and reconnected the disks to have a more "correct" setup.

The machine has been working for more that two years but when I started
upgrading the Motherboard battery was out and that of course reset the
BIOS. I do not remember how I configured the disks then (which I will
document now so that I'll not make the same mistake once more)

Luckily the system RAID is working so I can try to fix the problem as
you describe.

I might even get new and larger disks in the same time, and maybe even a
genuine SATA III controller.

/Leslie
0 new messages