how to configure terabyte raid5
x
I'm trying to build a terabyte raid5 NFS OpenBSD 4.0 i386 server, but
have run into the terabyte size limit documented in
http://www.openbsd.org/faq/faq14.html#LargeDrive
I've got four 400g Seagate drives, so get something like 1.2T when
configured as raid5 device. I'm not using one as a hot spare.
Can I override the physical geometry and convince diskedit the drives
are smaller?
The raid5 drive is not intended to be bootable, if that makes a
difference.
I was hoping to go w/ OpenBSD, but now am wondering if FreeBSD would
be simpler.
Thanks!
Josh Grosse
I don't know what architecture you are using, but if it uses MBRs,
OpenBSD's fdisk command will let you describe chs values up to 2016GB on
the command line (-c -h -s). After creating the A6 partition, disklabel
will let you set up partitions without any special requirements. Note
that if you are setting up FFS partitions, limit them to no larger than
the maximum sector size described in FAQ 14.
(I tested this with i386 under qemu and a virtual 2TB hard drive with
cd40.iso -- the bsd.rd install kernel.)
--
Replying directly will get you locally blacklisted.
Change the address; use my first name in front of the @ if you want to
communicate privately.
Josh Grosse
> On Tue, 12 Dec 2006 08:54:26 -0700, x wrote:
>
>> Greetings,
>>
>> I'm trying to build a terabyte raid5 NFS OpenBSD 4.0 i386 server
I missed that you were using i386. So try using fdisk with the
appropriate chs values. Max cylinder value is 262144, max head is 256,
and max sector is 63: 4,227,858,432 512-byte blocks, or 2016GB.
x
<spam...@jggimi.homeip.net> wrote:
>On Tue, 12 Dec 2006 23:09:05 -0500, Josh Grosse wrote:
>
>> On Tue, 12 Dec 2006 08:54:26 -0700, x wrote:
>>
>>> Greetings,
>>>
>>> I'm trying to build a terabyte raid5 NFS OpenBSD 4.0 i386 server
>
>I missed that you were using i386. So try using fdisk with the
>appropriate chs values. Max cylinder value is 262144, max head is 256,
>and max sector is 63: 4,227,858,432 512-byte blocks, or 2016GB.
Thanks for the help. Amazingly enough, I'm having a tough time
getting physical geometry for Seagate drive! Their website only lists
logical LBA:
400 gig drive: model st3400633a
http://www.seagate.com/support/disc/specs/ata/st3400633a.html
Cylinders: 1024
Heads: 256
Sectors: 63
Which obviously isn't correct... Asking their tech support has so far
been a waste of time: they just sent me back the above useless
values...
Any idea how I can derive correct geometry? I'm going to try Knoppix
and see if it can figure things out...
Thanks!
Josh Grosse
> Thanks for the help. Amazingly enough, I'm having a tough time
> getting physical geometry for Seagate drive! Their website only lists
> logical LBA:
>
> 400 gig drive: model st3400633a
>
> http://www.seagate.com/support/disc/specs/ata/st3400633a.html
>
> Cylinders: 1024
> Heads: 256
> Sectors: 63
>
> Which obviously isn't correct... Asking their tech support has so far
> been a waste of time: they just sent me back the above useless
> values...
>
> Any idea how I can derive correct geometry? I'm going to try Knoppix
> and see if it can figure things out...
With modern IDE drives, physical device geometry has nothing to do with
what the drive electronics report to the BIOS, or what the BIOS reports to
the OS, or what the OS uses. If you think about it, 256 heads = 128
platters, which would be physically ridiculous.
All of these various translations are to get around various limitations
in CHS values. On that seagate webpage, you will note it tells you about
various BIOS cylinder limitations. That's where CHS translation comes
into play. Then, below all of those BIOS warnings, you will see:
Default CHS translation:
16,383 cyl, 16 heads, 63 sectors = 8,455,200,780
The number of sectors is the key value. Everything else is translation.
In your particular situation, if I understood correctly, you will be using
a virtual device anyway, as you are presenting a RAID set to OpenBSD.
x...@aol.com
Josh, you rock! What I wound up doing was configure each drive in the
array to only use a little under 340 gig, so that when mulitplied by
three would be just less than a terabyte. I think my earlier attempt
at using the entire 400 gig created a partition > 1T, which OpenBSD
choked on.
I've configured it to talk to the APC SmartUPS 700, got Samba hooked
up so my Windows machine can see the raid 5 partition, and things look
pretty good!
The one final bit of confusion was running "raidctl -A root raid0" on
the raid5 array because I was following web instructions for setting
up a bootable raid1. That may have been the original problem, but
things seem stable now.
I see "raid0: Device already configured!" coming up during the boot,
but it doens't seem to be causing any problems. I tried running
"raidctl -A no raid0" but I don't think that fixed the problem.
Thanks again for your help!
OpenBSD 4.0 (GENERIC) #0: Tue Dec 19 12:26:11 MST 2006
root@xxx:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Pentium(R) 4 CPU 1.60GHz ("GenuineIntel" 686-class)
1.62 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM
real mem = 402223104 (392796K)
avail mem = 357855232 (349468K)
using 4256 buffers containing 20213760 bytes (19740K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(cd) BIOS, date 10/29/01, BIOS32 rev. 0 @
0xfafb0, SMBIOS rev. 2.2 @ 0xf0800 (36 entries)
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
apm0: flags 70102 dobusy 1 doidle 1
pcibios0 at bios0: rev 2.1 @ 0xf0000/0xdf84
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdee0/160 (8 entries)
pcibios0: PCI Exclusive IRQs: 5 10 11 12
pcibios0: PCI Interrupt Router at 000:31:0 ("Intel 82371SB ISA" rev
0x00)
pcibios0: PCI bus #2 is the last bus
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x2200 0xcb000/0x2800
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82845 Host" rev 0x03
ppb0 at pci0 dev 1 function 0 "Intel 82845 AGP" rev 0x03
pci1 at ppb0 bus 1
ppb1 at pci0 dev 30 function 0 "Intel 82801BA AGP" rev 0x12
pci2 at ppb1 bus 2
pciide0 at pci2 dev 0 function 0 "HighPoint HPT302 IDE" rev 0x02: DMA
pciide0: using irq 11 for native-PCI interrupt
wd0 at pciide0 channel 0 drive 0: <ST3400633A>
wd0: 16-sector PIO, LBA48, 381554MB, 781422768 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
wd1 at pciide0 channel 1 drive 0: <ST3400633A>
wd1: 16-sector PIO, LBA48, 381554MB, 781422768 sectors
wd1(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 5
pciide1 at pci2 dev 1 function 0 "HighPoint HPT302 IDE" rev 0x02: DMA
pciide1: using irq 10 for native-PCI interrupt
wd2 at pciide1 channel 0 drive 0: <ST3400620A>
wd2: 16-sector PIO, LBA48, 381554MB, 781422768 sectors
wd2(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5
wd3 at pciide1 channel 1 drive 0: <ST3400633A>
wd3: 16-sector PIO, LBA48, 381554MB, 781422768 sectors
wd3(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5
vga1 at pci2 dev 2 function 0 "S3 Trio32/64" rev 0x54
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
rl0 at pci2 dev 3 function 0 "Realtek 8139" rev 0x10: irq 12, address
00:07:95:17:6a:80
rlphy0 at rl0 phy 0: RTL internal PHY
ichpcib0 at pci0 dev 31 function 0 "Intel 82801BA LPC" rev 0x12
pciide2 at pci0 dev 31 function 1 "Intel 82801BA IDE" rev 0x12: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
wd4 at pciide2 channel 0 drive 0: <ST320011A>
wd4: 16-sector PIO, LBA, 19092MB, 39102336 sectors
wd4(pciide2:0:0): using PIO mode 4, Ultra-DMA mode 5
pciide2: channel 1 disabled (no drives)
ichiic0 at pci0 dev 31 function 3 "Intel 82801BA SMBus" rev 0x12: irq
11
iic0 at ichiic0
"it8712f-a" at iic0 addr 0x2d not configured
isa0 at ichpcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
it0 at isa0 port 0x290/8: IT87
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
biomask efe5 netmask ffe5 ttymask ffe7
Kernelized RAIDframe activated
raid0 (root): (RAID Level 5) total number of sectors is 2139095424
(1044480 MB)
pctr: user-level cycle counter enabled
dkcsum: wd0 matches BIOS drive 0x82
dkcsum: wd1 matches BIOS drive 0x81
dkcsum: wd2 matches BIOS drive 0x83
dkcsum: wd3 matches BIOS drive 0x84
dkcsum: wd4 matches BIOS drive 0x80
root on wd4a
rootdev=0x40 rrootdev=0x340 rawdev=0x342
raid0: Device already configured!