Can't use gpt labels re-importing pool
Arnaud Houdelette
Under 7.2 the pool was like :
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
ad6p1 ONLINE 0 0 0
ad10p1 ONLINE 0 0 0
ad8p1 ONLINE 0 0 0
ad4p1 ONLINE 0 0 0
I'd like to use gpt labels or gptids reimporting the pool.
# ls /dev/gpt
disk1 disk2 disk3 disk4
# ls /dev/gptid
0902db4e-d462-11de-96bd-001d923bc7a0 9fb111f8-d426-11de-99bc-001d923bc7a0
1be29091-d9dc-11de-9f4a-001d923bc7a0 ffb4e96a-d497-11de-96bd-001d923bc7a0
I did # zpool import -d /dev/gpt tank
and I got
tank ONLINE
raidz1 ONLINE
ada1p1 ONLINE
ada3p1 ONLINE
ada2p1 ONLINE
gpt/disk4 ONLINE
Now, how to force zpool import to only use gpt labels (or uuids) ? Or at
least get back to coherent situation ( zpool export tank / zpool import
gives the same now. and zpool import -d /dev, as zpool import -d
/dev/gptid gives nothing ) ?
Thanks in advance.
Arnaud
Scot Hetzel
Use 'zpool replace' to change the zfs pool to use the gpt names:
zpool replace tank ada1p1 gpt/disk1
zpool replace tank ada2p1 gpt/disk2
zpool replace tank ada3p1 gpt/disk3
NOTE: make sure you use the correct gpt names for each partition, as
the above assumes that ada1p1 is labeled as disk1.
Scot
Jeremy Chadwick
I'm a bit curious about something, so maybe someone can help me
understand:
Why are people bothering with GPT labels (or in some cases, glabels)
when AHCI (whether it be ataahci.ko or ahci.ko) is in use? Under what
circumstance would the device name change dynamically in this situation?
I've never witnessed this happening with AHCI, at least on Intel
systems, and I've hot-swapped hard disks many times over.
--
| Jeremy Chadwick j...@parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
Tom Evans
<fre...@jdc.parodius.com>wrote:
> I'm a bit curious about something, so maybe someone can help me
> understand:
>
> Why are people bothering with GPT labels (or in some cases, glabels)
> when AHCI (whether it be ataahci.ko or ahci.ko) is in use? Under what
> circumstance would the device name change dynamically in this situation?
>
> I've never witnessed this happening with AHCI, at least on Intel
> systems, and I've hot-swapped hard disks many times over.
>
>
My home server has 6 x ICH10 SATA ports using ahci(4), and 2 x SiL 3128 SATA
ports using siis(4). When I first set it up, I created a raidz pool using
MBR/BSD slices/partitions on the drives on the ahci controllers, (ie zpool
create tank raidz ada[0-5]s1d). I then shutdown, connected a couple of
drives to the siis controller, and booted up again. This caused the pool to
fail to be imported, as the drives on siis came up as ada0 and ada1.
I then wiped out the pool, and restarted the install, but this time using
GPT partitioning and labelling each partition that I use. Now I can connect
my drives on any interface, any order and it works correctly, always. I also
get a nice label for each drive that I can scribble on the drive cage, and I
can tell exactly what physical device is referred to by a label.
The only cost to this was having to remember to label the drives - well
worth it imo.
Cheers
Tom
Scot Hetzel
> I'm a bit curious about something, so maybe someone can help me
> understand:
>
> Why are people bothering with GPT labels (or in some cases, glabels)
> when AHCI (whether it be ataahci.ko or ahci.ko) is in use? Under what
> circumstance would the device name change dynamically in this situation?
>
There was a thread about this problem where the drives had changed
their device names due to a change in the kernel drive (Current list
from July):
http://lists.freebsd.org/pipermail/freebsd-current/2009-July/009377.html
Through out this thread there were various suggests on how he could
recover the system, and prevent the problem from occurring in the
future.
One of the suggestions was that use of zpool replace to change from
device names to using glabel labels.
http://lists.freebsd.org/pipermail/freebsd-current/2009-July/009440.html
> I've never witnessed this happening with AHCI, at least on Intel
> systems, and I've hot-swapped hard disks many times over.
>
Using glabels, gpt labels, or gptid solves the problem of not needing
to remember which device name the drive originally had. For instance,
on a zfs zraid pool with 3 disks (ad1 ad2 ad3) you could disconnect
the pool from one computer and connect it to another system and it
wouldn't matter which order you reconnected the drives (1 2 3 or 3 1
2) as the pool would still be recognized when it is imported on the
new system. Also if the new system already has drives ad1 and ad2, it
wouldn't matter.
Scot
Arnaud Houdelette
>>
>
> Yes, it will resilver the drive, but it shouldn't take as long as it
> will recognize that it is the same drive. Just wait for the resilver
> process to finish before changing the next drive.
>
>
>> Moreover, I can't use replace as when adaXp1 is used, the corresponding
>> /dev/gpt entry disapears. Unless you mean that I can use replace before
>> import ?
>>
>>
>
> There was a discussion about this back in July:
>
> http://lists.freebsd.org/pipermail/freebsd-current/2009-July/009440.html
>
> In that discussion, they had used glabel to label each drive, and then
> use zpool replace on the active pool to change it to use the label.
>
> Just make sure that you use the gpt name that you assigned to ada1p1,
> when replacing it.
>
> Scot
>
That does not work or there's something I'm doing wrong.
# uname -a
FreeBSD carenath.tzim.net 8.0-RELEASE FreeBSD 8.0-RELEASE #26: Wed Nov
25 23:43:22 CET 2009
tz...@carenath.tzim.net:/usr/obj/usr/src/sys/CARENATH amd64
As I said, the label is not present in /dev/gpt while used with adaXp1 (
here ada1p1 has label HD753LJ-1) :
# ls /dev/gpt
HD753LJ-4 zfs-boot
# zpool replace tank ada1p1 gpt/HD753LJ-1
cannot open 'gpt/HD753LJ-1': no such GEOM provider
If I offline the drive first, I get another error :
# zpool offline tank ada1p1
# zpool replace tank ada1p1 gpt/HD753LJ-1
invalid vdev specification
use '-f' to override the following errors:
/dev/gpt/HD753LJ-1 is part of active pool 'tank'
# zpool replace -f tank ada1p1 gpt/HD753LJ-1
invalid vdev specification
the following errors must be manually repaired:
/dev/gpt/HD753LJ-1 is part of active pool 'tank'
I know there is the solution of cleaning the disk and resilvering, but
if I could avoid this ?
Arnaud
Edho P Arief
for some reasons it sounds to me like 'avoiding problem' since device
name shouldn't matter in zfs (or so I read)
--
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Ollivier Robert
>As I said, the label is not present in /dev/gpt while used with
>adaXp1 ( here ada1p1 has label HD753LJ-1) :
># ls /dev/gpt
>HD753LJ-4 zfs-boot
># zpool replace tank ada1p1 gpt/HD753LJ-1
>cannot open 'gpt/HD753LJ-1': no such GEOM provider
>I know there is the solution of cleaning the disk and resilvering,
>but if I could avoid this ?
Do you load glabel in loader.conf?
I think that in order to be able to use GPT labels in GEOM (and thus ZFS), you set them with gpart, glabel will "load" them in the kernel and you then can use them for zfs.
I recently moved from ata to ahci and wanted to use labels for the swap partitions. So I added a label to my swap partitions with gpart, they got recognized by geom thru glabel and I was able to use /dev/gpt/swapN instead of the usual /dev/adNp2 that were there before.
--
Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- rob...@keltia.freenix.fr
In memoriam to Ondine : http://ondine.keltia.net/
Ivan Voras
> for some reasons it sounds to me like 'avoiding problem' since device
> name shouldn't matter in zfs (or so I read)
Theoretically, you should be right, but the details are still fuzzy. At
the very least, the sequence "zpool export" - shuffle drives around -
"zpool import" should work without problems, though ZFS might pick up
different drive names if multiple labels are present; for example if
using partitions, some of the drives in the pool could be imported as
"adXp1" and others as "gptid/xxxx..." so a manual intervention might be
needed to keep things consistent.
Nikolay Denev
On Nov 26, 2009, at 1:56 PM, Ivan Voras wrote:
> Jeremy Chadwick wrote:
>
>> Why are people bothering with GPT labels (or in some cases, glabels)
>> when AHCI (whether it be ataahci.ko or ahci.ko) is in use? Under what
>> circumstance would the device name change dynamically in this situation?
>> I've never witnessed this happening with AHCI, at least on Intel
>> systems, and I've hot-swapped hard disks many times over.
>
> It isn't specific to AHCI, but this is where most people encountered it first. The issue in question is "how to make ZFS survive drive renaming" for any cause - driver change, controller change, drive shuffling, etc.
>
> In general, ZFS will taste individual drives on "zpool import" so will try to do the right thing, but it might pick up different labels for different drives, etc. Using glabel tricks simply makes the naming a bit more consistent.
>
>
What about "zpool import -d /dev/gpt/" ?
Though last time I tried this it refused to work for some reason.
Regards,
Niki Denev
Jeremy Chadwick
> Jeremy Chadwick wrote:
>
> >when AHCI (whether it be ataahci.ko or ahci.ko) is in use? Under what
> >circumstance would the device name change dynamically in this situation?
> >
> >I've never witnessed this happening with AHCI, at least on Intel
> >systems, and I've hot-swapped hard disks many times over.
>
> it first. The issue in question is "how to make ZFS survive drive
> renaming" for any cause - driver change, controller change, drive
> shuffling, etc.
>
> In general, ZFS will taste individual drives on "zpool import" so
> will try to do the right thing, but it might pick up different
> labels for different drives, etc. Using glabel tricks simply makes
> the naming a bit more consistent.
What I'm having trouble understanding is where the "more consistent"
concept comes from. I guess it ultimately depends on one's system
configuration.
For example: Tom Evans' situation greatly benefits from use of labels,
since his configuration consists of 1) multiple SATA controllers, and 2)
moving drives around between different controllers.
This isn't going to happen in most production server environments, where
the there is a static number of disks and (usually) a single controller.
This is the "situation" I was referring to; "environment" or
"configuration" would have been a more accurate choice of word.
On 4-disk Supermicro systems (Intel ICHx + AHCI based, with hot-swap
drive bays, with FreeBSD 7.x/8.x and ataahci), depending on what ICHx
ports the drives are plugged into, your drive bays/disks are going to be
static/consistent. SATA port #0 = ad4, SATA port #1 = ad6, and so on.
These won't change unless you do something like, say, disable a SATA
port or disable AHCI mode in the BIOS.
Regardless, I'm almost certain I've made the mistake on a 4-disk system
of doing (physical) system cleaning, which involves dusting out all of
the bays and so on -- and ended up inserting a drive/carrier/disk into a
bay which it wasn't part of prior to the shutdown. "zpool import" took
care of things for me.
If someone wants me to validate my own memory (the more I work Grave
shift the more I find my memory failing me...), I'd be more than happy
to spend a weekend messing around moving disks + reporting back what the
behaviour is on RELENG_8. I have a "spare" 5-disk (6 ports) system
which can be used for this purpose (Supermicro X7SBA + 5 disks). I
spend most of my time messing around with disks at my night job as is...
:-)