changing uuid of mdadm array

[Tim Woodall]

I wanted to change the uuid on a component of a md raid1 array.

I found the following worked and didn't delete any data:
mdadm --create /dev/md1 --level=raid1 --force --raid-devices=1 /dev/sdb3

but I couldn't find any way to change the uuid in the member without
creating the array like this.

This didn't work
mdadm --assemble --update=uuid /dev/md/1
Started the array but didn't change the uuid

This doesn't work at all
mdadm --assemble --update=uuid /dev/sdb3

Is there a good way to do this without starting the array?

[Andy Smith]

Hello,

On Sun, Oct 09, 2022 at 07:15:02PM +0100, Tim Woodall wrote:
> I wanted to change the uuid on a component of a md raid1 array.

The same thing was asked on the linux-raid list last month (though for
RAID-10) and didn't really get an answer:

https://lore.kernel.org/linux-raid/2341a2a9-b86e-f0e5...@thelounge.net/

It went down a rabbit hole of trying to convince the OP to do what
they needed to do in some other way.

I don't think changing the device UUID is possible with any
purpose-built tool but if it is you might want to try
asking about it on linux-raid as it must be a pretty exotic thing to
do. Just prepare to be asked why you want to do it and if there is
some way around it, I guess!

> I found the following worked and didn't delete any data:
> mdadm --create /dev/md1 --level=raid1 --force --raid-devices=1 /dev/sdb3

I know you probably know this, but you didn't say, so for everyone
else:

**********************************************************************

This is extremely dangerous so don't do this unless you have backups
and know exactly what you are doing!

**********************************************************************

While you CAN re-create an MD RAID array like this and not destroy
existing data, you at minimum have to use exactly the same options
to re-create it as you did when you created it. That may sound
simple, but there are many options that are set as defaults and
THOSE DEFAULTS HAVE CHANGED OVER TIME, so this gets harder to get
right the older the array is.

Also, you did not specify --assume-clean, which makes sense because
it isn't relevant to RAID-1, but if someone used that --create
command for any RAID level with stripes or parity, md would
immediately sync over the array and destroy the existing data!

When doing something like this as a rescue attempt it is normally
advised to do it with dm-overlay devices so that changes are
temporarily stored on a small amount of scratch space. This allows
you to assemble the RAID and do basic checks like a mount and fsck
and any destructive writes only hit the temporary storage. If
there's a filesystem on there you'll fail pretty quickly when
mounting and doing a fsck, if the create command is wrong. Once you
know you have a working "mdadm --create" command you can stop the
array and repeat with the real devices.

A warning to never use "--create" on an existing array is on the
front page of the Linux RAID wiki:

https://raid.wiki.kernel.org/index.php/Linux_Raid#When_Things_Go_Wrogn [sic]

> but I couldn't find any way to change the uuid in the member without
> creating the array like this.

I don't think there is a way without either using --create as you
did or stopping the array and hand editing the metadata of the
device in question.

Cheers,
Andy

--
https://bitfolk.com/ -- No-nonsense VPS hosting

[Tim Woodall]

On Sun, 9 Oct 2022, Andy Smith wrote:

>> but I couldn't find any way to change the uuid in the member without
>> creating the array like this.
>
> I don't think there is a way without either using --create as you
> did or stopping the array and hand editing the metadata of the
> device in question.
>

Thanks. That --update=uuid convinced me there ought to be a way to do
it.

The uuid option will change the uuid of the array. If a UUID is given
with the --uuid option that UUID will be used as a new UUID and will NOT
be used to help identify the devices in the array. If no --uuid is
given, a random UUID is chosen.

Perhaps I needed to remove the uuid from mdadm.conf.


FWIW, I didn't need to change the uuid here, I could have just started
from scratch with --zero-superblock but I've previously got into a mess
where I duplicated a machine by splitting a raid where I now had two
raids with the same uuid and this was a good opportunity to see what I
should have done on a disk where I didn't care about the data on it.


Tim.

[Lucas Castro]

Every time you remove a device from raid, you indeed need to erase the
superblock.

mdadm use the superblock to identify its raid members,

if you try to add a device to a raid block it'll fail because the device
already contain a superblock identify at its 'header'. --zero-superblock
it's the right way add to anothor md array.


Following the mdadm manual instruction

"

      The  uuid  option will change the uuid of the array.  If a UUID
is given with the --uuid option that UUID

      will be used as a new UUID and will NOT be used to help identify
the devices in the array.  If no --uuid is given, a random UUID is chosen.

"

The --update=uuid, in my conception will update 'array' uuid, for that
means will change the uuid of all of the array members, not the removed
ones.


>
>
> Tim.
>

[Tim Woodall]

On Sun, 9 Oct 2022, Lucas Castro wrote:

>
> Every time you remove a device from raid, you indeed need to erase the
> superblock.
>
> mdadm use the superblock to identify its raid members,
>
> if you try to add a device to a raid block it'll fail because the device
> already contain a superblock identify at its 'header'. --zero-superblock it's
> the right way add to anothor md array.
>

FTAOD, I'm not trying to add a device to an array, I'm trying to change
the uuid on an array.

>
> Following the mdadm manual instruction
>
> "
>

> ????? The? uuid? option will change the uuid of the array.? If a UUID is

> given with the --uuid option that UUID
>

> ????? will be used as a new UUID and will NOT be used to help identify the
> devices in the array.? If no --uuid is given, a random UUID is chosen.

>
> "
>
>
> The --update=uuid, in my conception will update 'array' uuid, for that means
> will change the uuid of all of the array members, not the removed ones.
>

but define 'array members'? The array is stopped, are they array
members? It's now started using that --update=uuid option, what uuid
should the array have? A new one or the one the members had recorded in
their superblock when it was stopped?

[Lucas Castro]

RAID device is a array  of disk organized to serve as disk device
distributed under physical disk.

Then it's easy to guess 'array members' as all physical devices set to
same array?

The array must be identified somehow, in mdadm, array is by uuid!

>
>
>

[Tim Woodall]

On Wed, 19 Oct 2022, Lucas Castro wrote:

> The array must be identified somehow, in mdadm, array is by uuid!
>

And I have two disks, which have the same uuid but are not part of the
same array.

How do I tell mdadm that they aren't part of the same array?

I can't believe I'm the only person who has cloned a machine by breaking
a raid1 array.

Or someone who has used dd to clone a disk that they then want to mount
in parallel with the original

[Dan Ritter]

mdadm --zero-superblock --force
can be used, carefully.

You will need to re-add the disk to the new array and let it
sync.

-dsr-

[Tim Woodall]

But why would I want to do that? If I wanted it to be part of the array
I'd have added it to the array, not cloned it with dd.

I have an array. I use dd to copy it onto a new disk. I now want to
start the original array (which is unchanged) and I also want to start
the cloned disk as a new array.


Or actually, what caused me grief:

I had a machine with two disks as part of a raid 1 pair.

I wanted to clone the machine, so sdb was taken out and put in a new
machine and two new disks were added as sdb in each machine. Arrays were
rebuilt, tweaks were made to fix up things like ip addresses and then
the two machines slowly diverged.

On one of the machines, one of the disks started reporting issues. The
other machine was getting tight for space, so I bought two new larger
disks, swapped one on the ok machine, rebuilt the array, swapped the
other, rebuilt, resized.

And then... Swapped the going faulty disk with one of the disks taken
out of the other machine - except that mdadm thought they were part of
the same array and things ended up in a terrible mess (partly because I
didn't realize what was going on so made some mistakes with hindsight)

Yes, I should have failed the disk I was going to remove and then zeroed
the superblock before removing it - but I'd completely forgotten that
the two machines had the same uuid for their raids.

So recently, when I ended up with a disk that had been a raid member, I
thought I'd take the opportunity to see how to change the uuid, which is
what I should have done on splitting the array originally.

It surprises me that there isn't an easy safe way to change the uuid,
but, as per my OP, there are 'risky' ways to achieve this if necessary.

[Dan Ritter]

Tim Woodall wrote:
> On Fri, 21 Oct 2022, Dan Ritter wrote:
>
> > Tim Woodall wrote:
> > > On Wed, 19 Oct 2022, Lucas Castro wrote:
> > >
> > > > The array must be identified somehow, in mdadm, array is by uuid!
> > > >
> > >
> > > And I have two disks, which have the same uuid but are not part of the
> > > same array.
> > >
> > > How do I tell mdadm that they aren't part of the same array?
> > >
> > > I can't believe I'm the only person who has cloned a machine by breaking
> > > a raid1 array.
> > >
> > > Or someone who has used dd to clone a disk that they then want to mount
> > > in parallel with the original
> >
> > mdadm --zero-superblock --force
> > can be used, carefully.
> >
> > You will need to re-add the disk to the new array and let it
> > sync.
> >
> > -dsr-
> >
> >
>
> But why would I want to do that? If I wanted it to be part of the array
> I'd have added it to the array, not cloned it with dd.

Sorry, I thought you wanted to move a disk from one array to
another on the same machine.

> I have an array. I use dd to copy it onto a new disk. I now want to
> start the original array (which is unchanged) and I also want to start
> the cloned disk as a new array.

You can't do that, to the best of my knowledge. You can take the
cloned disk to a new machine and use it to create a new array.

> Or actually, what caused me grief:
>
> I had a machine with two disks as part of a raid 1 pair.
>
> I wanted to clone the machine, so sdb was taken out and put in a new
> machine and two new disks were added as sdb in each machine. Arrays were
> rebuilt, tweaks were made to fix up things like ip addresses and then
> the two machines slowly diverged.

Right, that's normal.

> On one of the machines, one of the disks started reporting issues. The
> other machine was getting tight for space, so I bought two new larger
> disks, swapped one on the ok machine, rebuilt the array, swapped the
> other, rebuilt, resized.
>
> And then... Swapped the going faulty disk with one of the disks taken
> out of the other machine - except that mdadm thought they were part of
> the same array and things ended up in a terrible mess (partly because I
> didn't realize what was going on so made some mistakes with hindsight)

You have correctly deduced the problem.

> Yes, I should have failed the disk I was going to remove and then zeroed
> the superblock before removing it - but I'd completely forgotten that
> the two machines had the same uuid for their raids.
>
> So recently, when I ended up with a disk that had been a raid member, I
> thought I'd take the opportunity to see how to change the uuid, which is
> what I should have done on splitting the array originally.
>
> It surprises me that there isn't an easy safe way to change the uuid,
> but, as per my OP, there are 'risky' ways to achieve this if necessary.

The unspoken assumption is that one of three things will happen
when you remove a disk from a RAID:

- the disk went bad and you will discard it / replace it under
warranty

- you are keeping it as a kind of long term archive, properly
labelled, and it takes some kind of disaster for you to bring
it up (on another machine) to look at the contents

- you are splitting the mirror so that you can clone to another
machine, and it will never come back here.

Most of these assumptions hold across RAID-like systems. It
might be of interest to you that ZFS has the "export" and
"import" commands, which are used to prep whole pools to be sent
to another machine. It still won't be very happy with the
precise actions you took here.

-dsr-