ZFS fails with error 6 on upgrade to 11.1

[Sebastian Schwarz]

Hello,

With 11.1 just officially announced I decided to give it a try
on my notebook.

After creating a boot environment I upgraded from
11.0-RELEASE-p11 to 11.1-RELEASE-p0 following the instructions
outlined in
https://www.freebsd.org/releases/11.1R/installation.html:

# freebsd-update fetch install
# freebsd-update -r 11.1-RELEASE upgrade
[manually merge /etc/mail/mailer.conf]
[manually merge /etc/syslog.conf]
# freebsd-update install
# shutdown -r now

At this point the boot process errored out with the following
error message:

> Mounting from zfs:ssd/ROOT/freebsd11.1.0 failed with error 6.

The following prompt lets me list all available devices. All
the drives for the ZFS pools are present with the same names as
in `zpool status`.

After that I rebooted and selected the previously created boot
environment with 11.0p11 and am trying to figure out the
problem. However as I'm fairly new to FreeBSD I'm in need of
some help.

Some background information on my setup:

- booted with EFI
- the drives are partitioned with GPT
- the two ZFS pools where originally created on Arch Linux
- from the FreeBSD installer I successfully installed into a
separate dataset
- 11.0 has been working fine like this since December 2016

More details:

- dmesg: http://ix.io/yHR
- gpart list: http://ix.io/yHT
- zpool status: http://ix.io/yHZ
- zfs list: http://ix.io/yHQ

Some advice would be very much appreciated.

Thanks in advance,
Sebastian
_______________________________________________
freebsd-...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questi...@freebsd.org"

[Trond Endrestøl]

Off the top of my head I would ensure zfs.ko is loaded at boot time,
by adding zfs_load="YES" to /boot/loader.conf within the
ssd/ROOT/freebsd11.1.0 filesystem.

Make sure the boot pool is in its imported state, and if you use
/boot/zfs/zpool.cache at all, make sure it's up-to-date.

--
Trond.

[Sebastian Schwarz]

On 2017-07-27, Trond Endrestøl wrote:
> Off the top of my head I would ensure zfs.ko is loaded at boot time,
> by adding zfs_load="YES" to /boot/loader.conf within the
> ssd/ROOT/freebsd11.1.0 filesystem.

zfs_load="YES" is already present in loader.conf.

> Make sure the boot pool is in its imported state, and if you use
> /boot/zfs/zpool.cache at all, make sure it's up-to-date.

I tried both updating and disabling the cachefile, but in both
cases I still get the same error:

> Mounting from zfs:ssd/ROOT/freebsd11.1.0 failed with error 6.

[Sebastian Schwarz]

On 2017-07-27, Sebastian Schwarz wrote:
> On 2017-07-27, Trond Endrestøl wrote:
>> Off the top of my head I would ensure zfs.ko is loaded at boot time,
>> by adding zfs_load="YES" to /boot/loader.conf within the
>> ssd/ROOT/freebsd11.1.0 filesystem.
>
> zfs_load="YES" is already present in loader.conf.

And the module does get loaded during boot, as indicated by the
following output during the boot:

> ZFS filesystem version: 5
> ZFS storage pool version: features support (5000)

And the hard drives themselves are detected as well.

[Goran Tepšić]

Just from top of my head, you mentioned that pool is created in Arch? Is
your pool upgraded? Maybe the difference in zpool features between FreeBSD
and ZoL is the culprit.

Try booting your OS or livecd like mfsBSD to import pool to alternate root
and running 'zpool upgrade ssd', maybe that'll do the trick.

I don't know anything about Linux ZFS system but I'd expect them t be
incompatible afterall.

Good luck

> To unsubscribe, send any mail to "freebsd-questions-
> unsub...@freebsd.org"

[Sebastian Schwarz]

On 2017-07-28 Goran Tepšić wrote:
> Try booting your OS or livecd like mfsBSD to import pool to alternate root
> and running 'zpool upgrade ssd', maybe that'll do the trick.

I gave that a try:

> # zpool upgrade ssd
> This system supports ZFS pool feature flags
>
> Enabled the following features on 'ssd':
> multi_vdev_crash_dump
> sha512
> skein
> # zpool upgrade hdd
> This system supports ZFS pool feature flags
>
> Enabled the following features on 'ssd':
> multi_vdev_crash_dump
> sha512
> skein

Then I rebooted but I'm still getting the same error.

> I don't know anything about Linux ZFS system but I'd expect
> them t be incompatible afterall.

Well, I'm booting off the pool with 11.0 just fine. It's 11.1
that's giving me trouble.

[Goran Tepšić]

Hmm. Can you do 'zfs get bootfs,cachefile ssd'?

> To unsubscribe, send any mail to "freebsd-questions-
> unsub...@freebsd.org"

[Sebastian Schwarz]

On 2017-07-28, Goran Tepšić wrote:
> Hmm. Can you do 'zfs get bootfs,cachefile ssd'?

Sure:

> # zpool get bootfs,cachefile ssd
> NAME PROPERTY VALUE SOURCE
> ssd bootfs ssd/ROOT/freebsd11.0.11 local
> ssd cachefile - default

That bootfs is my currently working dataset. I selected the
dataset with 11.1 using the boot loader's menu.

I tried setting bootfs to `ssd/ROOT/freebsd11.1.0` to see if
that would make a difference, but it didn't.

[Goran Tepšić]

Couple of days ago I had issue with upgrading to 11.1 where I tried a bit
exotic way of upgrading by creating boot environment, mounting it and
running freebsd-update to send files to that boot environment's directory.
After I tried to boot, ZFS complained.

I tried couple of things to recover but I think 'zfs set cachefile
/boot/zfs/zpool.cachefile' actually saved my 🥓. Not entirely positive but
worth trying.

[Sebastian Schwarz]

I performed a fresh install of 11.0-RELEASE into a new boot
environment as well as another one of 11.1-RELEASE. The 11.0
install boots, the 11.1 one doesn't. Both were created using
the same procedure. Therefore I presume it to be a bug in 11.1
and have filed a PR accordingly:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221075

Still any ideas to get 11.1 to boot off my ZFS pool are very
much appreciated.

[Goran Tepšić]

Bummer. Let me know when you sort it out, I'd like to know what happened in
case i need it sometimes god forbid :)

> To unsubscribe, send any mail to "freebsd-questions-
> unsub...@freebsd.org"

[Matthias Fechner]

Am 28.07.2017 um 23:01 schrieb Sebastian Schwarz:
>> # zpool upgrade hdd
>> This system supports ZFS pool feature flags
>>
>> Enabled the following features on 'ssd':
>> multi_vdev_crash_dump
>> sha512
>> skein
> Then I rebooted but I'm still getting the same error.
>

have you also upgraded your bootloader?

Gruß
Matthias

--

"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook

[Sebastian Schwarz]

On 2017-07-29, Matthias Fechner wrote:
> have you also upgraded your bootloader?

Yes, I copied the new /boot/boot1.efi to my ESP and selected
that when booting with rEFInd.

[Sebastian Schwarz]

On 2017-07-29, Goran Tepšić wrote:
> Let me know when you sort it out, I'd like to know what
> happened in case i need it sometimes god forbid :)

I'm beginning to think that my ZFS pool and datasets are just
fine. Booting the 11.1 installer from a USB drive allows me to
access all the datasets from my pool without any problems.

I guess something is wrong with the device discovery during
early boot. Which is confusing, since all the ZFS pool's
devices are present in the "List of GEOM managed disk devices"
(printed by pressing "?" at the "mountroot>" prompt).