pools will not automatically import after Mavericks/MacZFS-74.3.2 upgrades

[xbytor]

When I boot into my updated Mac, I now get a set these messages:

"This disk you inserted was not readable by this computer.
Initialize | Ignore | Eject"

So, I  do a zpool import:

$ sudo zpool import
Password:
  pool: zarchive
    id: 11473564137836478606
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

    zarchive     ONLINE
      mirror     ONLINE
        disk4s2  ONLINE
        disk7s2  ONLINE

etc...

I try to import one pool:
$ sudo zpool import zarchive
Password:
cannot import 'zarchive': pool may be in use from other system
use '-f' to import anyway

along with another "not readable" popup.

I then force the matter:
$ sudo zpool import -f zarchive

This works but I get another "not readable" popup.

Now, do a export/import pair:
$ sudo zpool export zarchive

xbytor@gondor ~
$ sudo zpool import zarchive

This works but I get another "not readable" popup.

When I reboot, the same thing happens all over again. I suspect the fact that I did
not export the pools during the OS upgrade is at the root of the problem but I though
the initial "import -f" would have resolved the issue.

Any ideas on what's going on here and how to fix this mess?

-X

[ilov...@icloud.com]

Is the file system bundle actually installed?

ls -ld /System/Library/Filesystems/zfs.fs

[X Bytor]

On 2013/12/13 5:56 PM, ilov...@icloud.com wrote:
> Is the file system bundle actually installed?
>
> ls -ld /System/Library/Filesystems/zfs.fs

Yes, but the dates on the files look a little suspect.

$ ls -ld /System/Library/Filesystems/zfs.fs
drwxr-xr-x 4 root wheel 136 Sep 22 2011 /System/Library/Filesystems/zfs.fs

$ ls -lR /System/Library/Filesystems/zfs.fs
total 64
drwxr-xr-x 5 root wheel 170 Sep 22 2011 Contents
-rwxr-xr-x 1 root wheel 31608 Jan 9 2011 zfs.util

/System/Library/Filesystems/zfs.fs/Contents:
total 16
-rw-r--r-- 1 root wheel 1670 Jan 9 2011 Info.plist
-rw-r--r-- 1 root wheel 8 Jan 9 2011 PkgInfo
drwxr-xr-x 4 root wheel 136 Sep 22 2011 Resources

/System/Library/Filesystems/zfs.fs/Contents/Resources:
total 392
drwxr-xr-x 3 root wheel 102 Sep 22 2011 English.lproj
-rw-r--r-- 1 root wheel 196764 Jan 9 2011 VolumeIcon.icns

/System/Library/Filesystems/zfs.fs/Contents/Resources/English.lproj:
total 8
-rw-r--r-- 1 root wheel 350 Jan 9 2011 InfoPlist.strings

[ilov...@icloud.com]

what is this output of 

diskutil list

[X Bytor]

On 2013/12/13 7:02 PM, ilov...@icloud.com wrote:
> what is this output of
> diskutil list

Everything looks kosher.

$ diskutil list
/dev/disk0
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *128.0 GB disk0
1: EFI 209.7 MB disk0s1
2: Apple_HFS Macintosh HD 127.2 GB disk0s2
3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *2.0 TB disk1
1: EFI 209.7 MB disk1s1
2: ZFS 2.0 TB disk1s2
/dev/disk2
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *2.0 TB disk2
1: EFI 209.7 MB disk2s1
2: ZFS 2.0 TB disk2s2
/dev/disk3
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk3
1: EFI 209.7 MB disk3s1
2: ZFS 999.9 GB disk3s2
/dev/disk4
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *3.0 TB disk4
1: EFI 209.7 MB disk4s1
2: ZFS 3.0 TB disk4s2
/dev/disk5
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk5
1: EFI 209.7 MB disk5s1
2: ZFS 999.9 GB disk5s2
/dev/disk6
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *500.1 GB disk6
1: EFI 209.7 MB disk6s1
2: Apple_HFS SuperDuper 140.1 GB disk6s2
3: Apple_HFS TimeMachine 287.6 GB disk6s3
/dev/disk7
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *3.0 TB disk7
1: EFI 209.7 MB disk7s1
2: ZFS 3.0 TB disk7s2
/dev/disk8
#: TYPE NAME SIZE IDENTIFIER
0: Apple_partition_scheme *131.1 GB disk8
1: Apple_partition_map 32.3 KB disk8s1
2: Apple_HFS Macintosh HD Backup 131.1 GB disk8s2

[X Bytor]

Also, I did verify that zfs.kext is 74.3.2 built 2013-11-30_23.

[Bjoern Kahl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi, something went wrong with the installation.

Am 14.12.13 07:04, schrieb X Bytor:

> Also, I did verify that zfs.kext is 74.3.2 built 2013-11-30_23.

Am 14.12.13 01:57, schrieb X Bytor:

> On 2013/12/13 5:56 PM, ilov...@icloud.com wrote:
>> Is the file system bundle actually installed?
>>
>> ls -ld /System/Library/Filesystems/zfs.fs
> Yes, but the dates on the files look a little suspect.
>
> $ ls -ld /System/Library/Filesystems/zfs.fs drwxr-xr-x 4 root
> wheel 136 Sep 22 2011 /System/Library/Filesystems/zfs.fs
>
> $ ls -lR /System/Library/Filesystems/zfs.fs total 64 drwxr-xr-x 5
> root wheel 170 Sep 22 2011 Contents -rwxr-xr-x 1 root wheel
> 31608 Jan 9 2011 zfs.util
>

The kext looks right, but the zfs.fs should have the same date. In my
build archive it looks this way:

(ignore the ownership bits, these are only set in the pkg)

drwxr-xr-x 3 bj staff 102 Nov 30 23:26
build/ZFS106/System/Library/Extensions/zfs.kext
- -rwxr-xr-x 1 bj staff 1743336 Nov 30 23:26
build/ZFS106/System/Library/Extensions/zfs.kext/Contents/MacOS/zfs
drwxr-xr-x 3 bj staff 136 Nov 30 23:26
build/ZFS106/System/Library/Filesystems/zfs.fs
- -rwxr-xr-x 1 bj staff 31992 Nov 30 23:26
build/ZFS106/System/Library/Filesystems/zfs.fs/zfs.util


the md5 sums are:
85e8193581630114d7363a3a27a1c2d0
build/ZFS106/System/Library/Extensions/zfs.kext/Contents/MacOS/zfs
c35736fd8a75fdee3916fbd39835849b
build/ZFS106/System/Library/Filesystems/zfs.fs/zfs.util

and "strings build/ZFS106/System/Library/Filesystems/zfs.fs/zfs.util
| grep -e 74"
should give
@(#)PROGRAM:zfs.util PROJECT:maczfs_74-3-2-0-ge6eed38
VERSION:74.3.2 BUILT:2013-11-30_23.25_+0100

But obviously your zfs.util is a different one (31608 v.s 31992 bytes)

Can you try to re-install the package?


You can also look into the package to verify its content. The sha1
sum of the downloaded installer should be
c14890a9bc2016f785ab3328445cd693d10377b1

It's content can be inspected by doing in terminal:

pkgutil --expand MacZFS-74.3.2b.pkg /tmp/t
cd /tmp/t/zfs106.pkg
cpio -i <Payload
ls -l System/Library/Filesystems/zfs.fs/zfs.util


Best regards

Björn

- --
| Bjoern Kahl +++ Siegburg +++ Germany |
| "googlelogin@-my-domain-" +++ www.bjoern-kahl.de |
| Languages: German, English, Ancient Latin (a bit :-)) |
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (Darwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQCVAgUBUqw5YFsDv2ib9OLFAQLqPAQAwfCxsFXNfmCqORgP/pp+POXO5ogLs+Ul
xbSvs+SEKBhguVbyZb3EVmb1o/FqkH9LDiMWbAdMXDBj4RqaALz31dh4SZXYV+N+
a+iv4GAX1OLNpT7G/vH9PbTGQOMHGcxcjEdAP1ixTshR9E1qxIREE8DnT/Fjb0r3
n+MN7YYObnA=
=BAfK
-----END PGP SIGNATURE-----

[X Bytor]

On Sat, Dec 14, 2013 at 4:56 AM, Bjoern Kahl <googl...@bjoern-kahl.de> wrote:

 Hi, something went wrong with the installation.

Figured.
 

 The kext looks right, but the zfs.fs should have the same date.  In my
 build archive it looks this way:

I definitely have the old zfs.fs.
 

 Can you try to re-install the package?

I have multiple times. No happy ending.
 

 You can also look into the package to verify its content.  The sha1
 sum of the downloaded installer should be
c14890a9bc2016f785ab3328445cd693d10377b1

This is what I do have.
 

 It's content can be inspected by doing in terminal:

 pkgutil --expand MacZFS-74.3.2b.pkg /tmp/t
 cd /tmp/t/zfs106.pkg
 cpio -i <Payload
 ls -l System/Library/Filesystems/zfs.fs/zfs.util

This looks good. Perhaps I should just delete the old
/System/Library/Filesystems/zfs.fs and try a reinstall?

-X

[Bjoern Kahl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 14.12.13 21:22, schrieb X Bytor:

> On Sat, Dec 14, 2013 at 4:56 AM, Bjoern Kahl
> <googl...@bjoern-kahl.de>wrote:
>
>

>> Can you try to re-install the package?
>>
>
> I have multiple times. No happy ending.
>
>>
>> You can also look into the package to verify its content. The
>> sha1 sum of the downloaded installer should be
>> c14890a9bc2016f785ab3328445cd693d10377b1
>>
>
> This is what I do have.
>
>>
>> It's content can be inspected by doing in terminal:
>>
>> pkgutil --expand MacZFS-74.3.2b.pkg /tmp/t cd /tmp/t/zfs106.pkg
>> cpio -i <Payload ls -l
>> System/Library/Filesystems/zfs.fs/zfs.util
>>
>>
> This looks good. Perhaps I should just delete the old
> /System/Library/Filesystems/zfs.fs and try a reinstall?

Yes, try deleting the old /System/Library/Filesystems/zfs.fs and
reinstall. If it doesn't help (maybe something in the package
database prevents the installer from installing the zfs.fs?) you can
either try to make OS X forget the package and retry, or copy the
unpacked content manually.

To make OS X forget a package, use the following steps:

- pkgutil --pkgs | grep -e zfs
to find all MacZFS, ZEVO, OSX-ZFS, ... packages.

To see which files OS X think they have install, do a
- pkgutil --files _package_id_from_1st_step_

Then do
- sudo pkgutil --forget _package_id_from_1st_step_
to actually forget them.

Best regards

Björn

- --
| Bjoern Kahl +++ Siegburg +++ Germany |
| "googlelogin@-my-domain-" +++ www.bjoern-kahl.de |
| Languages: German, English, Ancient Latin (a bit :-)) |
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (Darwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQCVAgUBUqzHH1sDv2ib9OLFAQKMmgP/UnOQAGTZvSuCHsz+se2xL0gXjXkUInZA
F/dnhXVVAfsMleMHJeLxhl/lhd/XQoPXyMPmFovuYo2EoGF2iQYF7xihbrFtfGZ8
gzy0DLXoyTKDS+BCEqf29f+Xe6znlqteRCBBWhQeF1Oh/z926tdizV3Tg42zQ1aS
+0o3MLEMb5I=
=6zzY
-----END PGP SIGNATURE-----

[X Bytor]

On 2013/12/14 3:01 PM, Bjoern Kahl wrote:
> -----BEGIN PGP SIGNED MESSAGE-----

> Yes, try deleting the old /System/Library/Filesystems/zfs.fs and
> reinstall.

Didn't work. Crashes in zfs on reboot.

> If it doesn't help (maybe something in the package
> database prevents the installer from installing the zfs.fs?) you can
> either try to make OS X forget the package and retry, or copy the
> unpacked content manually.
>
> To make OS X forget a package, use the following steps:
>
> - pkgutil --pkgs | grep -e zfs
> to find all MacZFS, ZEVO, OSX-ZFS, ... packages.

I get:
com.bandlem.mac.zfs.106.pkg
net.spy.zfs.fs.pkg
net.spy.zfs.kext.pkg
net.spy.zfs.libzfs.pkg
net.spy.zfs.zfs.pkg
net.spy.zfs.zpool.pkg
org.maczfs.zfs.106.pkg

> To see which files OS X think they have install, do a
> - pkgutil --files _package_id_from_1st_step_

Should I manually delete all of these files?

>
> Then do
> - sudo pkgutil --forget _package_id_from_1st_step_
> to actually forget them.
>

Should I delete all of the packages listed above?

-X

[X Bytor]

So, I deleted all of the old files and packages, reinstalled 74.3.2b and
verified (manually)
that the right files were in the right places. Fail to boot due to the same
crash mentioned before.
I'll post up the crash log (or a pic from the screen) tomorrow.

As a fall back, I put the old zfs.fs stuff back in there.

[X Bytor]

More of my adventures.

I rolled back to 10.8.5 boot up and everything is fine. Try to install/reinstall 74.3.2b but the installer will not overwrite the old zfs.fs. I also removed all of the phantom packages. Mavericks boots ok but zfs.kext doesn't load correctly and zfsutil fails because of the version mismatch. I also get com.app.kextd warings aout zfs.kext having an Invalid Signature.

So, I decide to get clever. I move the old zfs.fs aside and reinstall once again. This time I get the new zfs.fs installed correctly. When I reboot, I get to the login screen then the system automagically reboots. This is the crash report:

Anonymous UUID:       080DF86C-BB44-98CF-E6B4-39D68E12A144

Tue Dec 17 20:11:29 2013

panic(cpu 7 caller 0xffffff80128dc19e): Kernel trap at 0xffffff8012e36ed0, type 14=page fault, registers:

CR0: 0x0000000080010033, CR2: 0xffffff8012e36ed0, CR3: 0x0000000058595000, CR4: 0x0000000000000660

RAX: 0xffffff8012e36ed0, RBX: 0xffffff821f7a3d58, RCX: 0x0000000001000000, RDX: 0x0000000000000be8

RSP: 0xffffff821f7a3c88, RBP: 0xffffff821f7a3cf0, RSI: 0x0000000000000000, RDI: 0xffffff803b646000

R8:  0xffffff803971f000, R9:  0xffffff8012e536f0, R10: 0x0000000000000018, R11: 0x0000000000000206

R12: 0x0000000021000000, R13: 0xffffff803ad18f00, R14: 0x0000000021000000, R15: 0xffffff803971f000

RFL: 0x0000000000010216, RIP: 0xffffff8012e36ed0, CS:  0x0000000000000008, SS:  0x0000000000000010

Fault CR2: 0xffffff8012e36ed0, Error code: 0x0000000000000011, Fault CPU: 0x7 Kernel NX fault

Backtrace (CPU 7), Frame : Return Address

0xffffff821f7a3910 : 0xffffff8012822f69 

0xffffff821f7a3990 : 0xffffff80128dc19e 

0xffffff821f7a3b60 : 0xffffff80128f3606 

0xffffff821f7a3b80 : 0xffffff8012e36ed0 

0xffffff821f7a3cf0 : 0xffffff8012a0caef 

0xffffff821f7a3d40 : 0xffffff80129fd7a0 

0xffffff821f7a3dc0 : 0xffffff80129f35f1 

0xffffff821f7a3e10 : 0xffffff8012bbeba3 

0xffffff821f7a3e40 : 0xffffff8012bf04e6 

0xffffff821f7a3f50 : 0xffffff8012c3de23 

0xffffff821f7a3fb0 : 0xffffff80128f3e06 

BSD process name corresponding to current thread: zfs

Mac OS version:

13A603

Kernel version:

Darwin Kernel Version 13.0.0: Thu Sep 19 22:22:27 PDT 2013; root:xnu-2422.1.72~6/RELEASE_X86_64

Kernel UUID: 1D9369E3-D0A5-31B6-8D16-BFFBBB390393

Kernel slide:     0x0000000012600000

Kernel text base: 0xffffff8012800000

System model name: MacPro3,1 (Mac-F42C88C8)

System uptime in nanoseconds: 70084165880

last loaded kext at 55012291482: com.apple.driver.AppleHWSensor 1.9.5d0 (addr 0xffffff7f9459d000, size 36864)

loaded kexts:

org.maczfs.zfs.fs 74.3.2

The only thing I can think of doing is manually uninstalling all of the zfs files then manually installing them after pulling them out of the pkg file.

Anybody else have any better ideas?

-X

[ilov...@icloud.com]

You should not need to roll back to 10.8.5.

In any case, please change your nvram settings so that your panic report is actually useful.

sudo nvram boot-args="keepsyms=y -v"

Since these are boot-args you have to reboot for them to take effect. Do whatever it is you do to panic it. And then post your new panic report here.

[X Bytor]

On Wed, Dec 18, 2013 at 2:39 AM, <ilov...@icloud.com> wrote:

You should not need to roll back to 10.8.5.

I'm actually in dual-boot mode until I get this straightened out.

 

In any case, please change your nvram settings so that your panic report is actually useful.

sudo nvram boot-args="keepsyms=y -v"

Since these are boot-args you have to reboot for them to take effect. Do whatever it is you do to panic it. And then post your new panic report here.

Will do. Am going to try rip-out-and-replace first. Will post results.

[X Bytor]

Rip-out-and-replace did not work. Here's the panic I've been getting with 74.3.2 on OSX 10.9.1:

Wed Dec 18 17:47:15 2013

panic(cpu 3 caller 0xffffff800b6dc19e): Kernel trap at 0xffffff800bc36ed0, type 14=page fault, registers:

CR0: 0x0000000080010033, CR2: 0xffffff800bc36ed0, CR3: 0x0000000002e55000, CR4: 0x0000000000000660

RAX: 0xffffff800bc36ed0, RBX: 0xffffff822fcbbd58, RCX: 0x0000000001000000, RDX: 0x0000000000000be8

RSP: 0xffffff822fcbbc88, RBP: 0xffffff822fcbbcf0, RSI: 0x0000000000000000, RDI: 0xffffff803512c000

R8:  0xffffff80331f3540, R9:  0xffffff800bc536f0, R10: 0x0000000000000018, R11: 0x0000000000000206

R12: 0x0000000021000000, R13: 0xffffff8033c4c3c0, R14: 0x0000000021000000, R15: 0xffffff80331f3540

RFL: 0x0000000000010216, RIP: 0xffffff800bc36ed0, CS:  0x0000000000000008, SS:  0x0000000000000010

Fault CR2: 0xffffff800bc36ed0, Error code: 0x0000000000000011, Fault CPU: 0x3 Kernel NX fault

Backtrace (CPU 3), Frame : Return Address

0xffffff822fcbb910 : 0xffffff800b622f69 mach_kernel : _panic + 0xc9

0xffffff822fcbb990 : 0xffffff800b6dc19e mach_kernel : _kernel_trap + 0x7fe

0xffffff822fcbbb60 : 0xffffff800b6f3606 mach_kernel : _return_from_trap + 0xe6

0xffffff822fcbbb80 : 0xffffff800bc36ed0 

0xffffff822fcbbcf0 : 0xffffff800b80caef mach_kernel : _spec_ioctl + 0x11f

0xffffff822fcbbd40 : 0xffffff800b7fd7a0 mach_kernel : _VNOP_IOCTL + 0x150

0xffffff822fcbbdc0 : 0xffffff800b7f35f1 mach_kernel : _utf8_normalizestr + 0x971

0xffffff822fcbbe10 : 0xffffff800b9beba3 mach_kernel : _fo_ioctl + 0x43

0xffffff822fcbbe40 : 0xffffff800b9f04e6 mach_kernel : _ioctl + 0x466

0xffffff822fcbbf50 : 0xffffff800ba3de23 mach_kernel : _unix_syscall64 + 0x1f3

0xffffff822fcbbfb0 : 0xffffff800b6f3e06 mach_kernel : _hndl_unix_scall64 + 0x16

BSD process name corresponding to current thread: zfs

Boot args: keepsyms=y -v

Mac OS version:

13B42

Kernel version:

Darwin Kernel Version 13.0.0: Thu Sep 19 22:22:27 PDT 2013; root:xnu-2422.1.72~6/RELEASE_X86_64

Kernel UUID: 1D9369E3-D0A5-31B6-8D16-BFFBBB390393

Kernel slide:     0x000000000b400000

Kernel text base: 0xffffff800b600000

System model name: MacPro3,1 (Mac-F42C88C8)

System uptime in nanoseconds: 41237120919

last loaded kext at 28683100266: com.apple.driver.AppleHWSensor 1.9.5d0 (addr 0xffffff7f8d2d0000, size 36864)

loaded kexts:

org.maczfs.zfs.fs 74.3.2

.....