ZFS-OSX hates GitHub

[Alex Wasserman]

I got a proper panic today, and this one actually appeared on screen and stayed there, as opposed to the usual flicker or straight-up freeze.

Right as I loaded GitHub up in a new Chrome session. Nothing else significant happening - emptying the trash, copying a directory. I think BackBlaze was running a back-up in the background.

Screen pic has the details.

Currently running:

commit 1706f31219aaca49b3910e2ec9c948bf89d111e4

Author: Jorgen Lundman

Date:   Fri Feb 28 15:05:57 2014 +0900

Thanks,

Alex

[lun...@lundman.net]

Love how it redrew chrome just enough to hide the stack.

But as it happens;

https://github.com/zfs-osx/zfs/blob/master/module/zfs/zfs_dir.c#L466

it's that unlinked_add() thing. Not seen it for quite some time though. If you read /var/log/system (and/or kernel) it should have the full
stack

[Alex Wasserman]

Not sure the direct reply from my phone worked.

I can't log into that account now. Every time it panics. This time it was a cleaner screen though.

System.log has:

Mar  4 22:25:20 smiley.local Google Chrome Helper[1103]: CGSLookupServerRootPort: Failed to look up the port for "com.apple.windowserver.active" (1100)

<crap load of ^@ which won't copy/paste> 

Mar  4 22:31:23 localhost bootlog[0]: BOOT_TIME 1393990283 0

Mar  4 22:31:26 localhost syslogd[21]: Configuration Notice:

[Jorgen Lundman]

Hmm maybe try importing as readonly one time and see if you can clear it.
But better if you pop into freenode #mac-zfs so we can help directly.

> --
> See http://MacZFS.org/ for more information on this project.
> ---
> You received this message because you are subscribed to the Google Groups
> "MacZFS-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to maczfs-devel...@googlegroups.com
> <mailto:maczfs-devel...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/maczfs-devel.

--
Jorgen Lundman | <lun...@lundman.net>
Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell)
Japan | +81 (0)3 -3375-1767 (home)

[Alex Wasserman]

Readonly does allow me to log in, but not sure how to clear it.

I can log into my spare account, which is on a non-zfs volume, that works fine. Just logging into my main account which has home dir on ZFS crashes.

Also, I've found that deleting the files on the ZFS triggers it. Not everything, just a bunch of stuff.

Attached are three such crashes for different files. The files are in large directories, so hard to narrow down to a single file, given that every test is a crash and reboot.

I'm in the room (AlexW) if you have any ideas.

[ilov...@icloud.com]

Readonly does allow me to log in, but not sure how to clear it.

When you say "readonly" what do you mean exactly? How are you importing the pool?

[Alex Wasserman]

Looks like I replied directly before and forgot to copy the mailing list - included here for the sake of completeness.

So, this is an interesting scenario as the pool is still mostly usable - 

• With ZFS mounted normally I can:
• Log onto the box as a non-ZFS-homed user
• Log onto the box as a ZFS-homed on console, or SSH
• Run a full scrub with only 2 checksum errors found, neither in critical files. Not unexpected given the number of panics I've experienced, while moving a lot of data around recently.
• With ZFS mounted I get KPs:
• Logging into the OSX GUI (not console) with a ZFS-homed user. Only have one, not sure if this would impact a clean new user, something to test tonight
• When I delete certain files. Most file operations are fine. After putting in 2 new larger drives, to try to get the data striped properly I've been copying and deleting a lot of data. Trying to narrow it down I've been working through the items in my trash manually to find what causes this, but the number of items is huge, and each failure requires a reboot - this is time-consuming.

I have about 4Tb on the pool, I don't have enough spare disks to create a new pool and copy over.

The existing pool is made of 3 mirrored devs, so I could split the mirrors, and then create a new pool with 1 half, then zfs send, or rsync, cp, etc the data from the old pool to the new. Of course, this would remove all redundancy. Data is backed up, but it would be an expensive pain to restore.

What is the chance that just copying from the one pool to the new would just bring this corruption with it?

Also, given that the data appearing to cause issues is in the trash and pending deletion, if I can create a new blank user that works, I could just migrate over what I need.

Trying to use OpenIndiana too - that's saved me in the past, but my USB key won't boot. Created a new one and I still get an issue after the mem check step. I think I still have an OI drive, I'll try booting from it if I can find it and plug it in.

Thanks for the help,

Alex

 

Begin forwarded message:

From: ilove zfs <ilov...@icloud.com>

Date: March 6, 2014 at 1:47:34 AM EST

To: Alex Wasserman <al...@alexwasserman.com>

Subject: Re: [MacZFS-devel] Re: ZFS-OSX hates GitHub

OK, good, that's what I was hoping you meant.

 

I suggested to lundman that he give you the recommendation to try the read only import. I am glad to hear this worked to prevent panic. The purpose here is to allow you to recover your data to another pool, not to make the pool usable again. Your pool is somehow corrupted, and your best bet is going to be just to copy the data off somewhere else given that you are now able to access the data on the pool again.

On Mar 05, 2014, at 10:43 PM, Alex Wasserman <al...@alexwasserman.com> wrote:

zpool import -o readonly=on ZFS_Pool

 

Causes some issues to osx because it can't save anything, but it worked to log in, didn't kp like normal.

-- 

Alex Wasserman

+1 646 460 5187

al...@alexwasserman.com