Zpool status ----- CRASH

[jasonbelec]

OK here is a new one for me. Was resolving an issue in a vm, when
things got really weird. Host OS X system locked up needing manual
button shutdown. Rebooted. Main OS drive fine, data accessible on
pool, stuff opens, etc.., but if I try to get the status of the pool,
WAMMO the magic you must shutdown screen appears.

Any ideas? Suggestions?

The pool is

Tank
Mirror disk 1
disk 2
Mirror disk 3
Disk 4

Probably going to be forced to move data to new drives, but thought it
worth picking some minds.

Thanks, Jason

[Matt Snow]

Have you tried scrubbing the pool? If you load system profiler and try
to view Hardware->Serial-ATA do you see all drives?

..Matt

[Jason Belec]

Yeah. Set it running. Then went to check -- boom! Yes all drives are visible. Kinda what makes this a weird one.

--

Jason Belec

Sent from my iPhone

[Jason Belec]

Hhhmmmm, doesn't like scrub either. Copying data then I'll start poking it with a stick. Open to any cool ideas, never tested visionary concepts, voodoo magic, even dark magic!

Jason

Sent from my iPad

[Chris Ridd]

On 3 May 2011, at 18:57, Jason Belec wrote:

> Hhhmmmm, doesn't like scrub either. Copying data then I'll start poking it with a stick. Open to any cool ideas, never tested visionary concepts, voodoo magic, even dark magic!

Do you get any stack traces?

Chris

[Bill Winnett]

On May 3, 2011, at 1:57 PM, Jason Belec wrote:

> Hhhmmmm, doesn't like scrub either. Copying data then I'll start
> poking it with a stick. Open to any cool ideas, never tested
> visionary concepts, voodoo magic, even dark magic!
>

I once had this problem, could not do anything with the ZFS volume as
OSX kept crashing. I ended up making a VM with Virtualbox, loaded
milux and repaired the disks with Solaris. Then all was copesetic.
This was more then a year ago, probably closer to 2 when I used
milux. Maybe you could achieve the same with a BSD VM, or a Oracle/
Sun release.

-bill w.

[Jason Belec]

No, will get to that tonight....

--

Jason Belec

Sent from my iPhone

[Jason Belec]

Hey Bill, any idea of the cause?? Remember what you did to restore normal operation? I do have the latest FreeBSD around.

In my poking around I was able to convert a snapshot to clone and copy data off. So things are partially working as they should.

It's a good exercise.

So if that's possible I should be able to destroy and start anew, revolting the data back.

These are 4 x 1TB server class drives which makes me wonder.

Anyone else got a few more pieces? ROFL

--

Jason Belec

Sent from my iPhone

On 2011-05-03, at 3:05 PM, Bill Winnett <bill.w...@gmail.com> wrote:

>
> On May 3, 2011, at 1:57 PM, Jason Belec wrote:
>
>> Hhhmmmm, doesn't like scrub either. Copying data then I'll start poking it with a stick. Open to any cool ideas, never tested visionary concepts, voodoo magic, even dark magic!
>>
>

> I once had this problem, could not do anything with the ZFS volume as OSX kept crashing. I ended up making a VM with Virtualbox, loaded milux and repaired the disks with Solaris. Then all was copesetic. This was more then a year ago, probably closer to 2 when I used milux. Maybe you could achieve the same with a BSD VM, or a Oracle/Sun release.

[jasonbelec]

Nothing to show for what the problem really is.

Can send/receive, well some snapshots. Getting a broken pipe on one.
Will know the final result by tomorrow.

Can clone one of the latest snapshots and copy files off. Testing
those tonight.

Then I'll destroy the pool and nuke the drives, test and start over to
see if it happens anew.

First time I've seen this kind of issue on any of the test or
production systems around here.

Oddly little info trackable to what the issue might be. Did notice
that at least at the current stage of things, data doesn't seem to get
written to the drive.

Appreciate the ideas/suggestions. Nice to know fundamentally that data
still exists.

Jason

On May 3, 4:33 pm, Jason Belec <jasonbe...@rogers.com> wrote:
> Hey Bill, any idea of the cause?? Remember what you did to restore normal operation? I do have the latest FreeBSD around.
>
> In my poking around I was able to convert a snapshot to clone and copy data off. So things are partially working as they should.
>
> It's a good exercise.
>
> So if that's possible I should be able to destroy and start anew, revolting the data back.
>
> These are 4 x 1TB server class drives which makes me wonder.
>
> Anyone else got a few more pieces? ROFL
>
> --
>
> Jason Belec
>
> Sent from my iPhone
>

> On 2011-05-03, at 3:05 PM, Bill Winnett <bill.winn...@gmail.com> wrote:
>
>
>
>
>
> > On May 3, 2011, at 1:57 PM, Jason Belec wrote:
>
> >> Hhhmmmm, doesn't like scrub either. Copying data then I'll start poking it with a stick.  Open to any cool ideas, never tested visionary concepts, voodoo magic, even dark magic!
>
> > I once had this problem, could not do anything with the ZFS volume as OSX kept crashing.  I ended up making a VM with Virtualbox, loaded milux and repaired the disks with Solaris.  Then all was copesetic.  This was more then a year ago, probably closer to 2 when I used milux.  Maybe you could achieve the same with a BSD VM, or a Oracle/Sun release.
>
> > -bill w.
>
> >> Jason
>
> >> Sent from my iPad
>

> >> On 2011-05-03, at 12:48 PM, Jason Belec <jasonbe...@rogers.com> wrote:
>
> >>> Yeah. Set it running. Then went to check -- boom! Yes all drives are visible. Kinda what makes this a weird one.
>
> >>> --
>
> >>> Jason Belec
>
> >>> Sent from my iPhone
>

> >>> On 2011-05-03, at 12:31 PM, Matt Snow <matts...@gmail.com> wrote:
>
> >>>> Have you tried scrubbing the pool? If you load system profiler and try
> >>>> to view Hardware->Serial-ATA do you see all drives?
>
> >>>> ..Matt
>

[Jason Belec]

Well, two of the drives had serious sector issues. Ha! RE drives at that! Rebuilt, beginning testing. At one point, even the system wouldn't boot with one of the particularly offensive drives connected, well after mucking around a bit. It is going through 'special' testing over the next few days to see what I initially missed. The other seems fine after zero-ing it twice. It has a less important role until it proves itself.

Jason

Sent from my iPad

[jasonbelec]

 

OK, I felt this should be followed up on. Totally my fault. Brain fart extrordinaire.

So at some point I made a Snapshot, kinda normal.

I Cloned said Snapshot. Also kinda normal.

However, what isn't normal is mounting both of these to the same /Users/idiot location.

Why I did not see this before is beyond comprehension.

Once resolved, Scrub works, Status works, and the really good news is that what I thought was a damaged pool is actually in perfect health all this time. 

Good thing I kept this Pool, in fact I've been using it and exploring it for a long time now, wondering just what was wrong. Palm to the head.

Hopefully this will stand as a resource for others. At the very least, todays humour moment.