pool/dataset disappeared?

[Durval Menezes]

Hello Folks,

I have an old machine here running RHEL4 with kernel 2.6.27.57 downloaded from kernel.org, compiled and installed manually. It also has zfs-fuse 0.5.0 with a single partition on its hard disk dedicated for zfs use (/dev/sda6), via the creation of a single pool called "testpool"; this configuration has been working great since 2008 or so.  

Yesterday, the machine was inadvertently powered off without a proper shutdown, and when powered back on, the pool simply refuses to mount:

# zfs mount testpool

cannot open 'testpool': dataset does not exist

# zfs list

no datasets available

Can anyone offer any help on how to recover from this? I've searched both this list and the Web at large, but failed to find anything really related to my situation. 

Thanks in advance,

-- 

   Durval Menezes.

[Björn Kahl]

Hello Durval Menezes,

Am 15.06.12 01:50, schrieb Durval Menezes:

> Hello Folks,
>
> I have an old machine here running RHEL4 with kernel 2.6.27.57
> downloaded from kernel.org, compiled and installed manually. It also
> has zfs-fuse 0.5.0 with a single partition on its hard disk dedicated
> for zfs use (/dev/sda6), via the creation of a single pool called
> "testpool"; this configuration has been working great since 2008 or
> so.

Pretty old. :-)

> Yesterday, the machine was inadvertently powered off without a proper
> shutdown, and when powered back on, the pool simply refuses to mount:
>
> # zfs mount testpool
> cannot open 'testpool': dataset does not exist
>
> # zfs list
> no datasets available
>
> Can anyone offer any help on how to recover from this? I've searched
> both this list and the Web at large, but failed to find anything
> really related to my situation.

Not enough data.


Is the disk still there?

Has it possibly been renamed by the kernel? I.e. is /dev/sda6 still
named sda6 after the latest reboot?

Anything interesting in the logs?

What says zpool status -v?

What says zpool import?


Best regards

Björn
--
| Bjoern Kahl +++ Siegburg +++ Germany |
| "googlelogin@-my-domain-" +++ www.bjoern-kahl.de |
| Languages: German, English, Ancient Latin (a bit :-)) |

[Durval Menezes]

Hi Bjorn,

On Thu, Jun 14, 2012 at 9:04 PM, Björn Kahl <googl...@bjoern-kahl.de> wrote:

 Hello Durval Menezes,

Am 15.06.12 01:50, schrieb Durval Menezes:

> Hello Folks,
>
> I have an old machine here running RHEL4 with kernel 2.6.27.57
> downloaded from kernel.org, compiled and installed manually. It also
> has zfs-fuse 0.5.0 with a single partition on its hard disk dedicated
> for zfs use (/dev/sda6), via the creation of a single pool called
> "testpool"; this configuration has been working great since 2008 or
> so.

 Pretty old. :-)

No doubt. But if it isn't broken, why fix it? ;-) This is the first hiccup this machine has given me in all those years... 

 

> Yesterday, the machine was inadvertently powered off without a proper
> shutdown, and when powered back on, the pool simply refuses to mount:
>
> # zfs mount testpool
> cannot open 'testpool': dataset does not exist
>
> # zfs list
> no datasets available
>
> Can anyone offer any help on how to recover from this? I've searched
> both this list and the Web at large, but failed to find anything
> really related to my situation.

 Not enough data.

You are right, sorry. I should have been more verbose.

 

 Is the disk still there?

Yep: the disk is there, my system log has no messages related to any errors on it, and smartctl -a shows absolutely no issues (no errors in the disk's smart log, all problem counters like Reallocated_Sector_Ct, Current_Pending_Sector, etc are all zero, and the disk has just went through a SMART long test which also completed without any errors). 

 

 Has it possibly been renamed by the kernel?  I.e. is /dev/sda6 still
 named sda6 after the latest reboot?

Nope; just checked and sda6 is the same as always. An  "hexdump -C /dev/sda6" even shows me a lot of zfs-suggestive strings on it... 

 

 Anything interesting in the logs?

Nothing at all. I routinely log ALL syslog messages (ie, "*.*")  to a single file, and I have just checked it in minute detail, but could find nothing at all: no disk messages, no zfs messages, nothing. 

 

 What says zpool status -v?

# zpool status -v 

no pools available

 What says zpool import?

Haven't tried yet. Is it perfectly safe (ie, completely read-only)? Or should I make a copy of the partition first?

 

Thanks for the help,

-- 

   Durval Menezes.

[Ryan How]

zpool import just lists the pools available for importing. It's probably
exactly what you need :)

usually the system remembers what was already imported in the system and
automatically brings it in, but something may have happened and it "forgot".

[Durval Menezes]

Hi Ryan,

On Thu, Jun 14, 2012 at 10:15 PM, Ryan How <rh...@exemail.com.au> wrote:

zpool import just lists the pools available for importing. 

 

You mean, it walks though all disk devices and then looks for signatures there, printing the ones it founds?

It's probably exactly what you need :)

I've started a network copy of that partition to my desktop... will wait for it to finish before continuing, just for the extra peace of mind ;-)

 

usually the system remembers what was already imported in the system and automatically brings it in, but something may have happened and it "forgot".

Humrmrmrmr... pray tell, where would the system "store" that info? A flag or something at each pool "superblock" or whatever?

Cheers,

-- 

   Durval.

 

--
To post to this group, send email to zfs-...@googlegroups.com
To visit our Web site, click on http://zfs-fuse.net/

[Ryan How]

I think it is some zfs cache file, /etc/zfs.cache or something, don't really have much idea :). But I run ZFS on a live linux system and have to import the zpools every time because that file isn't persisted.

So it doesn't store it on the zfs disk. I guess it must be like the zfs version of fstab

Because you didn't export the disk, you might have to do a force on the import. Don't be scared, I've done it a million times and I'm a bit of a newbie to zfs and haven't managed to lose anything yet :).

Cheers,

Ryan

[Durval Menezes]

Hello folks,

After finishing and verifying my full-partition dd copy to another machine, I finally ran (with great anticipation) the "zpool import" command, here's how it went:

# zpool import

  pool: testpool

    id: 1571407782190788899

 state: ONLINE

action: The pool can be imported using its name or numeric identifier.

config:

        testpool    ONLINE

          sda6      ONLINE

And then:

# zpool import testpool

(took a few seconds, and didn't output anything).

At the end of it, the disappeared pool was imported and even mounted, so I didn't even had to do a "zfs mount" on it, and it now shows on "zfs list" too:

# zfs list

NAME       USED  AVAIL  REFER  MOUNTPOINT

testpool   186G  13.0G   186G  /testpool

So I guess it's fully recovered now... just to make sure everything is alright, I started a "zpool scrub" on the pool, waiting for it to finish... still 4h to go, but I don't anticipate any further problems. 

Thanks to everyone for the help.

Cheers,

-- 

   Durval.

[Christ Schlacta]

Before you add sizable data to the new disk, you should attempt to reboot the system again just to be certain, however, it should just continue to just work again now.

[Durval Menezes]

Hello Christ,

On Sat, Jun 16, 2012 at 9:11 PM, Christ Schlacta <aar...@gmail.com> wrote:

Before you add sizable data to the new disk, you should attempt to reboot the system again just to be certain, however, it should just continue to just work again now.

Will do, right after my "zfs scrub" finishes (63.5% done already, and no errors so far...)

Thanks for the tip,

-- 

   Durval Menezes.

[Durval Menezes]

Hello Folks,

Just to keep everyone posted: the scrub finished with no errors whatsoever, and I've just rebooted the machine and the ZFS partition is now mounting alright. so everything seems fully recovered.

Thanks everyone for the great help.

Cheers,

-- 

   Durval.