Probable data corruption when importing v14 zpool from Solaris to zfs-fuse on Ubuntu

[Yaniv Aknin]

Hi,

Briefly, I believe I have a case where a zpool that was created and
works fine in Solaris has some (not all) of the filesystems in it
silently appear empty after importing it to zfs-fuse.

I've filed a bug report (http://zfs-fuse.net/issues/88/view), but
believe I really didn't gather any significant data.

This is part of a data migration (zfs on OpenSolaris -> ext4 on
Ubuntu, never mind why now), and I'd like to reuse the disks
containing the affected pool. I'll be holding out on that for a bit,
but if you need more information, speak now or forever hold your
peace.

Thanks for zfs-fuse and good luck,
- Yaniv

[Emmanuel Anne]

You hit quite a serious bug and you don't have the idea to try 0.6.9 instead ?
And even if it's not in your distribution, it can be easily compiled from sources !

2010/9/11 Yaniv Aknin <ya...@aknin.name>

--
To post to this group, send email to zfs-...@googlegroups.com
To visit our Web site, click on http://zfs-fuse.net/

--
zfs-fuse git repository : http://rainemu.swishparty.co.uk/cgi-bin/gitweb.cgi?p=zfs;a=summary

[sgheeren]

Yaniv, you don't mention which version of zfs-fuse you are using.

For the record: there have been bugs with disappearing datasets going
from 0.5* to 0.6*. Starting with 0.6.1 (nicknamed "critical" in some
packages) these problems should no longer occur.

Also, it makes complete sense that you might have a pool with mixed
filesystem versions. It is technically possible to have filesystems of
unsupported versions inside a pool of 'older' versions[1]. In that case,
the following theory might apply: zfs-fuse is compatible enough to
import the pool (pool version is low enough) but it doesn't yet support
the filesystem version.

In any case, try to explicitely 'zfs mount tank/my/unmounted/fs' and see
what the output is.

In all cases, it is recommended to upgrade to 0.6.9. It should support
up to pool version 23 (from the top of my head) and that should be on
par with a Solaris nv134 at least.
If you were using an even more recent version of OpenSolaris [2] you
might _have_ to resort to an unstable build with Emmanuels ongoing synch
work with upstream. I believe it supports pool version 25 now. Beware
this is dev grade so use with backups!

Here is the link for our repo:
http://zfs-fuse.net/releases/0.6.9 (contains link to maint snapshots)

gitweb, official maint:
http://gitweb.zfs-fuse.net/?p=official;a=shortlog;h=refs/heads/maint

gitweb, emmanuels dev branch:
http://gitweb.zfs-fuse.net/?p=emmanuel;a=shortlog;h=refs/heads/master

[1] the most playsible way in which such a situation may arise is when
receiving filesystems into a pool using zfs send/receive
[2] not yet published via genunix.org

[sgheeren]

fyi: http://zfs-fuse.net/documentation/upgrading

[sgheeren]

Oh, and since you are <anonymous> please remember to read the followups at http://zfs-fuse.net/issues/88

[Yaniv Aknin]

OK, here goes. First the two main good things:
(1) doing an explicit zfs mount on the filesystem worked (it was
mounted, though it /still/ didn't show up in zfs list)
(2) upgrading to 0.6.9 from ppa made the problem disappear.

Even though the problem was resolved, if you care to listen to my
ramblings, I'll add here what I think is a bad thing. This isn't just
some-filesystems-don't-show-up. This can be a very subtle bug (only /
some/ zfss disappear, silently, zfs encourages you to have tens or
hundreds or more zfss). And like it or not, it appears with the
version shipping with Ubuntu 10.04.1, not exactly a lightweight
distribution. So maybe /I/ was the moron here, but you see how naively
I might have just happily done the apt-get install zfs-fuse, seen most
of my filesystems are there, migrate the data and destroy the zpool,
and realize the damage I've done only later.

So while I really don't know anything about zfs-fuse as a project -
what's your target audience, target platforms, how much you still care
about 0.6.0, etc, I think you guys should consider adding a clear
warning about this on your project homepage, like, say, btrfs has:
"Btrfs is under heavy development, but every effort is being made to
keep the filesystem stable and fast. As of 2.6.31, we only plan to
make forward compatible disk format changes, and many users have been
experimenting with Btrfs on their systems with good results. Please
email the Btrfs mailing list if you have any problems or questions
while using Btrfs."

Anyway, this is up to you to decide, and anyway, thanks for the prompt
help. I'll update the bugtracker with the zfs get / zfs list commands
you requested, I'm pretty much leaving this behind.

- Yaniv

On Sep 11, 1:18 pm, sgheeren <sghee...@hotmail.com> wrote:
> Oh, and since you are <anonymous> please remember to read the followups

> athttp://zfs-fuse.net/issues/88

>
> On 09/11/2010 12:13 PM, sgheeren wrote:
>
>
>
> > fyi:http://zfs-fuse.net/documentation/upgrading
>
> > On 09/11/2010 12:03 PM, sgheeren wrote:
> >> Yaniv, you don't mention which version of zfs-fuse you are using.
>
> >> For the record: there have been bugs with disappearing datasets going
> >> from 0.5* to 0.6*. Starting with 0.6.1 (nicknamed "critical" in some
> >> packages) these problems should no longer occur.
>
> >> Also, it makes complete sense that you might have a pool with mixed
> >> filesystem versions. It is technically possible to have filesystems of
> >> unsupported versions inside a pool of 'older' versions[1]. In that case,
> >> the following theory might apply: zfs-fuse is compatible enough to
> >> import the pool (pool version is low enough) but it doesn't yet support
> >> the filesystem version.
>
> >> In any case, try to explicitely 'zfs mount tank/my/unmounted/fs' and see
> >> what the output is.
>
> >> In all cases, it is recommended to upgrade to 0.6.9. It should support
> >> up to pool version 23 (from the top of my head) and that should be on
> >> par with a Solaris nv134 at least.
> >> If you were using an even more recent version of OpenSolaris [2] you
> >> might _have_ to resort to an unstable build with Emmanuels ongoing synch
> >> work with upstream. I believe it supports pool version 25 now. Beware
> >> this is dev grade so use with backups!
>
> >> Here is the link for our repo:

> >>http://zfs-fuse.net/releases/0.6.9(contains link to maint snapshots)

[sgheeren]

On 09/12/2010 06:00 AM, Yaniv Aknin wrote:

So maybe /I/ was the moron here, but you see how naively
I might have just happily done the apt-get install zfs-fuse, seen most
of my filesystems are there, migrate the data and destroy the zpool,
and realize the damage I've done only later.
  

I fully agree with the fact that this was a serous issue with the 0.6.0 point release. It has hampered adoption of 0.6* to the point where we see people switching from 0.5.1 to 0.6.9 even now. This is a shame of all the positive changes that were available.

Sadly I don't see how I can have control over the version in your repository. The exact same package maintainers (Seb. Delafond and Mike Hommey) have _lllllllllooooonnggg_ since published a 0.6.1 ("critical") package in debian and even now they are at 0.6.9 in debian collab-maint.

So, really the place to go with the complaint are the ubuntu MOTU team, I suppose. Or you can simply go to the debian repos.

[sgheeren]

I've updated the release comments and pages with fair warning

IMPORTANT NOTICE

That this release has been superseded by 0.6.9. The initial release of this version included a potentially dangerous problem with larger pools that would 'hide' filesystems. Such issues could lead to data loss if unnoticed, which is why it absolutely recommended to upgrade to the 0.6.9 release from this website, if at all possible.

Cheers,
Seth

On 09/12/2010 06:00 AM, Yaniv Aknin wrote:

[Yaniv Aknin]

Fair enough, this is pretty much what I'd expect from a serious
project and indeed this should be taken to the Ubuntu folks.

Thanks for the help,
- Yaniv

> >>>>http://zfs-fuse.net/releases/0.6.9(containslink to maint snapshots)