pools version 21, deduplication, zle, and stuff...

[Emmanuel Anne]

ok, it's done, the dedup is commited to new-solaris.
So... please consider this experimental at first, I won't tell you how many files this commit modified but so far it's the biggest commit for zfs I ever saw. I tested it, and it seems to work. The funny thing is that when you run a df, the used part still increases, but the free space increases too, so you have the impression that your volume is growing which is a little weird...

Also zfs send does not use dedups for now since it's part of a commit not yet included in zfs-fuse. I stopped at 10982, which is on friday of last week, we are very close now ! And this commit seems to use an external lib for sha256 which was no required before, so I'll take more time for this.

This also adds a new type of compression, zle, not tested yet.

If you wonder where pools version 20 went, I wondered too, but in fact commit 10922 switched directly from pools version 19 to pools version 21 !

You'll find more details on dedup there :
http://www.theregister.co.uk/2009/11/02/zfs_gets_dedupe/

Also there were quite a few more fixes commited, I have upgraded all my pools by mistake to version 21 (don't type zpool upgrade -a instead of zpool upgrade -v, or don't even use zpool when you get tired !).
Oh well, everything should be allright ! :)

[mrsvan]

Hi Emmanuel,

Thanks for the tip on how to get your latest version to compile.

I have installed zfs-fuse w/ zpool version 21 on my home server and it
seems to work just fine.

I noticed a bug regarding the naming of my vdevs... they are listed as
-0 and -1, rather than by their type.
Is this a cosmetic problem, or should I be worried ?

# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
tank 4.08T 3.48T 616G 85% 1.00x ONLINE -

# zpool status -x
pool: tank
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool
can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done,
the
pool will no longer be accessible on older software versions.
scrub: resilver completed after 0h0m with 0 errors on Tue Nov 10
21:05:15 2009
config:

NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
-0 ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0 17.8M resilvered
sdc ONLINE 0 0 0
-1 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdf ONLINE 0 0 0 1.17M resilvered

[Emmanuel Anne]

Thanks for the info. Well I can't reproduce it here, so I guess it's probably some kind of pool which is displayed like that... a raidz ?
Anyway it's probably just because of one of the warnings from the sprintfs, I left them uncorrected since they are directly from the opensolaris sources, but I should probably have a closer look.
Anyway AFAIK it's harmless, but I'll check better than that later !

(notice that you need to upgrade the pools to version 21 to be able to use dedup).

2009/11/10 mrsvan <s...@p-group.cc>

[Emmanuel Anne]

I have just fixed it in the new-solaris branch.
Apparently it was a pure opensolaris bug, something related to mixing char * with char[], the kind of bug that a C beginner could do ! At least it shows they are mere humans too after all ! ;-) The funny thing is that they didn't fix it earlier, it's been here since at least november 6th, so congratulations for being the 1st one to spot it.

Now when you run zpool state you should see
raidz1-0 instead of just -0

2009/11/10 Emmanuel Anne <emmanu...@gmail.com>

[mrsvan]

Thanks for the ultra-fast response!

Your patch fixed my problem:

# zpool status

pool: tank
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool
can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done,
the
pool will no longer be accessible on older software versions.
scrub: resilver completed after 0h0m with 0 errors on Tue Nov 10
21:05:15 2009
config:

NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0

raidz1-0 ONLINE 0 0 0

sdd ONLINE 0 0 0
sde ONLINE 0 0 0 17.8M resilvered
sdc ONLINE 0 0 0

raidz1-1 ONLINE 0 0 0

[Rudd-O]

Let's be good netizens and remember to send the patch upstream so they
can fix it in their code! :-D

On Nov 11, 2:19 am, Emmanuel Anne <emmanuel.a...@gmail.com> wrote:
> I have just fixed it in the new-solaris branch.
> Apparently it was a pure opensolaris bug, something related to mixing char *
> with char[], the kind of bug that a C beginner could do ! At least it shows
> they are mere humans too after all ! ;-) The funny thing is that they didn't
> fix it earlier, it's been here since at least november 6th, so
> congratulations for being the 1st one to spot it.
>
> Now when you run zpool state you should see
> raidz1-0 instead of just -0
>

> 2009/11/10 Emmanuel Anne <emmanuel.a...@gmail.com>

[mrsvan]

Hello again Emmanuel,

I believe I have uncovered a bug in the latest version of zfs-fuse.

I am not able to destroy my pools anymore.

I recently added a new SATA backplane to my home server. As a result
of this, the device labels of my HDDs got jumbled in Ubuntu, and ZFS-
Fuse was unable to reassemble my pool. This has happened to me
before, and the usual solution is to destroy the pool and then just
reimport it...

But now when I attempt to destroy the pool, I get an "internal error"
message.

Here is the read-out of my commandline:

root@stein-server:/# zpool status
pool: tank
state: UNAVAIL
status: One or more devices could not be used because the label is
missing
or invalid. There are insufficient replicas for the pool to continue
functioning.
action: Destroy and re-create the pool from
a backup source.
see: http://www.sun.com/msg/ZFS-8000-5E
scrub: none requested

config:

NAME STATE READ WRITE CKSUM

tank UNAVAIL 0 0 0 insufficient replicas
raidz1-0 UNAVAIL 0 0 0 insufficient replicas
sdd UNAVAIL 0 0 0 corrupted data
sde UNAVAIL 0 0 0 corrupted data

sdc ONLINE 0 0 0

raidz1-1 ONLINE 0 0 0

sdb ONLINE 0 0 0
sdf ONLINE 0 0 0

root@stein-server:/# zpool destroy tank
internal error: Invalid argument
Aborted

[sghe...@hotmail.com]

mrsvan wrote:
> Hello again Emmanuel,
>
> I believe I have uncovered a bug in the latest version of zfs-fuse.
>
> I am not able to destroy my pools anymore.
>

This is certainly an interesting symptom... hang on

> I recently added a new SATA backplane to my home server. As a result
> of this, the device labels of my HDDs got jumbled in Ubuntu, and ZFS-
> Fuse was unable to reassemble my pool. This has happened to me
> before, and the usual solution is to destroy the pool and then just
> reimport it...
>

You what?

The usual way to handle this is to import the pool using '-d
/dev/disk/by-id' to avoid being caught up in the /dev/sd? confusion.
If you forgot/did not know beforehand, the usual route is:

killall zfs-fuse
rm /etc/zfs/zpool.cache
/etc/init.d/zfs-fuse start

replace the last command with your favourite method of launching zfs-fuse

Destroying a pool that it could not find is a bit draconian. What's
more, if it actually succeeded, you would in effect be actively
_destroying_ your data. I cannot imagine you really want to walk so
close to the cliff.

$0.02

[mrsvan]

Hmm - my memory must be messing with me, I would have sworn that I
could do an import after destroying my pool...

Anyway, your method of erasing zpool.cache and then reimporting
worked!
My tank pool is back up and running.

BTW, I used to build my pools with the '-d /dev/disk/by-id' option,
but when I upgraded from Ubuntu Gutsy to Karmic, I discovered that the
disk IDs were modified during the upgrade! So using the default 'sdX'
labels is fine with me since I can always use your method to rebuild
the pool after a label change...

Thanks for your help!

[Emmanuel Anne]

Yes you can import a destroyed pool, see the man page of zpool for that (never tested it, but it should work !).

2009/11/12 mrsvan <s...@p-group.cc>

[sghe...@hotmail.com]

Yes, and you can pick your keys up with a stick after you drop them in
the sewer.

But you don't do that _on purpose_