problem with creating and mounting zfs

[Marcin Mirosław]

Hi!
I'm using:
zfs-fuse-0.6.9
fuse-2.8.1
and i've got similar problem to this issue: http://zfs-fuse.net/issues/46 .

# lsmod |grep fuse
fuse 56678 1

# zpool create -m /mnt/zfs test /dev/mapper/malowazne-test
cannot mount 'test': Input/output error.
Make sure the FUSE module is loaded.

====== message.log:
2010-06-22T10:56:27.710851+02:00 localhost zfs-fuse: kstat: fuse_mount
error - trying to umount
[the same message many times]
2010-06-22T10:56:28.247673+02:00 localhost zfs-fuse: WARNING: Failed to
flush write cache on device '/dev/mapper/malowazne-test'. Data on pool
'test' may be lost if power fails. No further warnings will be given.
2010-06-22T10:56:28.816794+02:00 localhost zfs-fuse: !created version 23
pool test using 23
2010-06-22T10:56:29.953583+02:00 localhost zfs-fuse: kstat: fuse_mount
error - trying to umount
2010-06-22T10:56:29.953601+02:00 localhost zfs-fuse: kstat: fuse_mount
error - trying to umount
2010-06-22T10:56:29.953717+02:00 localhost zfs-fuse: kstat: fuse_mount
error - trying to umount
======

# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
test 7,94G 87K 7,94G 0% 1.00x ONLINE -

# zfs create test/aaaaa
cannot mount 'test/aaaaa': Input/output error.
Make sure the FUSE module is loaded.
filesystem successfully created, but not mounted

===== message.log:
2010-06-22T11:02:55.641937+02:00 localhost zfs-fuse: kstat: fuse_mount
error - trying to umount
2010-06-22T11:02:55.641955+02:00 localhost zfs-fuse: kstat: fuse_mount
error - trying to umount
2010-06-22T11:02:55.642078+02:00 localhost zfs-fuse: kstat: fuse_mount
error - trying to umount
======

Have you got any idea what is the reason of this problem?
Thanks,
Marcin

[sgheeren]

PS. +1: Oh and of course, use --no-kstat-mount as a temporary workaround?

Nope,

I'm still looking for more info on reproducing this. So -you- can help. See my breakdown here:

http://groups.google.com/group/zfs-fuse/browse_thread/thread/056166f5fdb0705c

So actually the reported issues most of interest for you are #51 and #60

I'm afraid there aren't any tangible theories. I suggest you tell me 'all' about your system; see the questions asked under #51 and anything specific that you can come up with yourself so we can get a handle on this thing.

Oh and of course, use --no-kstat-mount as a temporary workaround?

Seth

[sgheeren]

It does look like a potential permissions issue... doesn't it?

Are you invoking zfs-fuse, zfs and zpool as root?
Are you using SELinux or AppArmour rules?
Are you using special configuration settings for fuse (allow_other, allow_root etc)?

--
To post to this group, send email to zfs-...@googlegroups.com
To visit our Web site, click on http://zfs-fuse.net/

[sgheeren]

I may finally be getting somewhere. Sit down, a lot of information coming up:

= [A] PERMISSIONS ===========

I can replicate the symptoms by not running as root. I don't know how you would get in such a state without gross hacking of permissions. I had to

sudo chown -Rc "$(whoami):" /var/{run,lock}/zfs /zfs-kstat/ /etc/fuse.conf

I found out that even after that, fuse would refuse to mount stuff for the following reasons:

(a) unable to create mountpoints (a mkdir -pv $mountpoint; chown -Rc "$(whoami):" $mounpoint fixed that)
(b) the allow_other option is passed to all zfs-fuse mounts; this needs to be configured by uncommenting/adding 'user_allow_other' in /etc/fuse.conf

I'd like to stress here, that running as non-root (especially the daemon) is _not supported_[1]. I only did this investigation because it allows me to reproduce the error condition you are getting. Summarizing, you might need to check permissions on the mentioned structured. They need to be as follows:

# sudo ls -ltR /var/{run,lock}/zfs /zfs-kstat/ /etc/fuse.conf
ls: cannot access /zfs-kstat/: No such file or directory
-rw-r----- 1 root root 215 2010-06-22 12:45 /etc/fuse.conf
/var/lock/zfs:
-rw------- 1 root root 0 2010-06-22 12:52 zfs_lock
/var/run/zfs:
srwxr-xr-x 1 root root 0 2010-06-22 12:52 zfs_socket

= [B] LOGGED MESSAGES ===========

The warning "Data on pool 'poolname' may be lost if power fails. No further warnings will be given" is normal, and in my experience occurs only when using non-block devices (like e.g. loop devices) as your storage backend. Using lvm2 is ok at my site. Note that the warning is obviously not to be disregarded, because your data may be at risk. I'm assuming you are testing things and know what you are doing.

The kstat mounting business is a known issue with kstat if it cannot be mounted. In at least one occasion it seems that kstat not mounting prevents regular fs mounts from happening (issue #60). Otherwise this is just (a) the same issue as the filesytems not mounting (b) can be disabled with --no-kstat-mount

Cheers,
Seth

[sgheeren]

On 06/22/2010 01:08 PM, sgheeren wrote:

a lot of information coming up:

Ah I remember now

= [B] MORE DEBUG INFO? ===========
If all else fails, you can run zfs-fuse in a terminal foreground with

# sudo zfs-fuse -n

If you add ',debug' to the fuse mount options  in /etc/zfs/zfsrc [1], like e.g.:

fuse-mount-options = default_permissions,noatime,debug

This will give you oodles of info, including libfuse error messages on stderr

[1] or if you run latest testing branch (get it http://zfs-fuse.net/releases/0.7.0 or here), you can safely

# sudo zfs-fuse -n -o debug

[Marcin Mirosław]

W dniu 22.06.2010 13:08, sgheeren pisze:

> I may finally be getting somewhere. Sit down, a lot of information
> coming up:
>
> = [A] PERMISSIONS ===========
>
> I can replicate the symptoms by not running as root. I don't know how
> you would get in such a state without gross hacking of permissions. I
> had to

[...]

Yes, this was problem with permissions. I'm using standard gentoo ebuild
and init script. It starts zfs-fuse as user "daemon" and group "disk".
I'm ashamed , but i didn't know about directory /zfs-stat.
When i start zfs-fuse as root it works without problem.
Sorry for taking time.
Regards,
Marcin

[sgheeren]

Good to know that the gentoo ebuild takes a different approach there.
You might simply create and chown the /zfs-kstat if you wished or run
--no-kstat-mount

I'm going to post this over at the other guys, because I think (iirc)
one of them was running gentoo as well!

Regards,
Seth

[Marcin Mirosław]

W dniu 22.06.2010 14:17, sgheeren pisze:

> Good to know that the gentoo ebuild takes a different approach there.
> You might simply create and chown the /zfs-kstat if you wished or run
> --no-kstat-mount

I've created /zfs-kstat, chowned daemon:disk /zfs-kstat , chmod 770
/zfs-kstat and after start up zfs-fuse (as daemon:disk) i'm geting empty
dir /zfs-kstat. zpool list is empty, and i can't do zpool import ("Make
sure the FUSE module is loaded", and other strange things).
So i've changed init script, when zfs-fuse is running as root everything
works correctly.

> I'm going to post this over at the other guys, because I think (iirc)
> one of them was running gentoo as well!

And it should be write using capital letters in doc ;)

Regards,
Marcin