sharenfs support

[Emmanuel Anne]

After reading some part of the thread about nfs problems, I thought about enabling support for the sharenfs option.
The advantages :
 - if you change the mountpoint with the mountpoint option, the share is moved too automatically
 - if there are subfilesystems created with zfs create, they inherit the sharenfs option, and so there will be no inode conflict while sharing.
 - Apparently the shared are automatically unshared when the daemon exits, but you need to add a "zfs share -a" line in the startup script so that they are shared automatically when the daemon starts.

It makes some kind of parallel way of sharing paths, a way which doesn't use the exports file but is still automatic.

The problem : normally sharenfs can take 3 values :
off : defautl - no share
on : shared with default options. For now I chose something to be close to the default nfs exports options, so it's ro,no_subtree_check,fsid=100. Didn't know what to take for the fsid value, I took 100 to lower the chances of conflict with another fsid value.

any other value is taken as the options. In opensolaris these are the options to the share command. In zfs-fuse it would be the options to the exportfs command. The problem is that it's not extremely convenient to specify hosts in this manner, you would have to do :
zfs set sharenfs="-o rw host1:path -o ro host2:path" share
The other problem is that if you want to specify different hosts, then you have to specify the mount paths too, which is really not convenient in this case. So maybe it would be better to find another way to specify the options...

So the default options are read-only, shared by any host.
And of course the default value for the option is off, so no shared is added if you don't change the value.
I have just commited this, you can test and comment on that, it can probably be improved.

The sharesmb option is not enabled yet, mainly because it was easier for me to test nfs !

--
zfs-fuse git repository : http://rainemu.swishparty.co.uk/cgi-bin/gitweb.cgi?p=zfs;a=summary

[sgheeren]

Woah

I'm really amazed how you pull this stuff.

I'm gonna have a look right now

You know, I already switched to OpenSolaris on my new NAS box ... mainly
for this reason (automatic nfs sharing)

Granted, I really liked the stuff that happened with SMB sharing and
'shadow copies' (older versions) on Win32 client machines.
Awesome-frickin-awesome stuff that too

> --
> To post to this group, send email to zfs-...@googlegroups.com
> To visit our Web site, click on http://zfs-fuse.net/

[sgheeren]

How do I get around

root@karmic:~# mount -o ro
karmic:/MUBI/zfs/PersoonlijkeBestanden/MyPictures /tmp/HerPictures
mount.nfs: access denied by server while mounting
karmic:/MUBI/zfs/PersoonlijkeBestanden/MyPictures

Syslog:

Feb 15 14:33:29 karmic mountd[4340]: refused mount request from
127.0.1.1 for /MUBI/zfs/PersoonlijkeBestanden/MyPictures (/): not exported

I'm assuming I need the in-kernel nfs server?

A bit more context:

root@karmic:~# zfs get sharenfs -s local
NAME
PROPERTY VALUE SOURCE
desktop/PersoonlijkeBestanden/MyPictures
sharenfs on local

root@karmic:~# zfs list desktop/PersoonlijkeBestanden/MyPictures
NAME USED AVAIL REFER MOUNTPOINT
desktop/PersoonlijkeBestanden/MyPictures 7.90G 12.0G 7.90G
/MUBI/zfs/PersoonlijkeBestanden/MyPictures

[Emmanuel Anne]

Yes you need the exportfs command, the one which maintains the mounts in /var/lib/nfs/etab.

So once you have done :
zfs set sharenfs=on filesystem
if it displays no error then you should see the share in /var/lib/nfs/etab with the default options used.

I just commited a big change which makes the non default options usable, because passing directly the exportfs options really didn't work.

So now the syntax is :
sharenfs=off -> disabled
on -> default options : ro,fsid=100,no_subtree_check for any host

any other value is an option string of this format :
host1:options1 host2:options2 ...

so for example :
zfs set sharenfs="host1:rw host2:rw" test
notice that you don't need to pass fsid or subtree_check, if one of the default option is not passed, it's added automatically.

I have checked that zfs set sharenfs=off test unshares all the shares and not just the 1st one, so everything should be ok now... !

2010/2/15 sgheeren <sghe...@hotmail.com>

[sgheeren]

Emmanuel Anne wrote:

Yes you need the exportfs command, the one which maintains the mounts in /var/lib/nfs/etab.

Ok, just checked, it's present. And indeed supplied by the kernel nfs driver:

sehe@karmic:~$ dpkg --search exportfs
nfs-kernel-server: /usr/share/man/man8/exportfs.8.gz
nfs-kernel-server: /usr/sbin/exportfs

So once you have done :
zfs set sharenfs=on filesystem
if it displays no error then you should see the share in /var/lib/nfs/etab with the default options used.

I just commited a big change which makes the non default options usable, because passing directly the exportfs options really didn't work.

Ok, I'm updating

So now the syntax is :
sharenfs=off -> disabled
on -> default options : ro,fsid=100,no_subtree_check for any host

Using sharenfs=on, does it make sense that it doesn't even call exportfs? I'm using a debug=2, running

# gdb --args zfs-fuse --no-daemon

and in another terminal:

# gdb zfs

(gdb) break zfsfuse_share
(gdb) run set sharenfs=on desktop/PersoonlijkeBestanden/MyPictures
(gdb) shell zfs get sharenfs -s local

NAME                                                                   PROPERTY  VALUE     SOURCE
desktop/PersoonlijkeBestanden/MyPictures                               sharenfs  on        local

(gdb) run share -a
Starting program: /usr/local/sbin/zfs share -a
[Thread debugging using libthread_db enabled]

Program exited normally.
(gdb) run unshare -a
Starting program: /usr/local/sbin/zfs unshare -a
[Thread debugging using libthread_db enabled]

Program exited normally.
(gdb) run share -a
Starting program: /usr/local/sbin/zfs share -a
[Thread debugging using libthread_db enabled]

Program exited normally.

Never once is the breakpoint hit, cat /var/lib/nfs/etab doesn't show any (new) exports. What am I missing?

Oh, sidenote: this pool is imported using an altroot setting because I cannot mount two fs-es on /home. Perhaps this is ruining the fun. Well, I'll try later without 'import -R'

[Emmanuel Anne]

It calls exportfs to add a share only if it can't detect another share in this path, and by the way it might also ignore it if there is a sub path which is already shared.

So you can't mix shares from /etc/exports and shares from sharenfs for the same filesystem.

You must make sure that grep ^mountpoint /var/lib/nfs/etab returns nothing before changing sharenfs. (where mountpoint is the mountpoint of the zfs filesystem you want to change).

Also it calls exportfs using the system(3) call, and I have read that on some distributions, when root uses the system call, they use a bash version which drops its privileges before executing the command. This is security in the path of efficiency, debian does not do that (so ubuntu does not neither probably). Don't know about fedora and redhat...

Hope you can make it work, it would be too bad it can work only here !
;-)

2010/2/15 sgheeren <sghe...@hotmail.com>

[sgheeren]

Emmanuel Anne wrote:
> It calls exportfs to add a share only if it can't detect another share
> in this path, and by the way it might also ignore it if there is a sub
> path which is already shared.
>
> So you can't mix shares from /etc/exports and shares from sharenfs for
> the same filesystem.

Only /home was in there. Not a conflict. I will try with an empty export
table, though

>
> You must make sure that grep ^mountpoint /var/lib/nfs/etab returns
> nothing before changing sharenfs. (where mountpoint is the mountpoint
> of the zfs filesystem you want to change).

>
> Also it calls exportfs using the system(3) call, and I have read that
> on some distributions, when root uses the system call, they use a bash
> version which drops its privileges before executing the command. This
> is security in the path of efficiency, debian does not do that (so
> ubuntu does not neither probably). Don't know about fedora and redhat...

Ubuntu here. I suppose priviliges aren't dropped. Although the
restricted bash shell may be used (bash -r if I recall). This may impact
PATH to make it fail.

>
> Hope you can make it work, it would be too bad it can work only here !
> ;-)
>

> 2010/2/15 sgheeren <sghe...@hotmail.com <mailto:sghe...@hotmail.com>>

>> <mailto:sghe...@hotmail.com>>

>> zfs-...@googlegroups.com <mailto:zfs-...@googlegroups.com>

>> >> To visit our Web site, click on http://zfs-fuse.net/
>> >>
>> >
>> >
>>
>> --
>> To post to this group, send email to

>> zfs-...@googlegroups.com <mailto:zfs-...@googlegroups.com>

>> To visit our Web site, click on http://zfs-fuse.net/
>>
>>
>>
>>
>> --
>> zfs-fuse git repository :
>> http://rainemu.swishparty.co.uk/cgi-bin/gitweb.cgi?p=zfs;a=summary
>> --
>> To post to this group, send email to zfs-...@googlegroups.com

>> <mailto:zfs-...@googlegroups.com>

>> To visit our Web site, click on http://zfs-fuse.net/
>
> --
> To post to this group, send email to zfs-...@googlegroups.com

> <mailto:zfs-...@googlegroups.com>

[Emmanuel Anne]

Ok, the shares handling apprently works in a way a little strange in opensolaris : when you set the sharenfs property, it just checks if there is already a share for this mountpoint, and there is one, it just enables it, ignoring the new sharenfs value !

So it's quite disturbing when you make some adjustements to the sharenfs value.
In the new commit, it updates the shares based on sharenfs value always, even if there is already another share.

It means that if you already have a share for host1 on /zip
and you do zfs set sharenfs=host2:rw zip
you'll just add a share for host2 while keeping the share for host1
but then if you do
zfs set sharenfs=off zip
you'll remove all the shares for zip, including the one for host1 (zfs is supposed to handle all the shares in this case).

Also I fixed a bug where zfsfuse_findshare could check only the beginning of the path, and not the whole path in some specific conditions.

Maybe I commited a little too fast yesterday, so excuse me then !

2010/2/15 sgheeren <sghe...@hotmail.com>

[sgheeren]

Hehe

at least you're committing :) I can't say the same here so no
complaints. I'm just quick to test ? Will try tomorrow night (no time
before that)

Emmanuel Anne wrote:
> Ok, the shares handling apprently works in a way a little strange in
> opensolaris : when you set the sharenfs property, it just checks if
> there is already a share for this mountpoint, and there is one, it
> just enables it, ignoring the new sharenfs value !
>
> So it's quite disturbing when you make some adjustements to the
> sharenfs value.
> In the new commit, it updates the shares based on sharenfs value
> always, even if there is already another share.
>
> It means that if you already have a share for host1 on /zip
> and you do zfs set sharenfs=host2:rw zip
> you'll just add a share for host2 while keeping the share for host1
> but then if you do
> zfs set sharenfs=off zip
> you'll remove all the shares for zip, including the one for host1 (zfs
> is supposed to handle all the shares in this case).
>
> Also I fixed a bug where zfsfuse_findshare could check only the
> beginning of the path, and not the whole path in some specific conditions.
>
> Maybe I commited a little too fast yesterday, so excuse me then !
>

> <mailto:sghe...@hotmail.com> <mailto:sghe...@hotmail.com

> >> <mailto:sghe...@hotmail.com <mailto:sghe...@hotmail.com>>>

> <mailto:zfs-...@googlegroups.com <mailto:zfs-...@googlegroups.com>>

> >> >> To visit our Web site, click on http://zfs-fuse.net/
> >> >>
> >> >
> >> >
> >>
> >> --
> >> To post to this group, send email to
> >> zfs-...@googlegroups.com
> <mailto:zfs-...@googlegroups.com>

> <mailto:zfs-...@googlegroups.com <mailto:zfs-...@googlegroups.com>>

> >> To visit our Web site, click on http://zfs-fuse.net/
> >>
> >>
> >>
> >>
> >> --
> >> zfs-fuse git repository :
> >>
> http://rainemu.swishparty.co.uk/cgi-bin/gitweb.cgi?p=zfs;a=summary
> >> --
> >> To post to this group, send email to
> zfs-...@googlegroups.com <mailto:zfs-...@googlegroups.com>

> >> <mailto:zfs-...@googlegroups.com

> <mailto:zfs-...@googlegroups.com>>
> >> To visit our Web site, click on http://zfs-fuse.net/
> >
> > --
> > To post to this group, send email to
> zfs-...@googlegroups.com <mailto:zfs-...@googlegroups.com>

> > <mailto:zfs-...@googlegroups.com

[Emmanuel Anne]

Good you waited, I had just forgotten the most important part : the choice of the fsid parameter !

This parameter must be unique for each zfs filesystem !
So now when you change the sharenfs parameter for a zfs filesystem, it opens the /var/lib/nfs/etab file, then if it finds the mountpoint in it, it takes the fsid used by this mountpoint, otherwise it takes the bigest value it found for any fsid + 1.

This makes it compatible with any other nfs share which would use the fsid parameter, no collision risks anymore ! (assuming the other fs is already shared when zfs-fuse starts).

And now it works correctly for multiple nfs filesystems shared from zfs.
I just added the "zfs share -a" command to my zfs-fuse startup script and removed all the zfs shares from the /etc/exports file.

2010/2/16 sgheeren <sghe...@hotmail.com>

[Mike Gracy]

I'm curious as to where the info for what is being shared via 'zfs
share' is stored?
I also seem to remember someone talking about what a pain it is to
have to type in long strings to make several shares at once.
Why not keep a file in /etc/zfs called zfs-share and keep the info
there or have it updated by the daemon when you create/delete a share
so that it can be auto-parsed on startup each time?

[Emmanuel Anne]

2010/2/17 Mike Gracy <ghst...@gmail.com>

I'm curious as to where the info for what is being shared via 'zfs
share' is stored?

easy : sharenfs and sharesmb properties.
When you change these properties, the shares are updated in real time, but when you just want to share/unshare all the filesystem which have these properties != off, you use the share/unshare command.
Before changing the sharesmb was doing nothing, now you'll get a message saying that smb protocol is not supported. By the way it's probably totally useless to try to add smb support : they don't have any inode problem in samba + there doesn't seem to be a way to add a samba share from the command line or from an easy api, we would have to create a config file on the fly, which would be quite crazy.
 

I also seem to remember someone talking about what a pain it is to
have to type in long strings to make several shares at once.
Why not keep a file in /etc/zfs called zfs-share and keep the info
there or have it updated by the daemon when you create/delete a share
so that it can be auto-parsed on startup each time?

Because with this, there is no need.
+ One of the advantagesis that the strings are quite shorter than before and much more readable (since you don't have to bother with the mount point in the sharenfs property).
You setup your sharenfs property until it does what you want, then leave it this way and completely forget about nfs setup, it's all done automatically after that.

[sgheeren]

Emmanuel,

I just restested (still with d2bae0abd8750). It worked all the time.

I found out the problem: I had the fs on canmount=noauto.

I just verified with OSolaris nv_132 that indeed the fs is not shared (not even with share -a and with the fs mounted manually) if canmount==noauto. It baffled me, but at least I found out why it didn't work for me.

I might have been confused, but afaict the fs wasn't even shared when I did (NOTE: on OpenSolaris that is!):

    0. zfs set sharenfs=on pool
    1. zfs set canmount=noauto pool/fs
    2. zpool export pool
    3. zpool import pool
    4. zfs share -a                             # of course doesn't share pool/fs since not mounted
    5. zfs mount pool/fs                    # still doesn't share pool/fs
    6. zfs share -a                             # still doesn't share pool/fs
    7. zfs set canmount=on pool/fs   # still doesn't share pool/fs (!!!!)
    8. zfs share -a                              # still doesn't share pool/fs (???!!!!)
   
    9. zpool export; pool zpool import pool; # pool/fs finally shared (automatically)

I'm gonna update to your latest code since you improved it, but I thought I'd let you know!

Thanks,
Seth

[Emmanuel Anne]

Good news then. Maybe there is a problem with the canmount property then ? I never tested this !

2010/2/18 sgheeren <sghe...@hotmail.com>

[sgheeren]

As you can see, I tested this with upstream. I get the same
(unexpected?) behaviour. I haven't found any documentation yet, so I
might post this at zfs-discuss.

Emmanuel Anne wrote:
> Good news then. Maybe there is a problem with the canmount property
> then ? I never tested this !
>

> 2010/2/18 sgheeren <sghe...@hotmail.com <mailto:sghe...@hotmail.com>>

>> <mailto:sghe...@hotmail.com>>

[sgheeren]

I'm gonna update to your latest code since you improved it, but I thought I'd let you know!

I just found out that these fixes are indeed quite crucial (unless you're happy to see the exact same content listed under all nfs exports).

That being fixed, I have gotten very close to simply being able to reboot my (OSolaris nv_132!) NAS box into linux and having my nfs exports automagically appear regardless. This is the sweetest thing since zfs-fuse on linux !

There is one gotcha: I normally use
       
        zfs set sharenfs="root=sehe" pool/fs

on Open-Solaris. Needless to say this consequently results in 'exportfs failed' on linux. Perhaps I could figure out a patch to 'iron out' the platform differences, but in the mean time I'll simply try to work without it (sharenfs=on). Once it annoys me enough, I could tryb some things out :)

Thanks a million mate, I'll report back on how this beast behaves in more exercise. I'll also update zfs-fuse.net to mention the necessary steps to enable nfs sharing on debian/ubuntu!

Cheers,
Seth

PS. I'm getting 55Mb/s read speed with flac/mp3 'cat >/dev/null' style. This is more or less the same as OSol. Write speed... It turns out I had to set sharenfs='*:rw' in order to even get it rwitable... Will report later

[Emmanuel Anne]

Cool, glad to know you like it !

For the default ro : it's in the default exportfs options, I guessed it was probably a good idea to keep it.

For the incompatible options with opensolaris : it's mainly because in the manpage for sharenfs, all they say is "the options are passed to share". Except that we don't have a share command in linux, so I don't even know what the options look like. So I just invented a syntax which seemed convenient ! If you have ideas to try to improve it and to make it more compatible with opensolaris, it's good...

So I guess the root=sehe means use this option + the default options for all the hosts, equivalent to "*:root=sehe" with current zfs-fuse syntax ?

On the other subjet, the uuid, I have finished the patch for libblkid-1.1.0. With the new lib installed and after a reboot, you see the correct uuid and the correct labels appearing in the /dev/disk sub directories which is quite cool, even if it's not used by zfs-fuse for now !
I'll probably put that in the contrib directory of the git repository, and I'll try to post it to the libblkid maintainers later...
And I found a new bug in zfs-fuse : when you import a pool in mac os x, its vdev name becomes disk0s8 instead of sda8. When rebooting in linux, the 1st opened has a converted name -> sda8, works. The 2nd open gets the mac os x and fails. And then it makes an infinite loop, trying to reopen it all the time. Apparently it's an old bug, it was already here in november, so I guess there are not a lot of people with this kind of configuration. Opening by uuid would fix it, but it would probably be also a good idea to try to fix the infinite loop in case the uuid can't be found !

2010/2/20 sgheeren <sghe...@hotmail.com>

[Seth Heeren]

On Feb 20, 6:23 pm, Emmanuel Anne <emmanuel.a...@gmail.com> wrote:
> Cool, glad to know you like it !

A lot, did I mention that?

>
> For the default ro : it's in the default exportfs options, I guessed it was
> probably a good idea to keep it.

> So I guess the root=sehe means use this option + the default options for all

> the hosts, equivalent to "*:root=sehe" with current zfs-fuse syntax ?

Well, yes that is the intent. The thing is the option itself:

exportfs -o root=sehe,fsid=11,no_subtree_check '*:/bbs/media/Video'
returned an error

[Emmanuel Anne]

Yeah root=user is not documented in the exports man page, so it's probably not supported, sorry ! (the opensolaris compatibility stops here then ?)

And I just noticed there is a fsid=uuid option, which might work with the libblkid patch, I'll have to check that.

2010/2/20 Seth Heeren <sghe...@hotmail.com>

--
To post to this group, send email to zfs-...@googlegroups.com
To visit our Web site, click on http://zfs-fuse.net/