NFS within a Jail?!

[blackfriar]

Hi everybody!
I'm wondering if it's possible to run in a "clear fashion" an NFS server
within a jail on FreeBSD 9.0.

I'm having some issues that make me think this is not supposed to work.
I've googled it but I couldn't find much especially on releases prior
5!!

A quick tip would be great .... I don't really wanna waste hours on this
not very relevant issue.

Many thanks in advance.

[Fbsd8]

Quick answer is "No, NFS only runs on the host system".

Long answer is; NFS requires rpcbind to function. rpcbind is dependent
on a network stack. Jails do not have their own network stack, they use
the hosts network stack. There is some experimental software to give
each jail its own network stack but I sure would not deploy a production
system based on this.


_______________________________________________
freebsd-...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questi...@freebsd.org"

[blackfriar]

Clear!
Thanks a lot.

[Adam Vande More]

On Wed, Aug 8, 2012 at 9:02 AM, Fbsd8 <fb...@a1poweruser.com> wrote:
>
> Long answer is; NFS requires rpcbind to function. rpcbind is dependent on
> a network stack. Jails do not have their own network stack, they use the
> hosts network stack.

Dealing with this has been SOP practice in jails since their inception.
See man 8 jail. The best way to run the NFS server is from the jail.
Running it host side is the hard part.

> There is some experimental software to give each jail its own network
> stack but I sure would not deploy a production system based on this.
>

There are a number of people who have reached the
opposite decision concerning VIMAGE/VNET enabled jails. They are much
easier to work with and provide nice capabilities.

--
Adam Vande More

[Mark Felder]

On Wed, 08 Aug 2012 11:55:51 -0500, Adam Vande More
<amvan...@gmail.com> wrote:

>
> There are a number of people who have reached the
> opposite decision concerning VIMAGE/VNET enabled jails. They are much
> easier to work with and provide nice capabilities.

I tried it on 9.0-RELEASE and was able to cause kernel panics quite
easily, so I've avoided it. I expect things to be worked out by the end of
the 9.x train and/or 10.0. They certainly do provide some nice
capabilities, though.

[Fbsd8]

Adam Vande More wrote:
> On Wed, Aug 8, 2012 at 9:02 AM, Fbsd8 <fb...@a1poweruser.com> wrote:
>> Long answer is; NFS requires rpcbind to function. rpcbind is dependent on
>> a network stack. Jails do not have their own network stack, they use the
>> hosts network stack.
>
>
> Dealing with this has been SOP practice in jails since their inception.
> See man 8 jail. The best way to run the NFS server is from the jail.
> Running it host side is the hard part.
>

http://www.freebsd.org/cgi/query-pr.cgi?pr=133265
The jail code maintainer says NFS server/client will not work jailed. So
since you say this is SOP (standard operation procedure) then why is
there no documentation available on how to do it? All the Google hits
for "NFS running from Freebsd jail" end with no one got it to work. Have
you done this? Do you have a procedure to post or know of a posted
procedure giving step-by-step sequence to get NFS running in a jail with
or without VIMAGE/VNET for Release 8.x or 9.x versions?

>
>> There is some experimental software to give each jail its own network
>> stack but I sure would not deploy a production system based on this.
>>
>

> There are a number of people who have reached the
> opposite decision concerning VIMAGE/VNET enabled jails. They are much
> easier to work with and provide nice capabilities.
>

Still doesn't change the FACT it's experimental!

[Wojciech Puchar]

>>
>> Many thanks in advance.
>>
>
> Quick answer is "No, NFS only runs on the host system".
>

but user space nfsd works. in ports - unfsd

[Blackfriar]

Really? Is that stable enough to serve files for months without disruption?

Wojciech Puchar <woj...@wojtek.tensor.gdynia.pl> wrote:

>>>
>>> Many thanks in advance.
>>>
>>
>> Quick answer is "No, NFS only runs on the host system".
>>
>but user space nfsd works. in ports - unfsd

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[Fbsd8]

Wojciech Puchar wrote:
>>>
>>> Many thanks in advance.
>>>
>>
>> Quick answer is "No, NFS only runs on the host system".
>>
> but user space nfsd works. in ports - unfsd
>
>

Close but no cigar.

In the ports system it's named unfs3 and described as

UNFS3 is a user-space implementation of the NFSv3 server specification.
It provides a daemon for the MOUNT and NFS protocols, which are used by
NFS clients for accessing files on the server.
http://unfs3.sourceforge.net/

Now here is the KEY. No where does it say it has the "server" side
function, only the client side. But it does have 18k downloads even
though it's labeled as beta version.

So the question back to Wojciech Puchar is; are you running this unfs3
in a client jail on one pc and the server side in a jail on the host?

Put another way can you confirm from experience that this unfs3 port has
both client and service side support and that it does work when
installed in a client jail and host server jail?

[Ruben de Groot]

On Fri, Aug 10, 2012 at 08:36:04AM -0400, Fbsd8 typed:

> Wojciech Puchar wrote:
> >>>
> >>>Many thanks in advance.
> >>>
> >>
> >>Quick answer is "No, NFS only runs on the host system".
> >>
> >but user space nfsd works. in ports - unfsd
> >
> >
>
> Close but no cigar.
>
> In the ports system it's named unfs3 and described as
>
> UNFS3 is a user-space implementation of the NFSv3 server specification.

^^^^^^^^^^^^^^^^^^^^^^^^^^

> It provides a daemon for the MOUNT and NFS protocols, which are used by

^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^ ^^^^

> NFS clients for accessing files on the server.
> http://unfs3.sourceforge.net/
>
> Now here is the KEY. No where does it say it has the "server" side
> function, only the client side.

It sais so very clearly.

Ruben

[Fbsd8]

Ruben de Groot wrote:
> On Fri, Aug 10, 2012 at 08:36:04AM -0400, Fbsd8 typed:
>> Wojciech Puchar wrote:
>>>>> Many thanks in advance.
>>>>>
>>>> Quick answer is "No, NFS only runs on the host system".
>>>>
>>> but user space nfsd works. in ports - unfsd
>>>
>>>
>> Close but no cigar.
>>
>> In the ports system it's named unfs3 and described as
>>
>> UNFS3 is a user-space implementation of the NFSv3 server specification.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
>> It provides a daemon for the MOUNT and NFS protocols, which are used by
> ^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^ ^^^^
>> NFS clients for accessing files on the server.
>> http://unfs3.sourceforge.net/
>>
>> Now here is the KEY. No where does it say it has the "server" side
>> function, only the client side.
>
> It sais so very clearly.
>
> Ruben
>
>

What you say: Just 2 words further on in that sentence "used by NFS
clients".

Read as unfs3 is run as client to access kernel nfs on host.

No where in any documentation on unfs3 does it ever say unfs3 has to be
run on both client and service side. That sentence infers that unfs3 is
only run on the client side. Now I do concede that the writer of that
sentence may not be a native English speaker and as such fails to
express fully the intent of what he was trying to say. Maybe unfs3
really has to be used on both the host server side and clients side for
it to work. Or this may just be a case of the author being to close to
the trees to see the forest.

[Adam Vande More]

On Wed, Aug 8, 2012 at 1:57 PM, Fbsd8 <fb...@a1poweruser.com> wrote:
>
> Dealing with this has been SOP practice in jails since their inception.
>> See man 8 jail. The best way to run the NFS server is from the jail.
>> Running it host side is the hard part.
>>
>>

> http://www.freebsd.org/cgi/**query-pr.cgi?pr=133265<http://www.freebsd.org/cgi/query-pr.cgi?pr=133265>

> The jail code maintainer says NFS server/client will not work jailed. So
> since you say this is SOP (standard operation procedure) then why is there
> no documentation available on how to do it? All the Google hits for "NFS
> running from Freebsd jail" end with no one got it to work. Have you done
> this? Do you have a procedure to post or know of a posted procedure giving
> step-by-step sequence to get NFS running in a jail with or without
> VIMAGE/VNET for Release 8.x or 9.x versions?

That PR is about mounting a fs in a jail, specifically one proved by NFS.
What does that have to do with the OP's question? It's quite clear you
didn't read the full thing.

> Still doesn't change the FACT it's experimental!

Which is your sole reason for poo=pooing it? Are you talking about the
arbitrary line between experimental and production? I wonder how a piece
of functionality transitions from experimental to production...is it
possible we get there by promoting mindshare of the new piece instead of
FUD?

--
Adam Vande More

[Ruben de Groot]

On Fri, Aug 10, 2012 at 10:42:45AM -0400, Fbsd8 typed:

> Ruben de Groot wrote:
> >On Fri, Aug 10, 2012 at 08:36:04AM -0400, Fbsd8 typed:
> >>Wojciech Puchar wrote:
> >>>>>Many thanks in advance.
> >>>>>
> >>>>Quick answer is "No, NFS only runs on the host system".
> >>>>
> >>>but user space nfsd works. in ports - unfsd
> >>>
> >>>
> >>Close but no cigar.
> >>
> >>In the ports system it's named unfs3 and described as
> >>
> >>UNFS3 is a user-space implementation of the NFSv3 server specification.
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^
> >>It provides a daemon for the MOUNT and NFS protocols, which are used by
> > ^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^ ^^^^
> >>NFS clients for accessing files on the server.
> >> http://unfs3.sourceforge.net/
> >>
> >>Now here is the KEY. No where does it say it has the "server" side
> >>function, only the client side.
> >
> >It sais so very clearly.
> >
> >Ruben
> >
> >
> What you say: Just 2 words further on in that sentence "used by NFS
> clients".
>
> Read as unfs3 is run as client to access kernel nfs on host.

No you read wrong. It is a userspace daemon that provides nfs service.
It is a daemon. Not a client.

> No where in any documentation on unfs3 does it ever say unfs3 has to be
> run on both client and service side.

Because it is not a client.

[Wojciech Puchar]

>>>
>>> Now here is the KEY. No where does it say it has the "server" side
>>> function, only the client side.

/usr/ports/net/unfs3

Even if someone "proved" that it doesn't do NFS server work, i will
continue to use it as NFS server, because it works very well ;)

[Fbsd8]

Wojciech Puchar wrote:
>>>>
>>>> Now here is the KEY. No where does it say it has the "server" side
>>>> function, only the client side.
>
> /usr/ports/net/unfs3
>
> Even if someone "proved" that it doesn't do NFS server work, i will
> continue to use it as NFS server, because it works very well ;)
>
>

Since your the expert on unfs3 because you have it working, would you
share some technical configuration information with us?

Such as

What statements do you have in the host:server and remote:client
/etc/rc.conf to auto start them at boot time?

How do you disable the kernel nfs version so it don't interfere with unfs3?

What does your export file look like on both the host:server and
remote:client sides.

Then about unfs3 performance; how many concurrent remote:clients do you
service? Doe's access elapse time get longer as more concurrent
remote:clients come online?

Do you run unfs3 in a jail on both the host:server and remote:client sides?

Are there any sysctl nob settings needed to make unfs3 run in a jail?

[Wojciech Puchar]

>
> Since your the expert on unfs3 because you have it working, would you share
> some technical configuration information with us?
>
> Such as
>
> What statements do you have in the host:server and remote:client /etc/rc.conf
> to auto start them at boot time?

in client - as with any NFS, use kernel klient.

on server - just run unfsd anywhere, like /etc/rc.local

Do you really cannot use any program without rc.d script? ;)

> How do you disable the kernel nfs version so it don't interfere with unfs3?

Just don't enable it in /etc/rc.conf

You may have it in kernel. i did, now i don't. both works.

> What does your export file look like on both the host:server and
> remote:client sides.

as described in manual of unfsd.
basics are same, details are not.

>
> Then about unfs3 performance; how many concurrent remote:clients do you
> service? Doe's access elapse time get longer as more concurrent

over 60 but not high load clients so performance doesn't matter. That's X
terminals booting over NFS.

Sometimes i do more - example is net-booting windoze PC to be able to do
some recovery OR backup large amount of data to server.

There is no practical performance difference in that settings.

BUT - do make configure, then search where fsync is called and comment it
out.

Right - not conformant, but the performance difference on writes are
enormous.

Unless you do such a stupid things like running database servers over NFS,
you don't need this conformance. Just do it.

> Do you run unfs3 in a jail on both the host:server and remote:client sides?

Not now. but tried. unfs doesn't need ANY special kernel calls. It runs
just like any program using UDP/TCP communication. jail/no jail doesn't
make a difference. As with most programs.

In kernel point of view it is just a program that use TCP/IP stack and
open/read/write/readdir/etc. Nothing else.

> Are there any sysctl nob settings needed to make unfs3 run in a jail?

as above.

[Fbsd8]

http://forums.freebsd.org/showthread.php?t=29968&highlight=nfsd

Found this which I think says it all at the conclusion.

March 30th, 2012
Received some information from the FreeBSD mailing list and apparently
exporting NFS from an jailed environment is not possible. For those who
have "managed" (by heavy tweaking of sysctl.conf) to export the NFS
probably have these concerns: 1) Security may have been compromised on
their own jails as a result of tweaks and 2) Even if you manage to
export the NFS share under such strained boundary conditions, it may
cause problems in some of the application's you would like to use (eg:
tinderbox) finally, 3) If you try to use net/unfs3 and succeed to export
NFS, this will not have a very fast (ro) transport rate and will have
many (rw) speed limitations.


My personal conclusion is to wait until the default kernel version of
nfs is updated to be jail-friendly before I try using nfs in jails.

[blackfriar]

Well, yes. That was my immediate decision. Just to find an
alternative/temporary way and to wait for something more reliable and
standard.
I don't really have the critical need to proceed with such a setup right
now and it's good to know a little bit more about the status of such a
feature and what others think and tried/managed to do about it.

I really wanna thank you all for your valuable inputs.
Have a good rest of the weekend.

[Wojciech Puchar]

> http://forums.freebsd.org/showthread.php?t=29968&highlight=nfsd
>
> Found this which I think says it all at the conclusion.
>

you are truly funny.