As part of a joint effort with Sun/Oracle to augment the Lustre file
system with ZFS support, we've been engaged in porting ZFS natively to
the Linux kernel. So far we have pretty much everything working
except the ZPL - this is because Lustre interfaces directly with the
DMU and the ZPL was not a priority for us. However, we connected with
folks at KQ Infotech who are also interested in a Linux kernel port
and they are working on the ZPL so it is on the way.
Anyway, the fruits of our labor are available here http://github.com/behlendorf/zfs/.
I don't know to what extent it's practical for the zfs-fuse community
and our project to collaborate. But since there is ZFS expertise on
both sides and a lot of common code, I wanted to propose that we at
least consider how we might help each other out.
Any thoughts?
Brian Behlendorf (no not the apache one, the other one)
If Sun/Oracle is involved, does this mean that the ZFS code (or at
least the part you are porting to the Linux kernel) will be licensed
in a way to allow it to be distributed generally? zfs-fuse began as a
way to dodge the licenses problem and simplify development. Are you
planning to merge your work upstream into the mainline kernel?
On Jun 3, 4:33 pm, Brian Behlendorf <brianbehlendo...@gmail.com>
wrote:
> As part of a joint effort with Sun/Oracle to augment the Lustre file
> system with ZFS support, we've been engaged in porting ZFS natively to
> the Linux kernel. So far we have pretty much everything working
> except the ZPL - this is because Lustre interfaces directly with the
> DMU and the ZPL was not a priority for us. However, we connected with
> folks at KQ Infotech who are also interested in a Linux kernel port
> and they are working on the ZPL so it is on the way.
> Anyway, the fruits of our labor are available herehttp://github.com/behlendorf/zfs/.
> I don't know to what extent it's practical for the zfs-fuse community
> and our project to collaborate. But since there is ZFS expertise on
> both sides and a lot of common code, I wanted to propose that we at
> least consider how we might help each other out.
> Any thoughts?
> Brian Behlendorf (no not the apache one, the other one)
On Jun 3, 4:48 pm, Stan Seibert <vols...@gmail.com> wrote:
> The obvious question is the license issue:
I thought this might be the first question asked. Below is an extract
from the zfs github project wiki FAQ page which tries to address this,
http://wiki.github.com/behlendorf/zfs/faq.
"What about the “licensing” issue?"
In a nutshell, the issue is that the Linux kernel which is licensed
under the GNU General Public License is incompatible with ZFS which is
licensed under the Sun CDDL. While both the GPL and CDDL are open
source licenses their terms are such that it is impossible to
simultaneously satisfy both licenses. This means that a single derived
work of the Linux kernel and ZFS cannot be legally distributed.
One way to resolve this issue is to implement ZFS in user space with
FUSE where it is not considered a derived work of the kernel. This
approach resolves the licensing issues but it has some technical
drawbacks. There is another option though. The CDDL does not restrict
modification and release of the ZFS source code which is publicly
available as part of OpenSolaris. The ZFS code can be modified to
build as a CDDL licensed kernel module which is not distributed as
part of the Linux kernel. This makes a Native ZFS on Linux
implementation possible if you are willing to download and build it
yourself.
Interesting idea, I tried to build it to see how it works, and got stuck on spl (solaris porting layer). Even a google search could not find it. Anyway maybe it's because it's too later for that now, I'll have another look tomorrow !
2010/6/4 Brian Behlendorf <brianbehlendo...@gmail.com>
> On Jun 3, 4:48 pm, Stan Seibert <vols...@gmail.com> wrote: > > The obvious question is the license issue:
> I thought this might be the first question asked. Below is an extract > from the zfs github project wiki FAQ page which tries to address this, > http://wiki.github.com/behlendorf/zfs/faq.
> "What about the “licensing” issue?"
> In a nutshell, the issue is that the Linux kernel which is licensed > under the GNU General Public License is incompatible with ZFS which is > licensed under the Sun CDDL. While both the GPL and CDDL are open > source licenses their terms are such that it is impossible to > simultaneously satisfy both licenses. This means that a single derived > work of the Linux kernel and ZFS cannot be legally distributed.
> One way to resolve this issue is to implement ZFS in user space with > FUSE where it is not considered a derived work of the kernel. This > approach resolves the licensing issues but it has some technical > drawbacks. There is another option though. The CDDL does not restrict > modification and release of the ZFS source code which is publicly > available as part of OpenSolaris. The ZFS code can be modified to > build as a CDDL licensed kernel module which is not distributed as > part of the Linux kernel. This makes a Native ZFS on Linux > implementation possible if you are willing to download and build it > yourself.
> -- > To post to this group, send email to zfs-fuse@googlegroups.com > To visit our Web site, click on http://zfs-fuse.net/
> On Jun 3, 4:48 pm, Stan Seibert <vols...@gmail.com> wrote:
> > The obvious question is the license issue:
> I thought this might be the first question asked. Below is an extract
> from the zfs github project wiki FAQ page which tries to address this,http://wiki.github.com/behlendorf/zfs/faq.
> "What about the “licensing” issue?"
> In a nutshell, the issue is that the Linux kernel which is licensed
> under the GNU General Public License is incompatible with ZFS which is
> licensed under the Sun CDDL. While both the GPL and CDDL are open
> source licenses their terms are such that it is impossible to
> simultaneously satisfy both licenses. This means that a single derived
> work of the Linux kernel and ZFS cannot be legally distributed.
> One way to resolve this issue is to implement ZFS in user space with
> FUSE where it is not considered a derived work of the kernel. This
> approach resolves the licensing issues but it has some technical
> drawbacks. There is another option though. The CDDL does not restrict
> modification and release of the ZFS source code which is publicly
> available as part of OpenSolaris. The ZFS code can be modified to
> build as a CDDL licensed kernel module which is not distributed as
> part of the Linux kernel. This makes a Native ZFS on Linux
> implementation possible if you are willing to download and build it
> yourself.
Is there a howto or simple instructions on how to go about using that
code? I think I have to build the kernel module and the utilities.
Which kernel have you built and tested it with?
I will try it in a VM.
-devsk
On Jun 3, 3:33 pm, Brian Behlendorf <brianbehlendo...@gmail.com>
wrote:
> As part of a joint effort with Sun/Oracle to augment the Lustre file
> system with ZFS support, we've been engaged in porting ZFS natively to
> the Linux kernel. So far we have pretty much everything working
> except the ZPL - this is because Lustre interfaces directly with the
> DMU and the ZPL was not a priority for us. However, we connected with
> folks at KQ Infotech who are also interested in a Linux kernel port
> and they are working on the ZPL so it is on the way.
> Anyway, the fruits of our labor are available herehttp://github.com/behlendorf/zfs/.
> I don't know to what extent it's practical for the zfs-fuse community
> and our project to collaborate. But since there is ZFS expertise on
> both sides and a lot of common code, I wanted to propose that we at
> least consider how we might help each other out.
> Any thoughts?
> Brian Behlendorf (no not the apache one, the other one)
Let me try and answer some of the above questions with single reply.
> I had hoped that perhaps the licensing situation had changed due to
> the Sun/Oracle/Lustre involvement.
We have been working on this for some time now and have been strongly
urging Sun/Oracle to make a change to the licensing. I'm sorry to say
we have not yet had any luck. However, we did feel it was time to
move
our efforts to a public forum.
> Is there a howto or simple instructions on how to go about using that
> code? I think I have to build the kernel module and the utilities.
Yes. I've tried to put this kind of information on the projects wiki
page.
There are instructions on how to build both the spl and zfs code at
the
following page. If you run in to an issue or gotcha please open an
issue
so I can either fix it or update the documention.
As for how to use the code I have posted a few examples but more of
this sort of thing is welcome! Here's an example of creating an ext2
file
system using a zvol block device and taking a snapshot and mounting
it read-only. The basic idea is it should work pretty much exactly
like it
does on OpenSolaris. Load the zfs module stack and your good to go
with the exact same utilities.
Quite a few actually. I mainly do my development using the current
Fedora kernels but I've been careful to support all the way back to
the
rhel5 2.6.18 based kernels. There is fairly elaborate autoconf build
system which detects the interfaces provided by your kernel and
tries to do the right thing. I've done testing with the following
kernels
but there's a decent chance it'll work with a random 2.6.18 - 2.6.32
based kernel. If it doesn't please open an issue.
Finally one more thing deserves special mention and that is that it
works
best with a 64-bit kernel. There are still issues with the limited
virtual
address space in the kernel on 32-bit platforms. Since our target
server
platforms for lustre are all 64-bit I haven't invested too much effort
in
dealing with 32-bit issues.
Thanks for the interest and please let me know what you think. I'd
love
the feedback!
Next try then, this time I got stuck with these undefined symbols in spl module : first_online_pgdat next_zone I tracked these 2 in the mm directory of the kernel sources, so I guess it has something to do with the memory model ? Which option exactly enables these symbols in the kernel then ?
2010/6/4 Brian Behlendorf <brianbehlendo...@gmail.com>
> Let me try and answer some of the above questions with single reply.
> > I had hoped that perhaps the licensing situation had changed due to > > the Sun/Oracle/Lustre involvement.
> We have been working on this for some time now and have been strongly > urging Sun/Oracle to make a change to the licensing. I'm sorry to say > we have not yet had any luck. However, we did feel it was time to > move > our efforts to a public forum.
> > Is there a howto or simple instructions on how to go about using that > > code? I think I have to build the kernel module and the utilities.
> Yes. I've tried to put this kind of information on the projects wiki > page. > There are instructions on how to build both the spl and zfs code at > the > following page. If you run in to an issue or gotcha please open an > issue > so I can either fix it or update the documention.
> As for how to use the code I have posted a few examples but more of > this sort of thing is welcome! Here's an example of creating an ext2 > file > system using a zvol block device and taking a snapshot and mounting > it read-only. The basic idea is it should work pretty much exactly > like it > does on OpenSolaris. Load the zfs module stack and your good to go > with the exact same utilities.
> > Which kernel have you built and tested it with?
> Quite a few actually. I mainly do my development using the current > Fedora kernels but I've been careful to support all the way back to > the > rhel5 2.6.18 based kernels. There is fairly elaborate autoconf build > system which detects the interfaces provided by your kernel and > tries to do the right thing. I've done testing with the following > kernels > but there's a decent chance it'll work with a random 2.6.18 - 2.6.32 > based kernel. If it doesn't please open an issue.
> Finally one more thing deserves special mention and that is that it > works > best with a 64-bit kernel. There are still issues with the limited > virtual > address space in the kernel on 32-bit platforms. Since our target > server > platforms for lustre are all 64-bit I haven't invested too much effort > in > dealing with 32-bit issues.
> Thanks for the interest and please let me know what you think. I'd > love > the feedback!
> Thanks, > Brian Behlendorf
> -- > To post to this group, send email to zfs-fuse@googlegroups.com > To visit our Web site, click on http://zfs-fuse.net/
Well after quite a few tests : - kernel must be < 2.6.33 (did it with 2.6.32.15). - must use sparse memory model, allow hotplug memory (had to compile my own kernel for that). - In the end, everything installs correctly, the test script works, but I can't mount anything (zpool create seems to work, zfs list and zpool list work correctly too, but zfs mount does absolutely nothing and even ignores its arguments after mount, zpool import returns immediately and nothing is mounted).
So is it still just highly experimental and totally unusable for now or did I miss something?
> Next try then, this time I got stuck with these undefined symbols in spl > module : > first_online_pgdat > next_zone > I tracked these 2 in the mm directory of the kernel sources, so I guess it > has something to do with the memory model ? > Which option exactly enables these symbols in the kernel then ?
> 2010/6/4 Brian Behlendorf <brianbehlendo...@gmail.com>
>> Let me try and answer some of the above questions with single reply.
>> > I had hoped that perhaps the licensing situation had changed due to >> > the Sun/Oracle/Lustre involvement.
>> We have been working on this for some time now and have been strongly >> urging Sun/Oracle to make a change to the licensing. I'm sorry to say >> we have not yet had any luck. However, we did feel it was time to >> move >> our efforts to a public forum.
>> > Is there a howto or simple instructions on how to go about using that >> > code? I think I have to build the kernel module and the utilities.
>> Yes. I've tried to put this kind of information on the projects wiki >> page. >> There are instructions on how to build both the spl and zfs code at >> the >> following page. If you run in to an issue or gotcha please open an >> issue >> so I can either fix it or update the documention.
>> As for how to use the code I have posted a few examples but more of >> this sort of thing is welcome! Here's an example of creating an ext2 >> file >> system using a zvol block device and taking a snapshot and mounting >> it read-only. The basic idea is it should work pretty much exactly >> like it >> does on OpenSolaris. Load the zfs module stack and your good to go >> with the exact same utilities.
>> > Which kernel have you built and tested it with?
>> Quite a few actually. I mainly do my development using the current >> Fedora kernels but I've been careful to support all the way back to >> the >> rhel5 2.6.18 based kernels. There is fairly elaborate autoconf build >> system which detects the interfaces provided by your kernel and >> tries to do the right thing. I've done testing with the following >> kernels >> but there's a decent chance it'll work with a random 2.6.18 - 2.6.32 >> based kernel. If it doesn't please open an issue.
>> Finally one more thing deserves special mention and that is that it >> works >> best with a 64-bit kernel. There are still issues with the limited >> virtual >> address space in the kernel on 32-bit platforms. Since our target >> server >> platforms for lustre are all 64-bit I haven't invested too much effort >> in >> dealing with 32-bit issues.
>> Thanks for the interest and please let me know what you think. I'd >> love >> the feedback!
>> Thanks, >> Brian Behlendorf
>> -- >> To post to this group, send email to zfs-fuse@googlegroups.com >> To visit our Web site, click on http://zfs-fuse.net/
Thanks for taking the time to build the projects and
kick the tires a bit. I'm sorry you had some trouble
getting it to build it is still a work in progress,
> So is it still just highly experimental and totally unusable for now or did
> I miss something?
Yes, the project it is still under very active development.
As I mentioned in my original post and on the wiki the
ZFS posix layer has not yet been implemented. Since
lustre directly links against DMU this isn't critical for our
purposes. That said there is work being done to get this
working. For the moment the only usable user space
interface is the ZVOL.
My main motivation for posting the code even if it's not
100% complete was to make it available for those who
are interested in pursuing a native Linux port. Either just
to follow along as development progresses or to actively
help develop and test.
I think he mentioned ZPL (ZFS Posix Layer) is not there yet. So, none
of the FS stuff is going to work. He is looking towards KQ Infotech
folks who haven't done anything since Oct 2009 (just a post on their
site) and emails asking them for details resulted in nothing. Whatever
they have, they are holding it to their chest.
The other option Brian is looking towards is zfs-fuse itself. If we
can utilize zfs-fuse somehow to insert the missing ZPL part, then we
can have a functional ZFS. But I am not sure about this approach
because the biggest advantage of native linux port is lost:
performance! Or may be I don't understand the overall architecture.
At least that's what I understood.
Brian, that zvol example is really cool BTW.
-devsk
On Jun 4, 9:17 am, Emmanuel Anne <emmanuel.a...@gmail.com> wrote:
> Well after quite a few tests :
> - kernel must be < 2.6.33 (did it with 2.6.32.15).
> - must use sparse memory model, allow hotplug memory (had to compile my own
> kernel for that).
> - In the end, everything installs correctly, the test script works, but I
> can't mount anything (zpool create seems to work, zfs list and zpool list
> work correctly too, but zfs mount does absolutely nothing and even ignores
> its arguments after mount, zpool import returns immediately and nothing is
> mounted).
> So is it still just highly experimental and totally unusable for now or did
> I miss something?
> Geting back to my good old zfs-fuse for now ! ;-)
> 2010/6/4 Emmanuel Anne <emmanuel.a...@gmail.com>
> > Next try then, this time I got stuck with these undefined symbols in spl
> > module :
> > first_online_pgdat
> > next_zone
> > I tracked these 2 in the mm directory of the kernel sources, so I guess it
> > has something to do with the memory model ?
> > Which option exactly enables these symbols in the kernel then ?
> > 2010/6/4 Brian Behlendorf <brianbehlendo...@gmail.com>
> >> Let me try and answer some of the above questions with single reply.
> >> > I had hoped that perhaps the licensing situation had changed due to
> >> > the Sun/Oracle/Lustre involvement.
> >> We have been working on this for some time now and have been strongly
> >> urging Sun/Oracle to make a change to the licensing. I'm sorry to say
> >> we have not yet had any luck. However, we did feel it was time to
> >> move
> >> our efforts to a public forum.
> >> > Is there a howto or simple instructions on how to go about using that
> >> > code? I think I have to build the kernel module and the utilities.
> >> Yes. I've tried to put this kind of information on the projects wiki
> >> page.
> >> There are instructions on how to build both the spl and zfs code at
> >> the
> >> following page. If you run in to an issue or gotcha please open an
> >> issue
> >> so I can either fix it or update the documention.
> >> As for how to use the code I have posted a few examples but more of
> >> this sort of thing is welcome! Here's an example of creating an ext2
> >> file
> >> system using a zvol block device and taking a snapshot and mounting
> >> it read-only. The basic idea is it should work pretty much exactly
> >> like it
> >> does on OpenSolaris. Load the zfs module stack and your good to go
> >> with the exact same utilities.
> >> > Which kernel have you built and tested it with?
> >> Quite a few actually. I mainly do my development using the current
> >> Fedora kernels but I've been careful to support all the way back to
> >> the
> >> rhel5 2.6.18 based kernels. There is fairly elaborate autoconf build
> >> system which detects the interfaces provided by your kernel and
> >> tries to do the right thing. I've done testing with the following
> >> kernels
> >> but there's a decent chance it'll work with a random 2.6.18 - 2.6.32
> >> based kernel. If it doesn't please open an issue.
> >> Finally one more thing deserves special mention and that is that it
> >> works
> >> best with a 64-bit kernel. There are still issues with the limited
> >> virtual
> >> address space in the kernel on 32-bit platforms. Since our target
> >> server
> >> platforms for lustre are all 64-bit I haven't invested too much effort
> >> in
> >> dealing with 32-bit issues.
> >> Thanks for the interest and please let me know what you think. I'd
> >> love
> >> the feedback!
> >> Thanks,
> >> Brian Behlendorf
> >> --
> >> To post to this group, send email to zfs-fuse@googlegroups.com
> >> To visit our Web site, click onhttp://zfs-fuse.net/
Yes and I don't even have a clue on how this zpl layer is supposed to work in the 1st place... Maybe the code can still be used anyway, I know wa had someone who tried to convert the zfs-fuse code to kernel space and gave up because he got some very unstable results, so maybe it's a better start this time. Anyway still a lot of work to do apparently...
> I think he mentioned ZPL (ZFS Posix Layer) is not there yet. So, none > of the FS stuff is going to work. He is looking towards KQ Infotech > folks who haven't done anything since Oct 2009 (just a post on their > site) and emails asking them for details resulted in nothing. Whatever > they have, they are holding it to their chest.
> The other option Brian is looking towards is zfs-fuse itself. If we > can utilize zfs-fuse somehow to insert the missing ZPL part, then we > can have a functional ZFS. But I am not sure about this approach > because the biggest advantage of native linux port is lost: > performance! Or may be I don't understand the overall architecture.
> At least that's what I understood.
> Brian, that zvol example is really cool BTW.
> -devsk
> On Jun 4, 9:17 am, Emmanuel Anne <emmanuel.a...@gmail.com> wrote: > > Well after quite a few tests : > > - kernel must be < 2.6.33 (did it with 2.6.32.15). > > - must use sparse memory model, allow hotplug memory (had to compile my > own > > kernel for that). > > - In the end, everything installs correctly, the test script works, but > I > > can't mount anything (zpool create seems to work, zfs list and zpool list > > work correctly too, but zfs mount does absolutely nothing and even > ignores > > its arguments after mount, zpool import returns immediately and nothing > is > > mounted).
> > So is it still just highly experimental and totally unusable for now or > did > > I miss something?
> > Geting back to my good old zfs-fuse for now ! ;-)
> > 2010/6/4 Emmanuel Anne <emmanuel.a...@gmail.com>
> > > Next try then, this time I got stuck with these undefined symbols in > spl > > > module : > > > first_online_pgdat > > > next_zone > > > I tracked these 2 in the mm directory of the kernel sources, so I guess > it > > > has something to do with the memory model ? > > > Which option exactly enables these symbols in the kernel then ?
> > > 2010/6/4 Brian Behlendorf <brianbehlendo...@gmail.com>
> > >> Let me try and answer some of the above questions with single reply.
> > >> > I had hoped that perhaps the licensing situation had changed due to > > >> > the Sun/Oracle/Lustre involvement.
> > >> We have been working on this for some time now and have been strongly > > >> urging Sun/Oracle to make a change to the licensing. I'm sorry to say > > >> we have not yet had any luck. However, we did feel it was time to > > >> move > > >> our efforts to a public forum.
> > >> > Is there a howto or simple instructions on how to go about using > that > > >> > code? I think I have to build the kernel module and the utilities.
> > >> Yes. I've tried to put this kind of information on the projects wiki > > >> page. > > >> There are instructions on how to build both the spl and zfs code at > > >> the > > >> following page. If you run in to an issue or gotcha please open an > > >> issue > > >> so I can either fix it or update the documention.
> > >> As for how to use the code I have posted a few examples but more of > > >> this sort of thing is welcome! Here's an example of creating an ext2 > > >> file > > >> system using a zvol block device and taking a snapshot and mounting > > >> it read-only. The basic idea is it should work pretty much exactly > > >> like it > > >> does on OpenSolaris. Load the zfs module stack and your good to go > > >> with the exact same utilities.
> > >> > Which kernel have you built and tested it with?
> > >> Quite a few actually. I mainly do my development using the current > > >> Fedora kernels but I've been careful to support all the way back to > > >> the > > >> rhel5 2.6.18 based kernels. There is fairly elaborate autoconf build > > >> system which detects the interfaces provided by your kernel and > > >> tries to do the right thing. I've done testing with the following > > >> kernels > > >> but there's a decent chance it'll work with a random 2.6.18 - 2.6.32 > > >> based kernel. If it doesn't please open an issue.
> > >> Finally one more thing deserves special mention and that is that it > > >> works > > >> best with a 64-bit kernel. There are still issues with the limited > > >> virtual > > >> address space in the kernel on 32-bit platforms. Since our target > > >> server > > >> platforms for lustre are all 64-bit I haven't invested too much effort > > >> in > > >> dealing with 32-bit issues.
> > >> Thanks for the interest and please let me know what you think. I'd > > >> love > > >> the feedback!
> > >> Thanks, > > >> Brian Behlendorf
> > >> -- > > >> To post to this group, send email to zfs-fuse@googlegroups.com > > >> To visit our Web site, click onhttp://zfs-fuse.net/
On Jun 4, 10:03 am, devsk <dev...@gmail.com> wrote:
> I think he mentioned ZPL (ZFS Posix Layer) is not there yet. So, none
> of the FS stuff is going to work. He is looking towards KQ Infotech
Actually, in the long term I would love to support both a native in-
kernel posix layer and a fuse based posix layer. The way the code is
structured you actually build the same ZFS code once in the kernel as
a set of modules and a second time as a set of shared libraries. The
in-kernel version is used by Lustre, the ZVOL, and will eventually be
used by the native posix layer. Currently the shared libraries are
only used by ztest for regression testing. However, they could be
used to form the basis of a fuse based implementation. The major
missing bit is the glue to tie it to fuse which you guys have already
shown can be done. IMHO the real advantage of this would be a shared
code base.
> so maybe it's a better start this time. Anyway still a lot
> of work to do apparently...
I hope so. I've spent the last year building a stable foundation in
the kernel so when we finally get to the posix layer, which is
probably the hardest part, we have a good chance of success. There's
a lot of work to be done but I think it is all very do able with
enough effort!
hi brian could you please summurize your ideas with a archtectural picture of both solutions, ie the zfs-fuse and the native ones? and also give us what do you expect of the native zfs for linux performances and features wrt the zfs-fuse solution, considering if i well understood that the bridge between the kernel and the both solutions is still fuse? could we for instance expect building a root zfs for launching the os in mirror? regards
Envoyé de mon iPhone
Le 5 juin 2010 à 00:44, Brian Behlendorf <brianbehlendo...@gmail.com> a écrit :
> On Jun 4, 10:03 am, devsk <dev...@gmail.com> wrote: >> I think he mentioned ZPL (ZFS Posix Layer) is not there yet. So, none >> of the FS stuff is going to work. He is looking towards KQ Infotech
> Actually, in the long term I would love to support both a native in- > kernel posix layer and a fuse based posix layer. The way the code is > structured you actually build the same ZFS code once in the kernel as > a set of modules and a second time as a set of shared libraries. The > in-kernel version is used by Lustre, the ZVOL, and will eventually be > used by the native posix layer. Currently the shared libraries are > only used by ztest for regression testing. However, they could be > used to form the basis of a fuse based implementation. The major > missing bit is the glue to tie it to fuse which you guys have already > shown can be done. IMHO the real advantage of this would be a shared > code base.
>> so maybe it's a better start this time. Anyway still a lot >> of work to do apparently...
> I hope so. I've spent the last year building a stable foundation in > the kernel so when we finally get to the posix layer, which is > probably the hardest part, we have a good chance of success. There's > a lot of work to be done but I think it is all very do able with > enough effort!
> -- > To post to this group, send email to zfs-fuse@googlegroups.com > To visit our Web site, click on http://zfs-fuse.net/
2010/6/5 Brian Behlendorf <brianbehlendo...@gmail.com>
> Actually, in the long term I would love to support both a native in- > kernel posix layer and a fuse based posix layer. The way the code is > structured you actually build the same ZFS code once in the kernel as > a set of modules and a second time as a set of shared libraries. The > in-kernel version is used by Lustre, the ZVOL, and will eventually be > used by the native posix layer. Currently the shared libraries are > only used by ztest for regression testing. However, they could be > used to form the basis of a fuse based implementation. The major > missing bit is the glue to tie it to fuse which you guys have already > shown can be done. IMHO the real advantage of this would be a shared > code base.
After a quick look, it's not so easy. Fuse interfaces the lowlevel file functions directly, so for example it needs zfs_readdir. Which is only in the zfs kernel module apparently (module/zfs/zfs_vnops.c). I guess ztest is only using the ioctl calls which makes it easier to build. Here you would have to reinvent completely zfs-fuse by extracting all the file functions from the kernel layer, no magical glue which would make everything to work together here...
Have you thought about a 3rd solution : interfacing directly the kernel filesystem functions without using a compatibility layer ? A lot of the function interfaces look very similar, now I didn't review everything so I don't know how hard it would be, but it seems to be worth a try at least !
Now I have hardly any kernel experience (just forget about that). But it is pretty obvious that interfacing to the fuse interface is much easier than programming a ZPL from scratch.
My syllogism would be
* If you want to be in-kernel,
AND
* You wish to share codebase with the fuse implementation
THEN
* You should obviously look at the fuse kernel module
Only in that way can you hope to reuse significant bits of zfs-fuse. In a thought experiment you can easily show that it should be a welldefined effort to:
(a) fork the fuse module (b) adapt the interface so you can have the same functional blocks BUT don't need to cross-over to user space at the fuse-layer (c) patch-up where a switch to user-space is still desired
So the real /job/ is in (c) only. I have a suspicion that it would be feasible to have 1:1 port into kernel space pretty soonish (unless there are technical reasons why the libzfs socket .e.g. could not continue to be a socket interface, e.g.). I have another suspicion that the important bits that now rely on being in userspace should be relatively few and probably not needed anyway once you are in kernel (e.g. the mounting operations)
> 2010/6/5 Brian Behlendorf <brianbehlendo...@gmail.com>
>> Actually, in the long term I would love to support both a native in- >> kernel posix layer and a fuse based posix layer. The way the code is >> structured you actually build the same ZFS code once in the kernel as >> a set of modules and a second time as a set of shared libraries. The >> in-kernel version is used by Lustre, the ZVOL, and will eventually be >> used by the native posix layer. Currently the shared libraries are >> only used by ztest for regression testing. However, they could be >> used to form the basis of a fuse based implementation. The major >> missing bit is the glue to tie it to fuse which you guys have already >> shown can be done. IMHO the real advantage of this would be a shared >> code base.
> After a quick look, it's not so easy. > Fuse interfaces the lowlevel file functions directly, so for example it > needs zfs_readdir. > Which is only in the zfs kernel module apparently (module/zfs/zfs_vnops.c). > I guess ztest is only using the ioctl calls which makes it easier to build. > Here you would have to reinvent completely zfs-fuse by extracting all the > file functions from the kernel layer, no magical glue which would make > everything to work together here...
> Have you thought about a 3rd solution : interfacing directly the kernel > filesystem functions without using a compatibility layer ? A lot of the > function interfaces look very similar, now I didn't review everything so I > don't know how hard it would be, but it seems to be worth a try at least !
On Jun 3, 6:04 pm, Brian Behlendorf <brianbehlendo...@gmail.com>
wrote:
> implementation possible if you are willing to download and build it
> yourself.
Modules being unavailable in the stock kernel have never really been a
big
issue, IMHO. For example, DRBD was only recently merged, BTRFS was an
external module, Ceph only recently got merged, the Nvidia and ATI
binary
drivers of course, Xen largely exists outside the kernel...
So, I've never seen what the big deal was about having a kernel module
that
wouldn't be released in the kernel.org kernel is. It seemed to me
that we
had some people in power turn up their noses at ZFS, and that slowed
down
adoption of ZFS under Linux by, what, 5+ years?
Anyway, ZFS-FUSE has mostly saved the day (though there are people who
shy away from it because it's FUSE and it's "gotta be slow" or
similar.
Thanks so much to both the FUSE and the Lustre efforts to bring us
Linux
users a modern file-system now. :-)
> On Jun 3, 6:04 pm, Brian Behlendorf <brianbehlendo...@gmail.com>
> wrote:
> > implementation possible if you are willing to download and build it
> > yourself.
> Modules being unavailable in the stock kernel have never really been a
> big
> issue, IMHO. For example, DRBD was only recently merged, BTRFS was an
> external module, Ceph only recently got merged, the Nvidia and ATI
> binary
> drivers of course, Xen largely exists outside the kernel...
> So, I've never seen what the big deal was about having a kernel module
> that
> wouldn't be released in the kernel.org kernel is. It seemed to me
> that we
> had some people in power turn up their noses at ZFS, and that slowed
> down
> adoption of ZFS under Linux by, what, 5+ years?
I don't think the issue is inclusion in the kernel.org release so much
as the illegality of shipping a binary package with "kernel ZFS". I
don't mind compiling stuff, but it would be a lot nicer if ZFS could
be included in the Ubuntu/Fedora/SuSE/RHEL repositories some day.
btrfs is already in Ubuntu 10.04, and might be an installation option
in two releases.
Hey you'll wonder about that when the code will actually work. For now it's just a base, probably a good base, but it needs very motivated people to make it work...
> On Jun 6, 1:56 pm, jafo <jaf...@gmail.com> wrote: > > On Jun 3, 6:04 pm, Brian Behlendorf <brianbehlendo...@gmail.com> > > wrote:
> > > implementation possible if you are willing to download and build it > > > yourself.
> > Modules being unavailable in the stock kernel have never really been a > > big > > issue, IMHO. For example, DRBD was only recently merged, BTRFS was an > > external module, Ceph only recently got merged, the Nvidia and ATI > > binary > > drivers of course, Xen largely exists outside the kernel...
> > So, I've never seen what the big deal was about having a kernel module > > that > > wouldn't be released in the kernel.org kernel is. It seemed to me > > that we > > had some people in power turn up their noses at ZFS, and that slowed > > down > > adoption of ZFS under Linux by, what, 5+ years?
> I don't think the issue is inclusion in the kernel.org release so much > as the illegality of shipping a binary package with "kernel ZFS". I > don't mind compiling stuff, but it would be a lot nicer if ZFS could > be included in the Ubuntu/Fedora/SuSE/RHEL repositories some day. > btrfs is already in Ubuntu 10.04, and might be an installation option > in two releases.
> -- > To post to this group, send email to zfs-fuse@googlegroups.com > To visit our Web site, click on http://zfs-fuse.net/
> Hey you'll wonder about that when the code will actually work.
> For now it's just a base, probably a good base, but it needs very motivated
> people to make it work...
> > On Jun 6, 1:56 pm, jafo <jaf...@gmail.com> wrote:
> > > On Jun 3, 6:04 pm, Brian Behlendorf <brianbehlendo...@gmail.com>
> > > wrote:
> > > > implementation possible if you are willing to download and build it
> > > > yourself.
> > > Modules being unavailable in the stock kernel have never really been a
> > > big
> > > issue, IMHO. For example, DRBD was only recently merged, BTRFS was an
> > > external module, Ceph only recently got merged, the Nvidia and ATI
> > > binary
> > > drivers of course, Xen largely exists outside the kernel...
> > > So, I've never seen what the big deal was about having a kernel module
> > > that
> > > wouldn't be released in the kernel.org kernel is. It seemed to me
> > > that we
> > > had some people in power turn up their noses at ZFS, and that slowed
> > > down
> > > adoption of ZFS under Linux by, what, 5+ years?
> > I don't think the issue is inclusion in the kernel.org release so much
> > as the illegality of shipping a binary package with "kernel ZFS". I
> > don't mind compiling stuff, but it would be a lot nicer if ZFS could
> > be included in the Ubuntu/Fedora/SuSE/RHEL repositories some day.
> > btrfs is already in Ubuntu 10.04, and might be an installation option
> > in two releases.
> > --
> > To post to this group, send email to zfs-fuse@googlegroups.com
> > To visit our Web site, click onhttp://zfs-fuse.net/
On 6 June 2010 22:57, Stan Seibert <vols...@gmail.com> wrote:
> Sure, I hope this works! I'd use it. :)
> Just want to make sure people realize that this can't be bundled with > any Linux distribution without Oracle deciding to change the ZFS > license.
It can certainly be bundled with Linux distributions in much the same way as other kernel-tainting modules. Ubuntu already bundles in a ton of binary-only drivers (Broadcom WiFi/ NVidia/ fglrx) and this would be no different. What makes you think it can't be included?
On Mon, Jun 7, 2010 at 08:11, David Sanders <dsuzukisand...@gmail.com> wrote: > On 6 June 2010 22:57, Stan Seibert <vols...@gmail.com> wrote: >> Sure, I hope this works! I'd use it. :)
>> Just want to make sure people realize that this can't be bundled with >> any Linux distribution without Oracle deciding to change the ZFS >> license.
> It can certainly be bundled with Linux distributions in much the same > way as other kernel-tainting modules. Ubuntu already bundles in a ton > of binary-only drivers (Broadcom WiFi/ NVidia/ fglrx) and this would > be no different. What makes you think it can't be included?
It's different because it's almost the complete opposite :P. This can only be legally distributed in source form, to be compiled by the user (who can't then distribute the resulting binary of course).
I believe what you're saying though is that it has the same legal status as the shim layer between the binary blob and the kernel - it just happens that rather than being a thin wrapper that has to be built by the end user, it's the whole thing. I don't know if module-assistant (or other distributions' equivalents) could be used as-is, but I'd expect so. The real question is whether you can persuade distributions that it is worth going to that effort. They understandably don't like having to jump through hoops to work around legal problems unless there is tremendous demand - probably no point in even trying until the ZPL is usable.
On Jun 7, 5:28 am, Aneurin Price <aneurin.pr...@gmail.com> wrote:
> It's different because it's almost the complete opposite :P. This can
> only be legally distributed in source form, to be compiled by the user
> (who can't then distribute the resulting binary of course).
What is it that makes it not possible to distribute binary copies? I
mean,
OpenSolaris and Nexenta are distributing binary copies of CDDL
sources.
As I understand it, the CDDL is based on the Mozilla Public License,
and
Mozilla licensed code is distributed as binaries all over the place.
> It's different because it's almost the complete opposite :P. This can > only be legally distributed in source form, to be compiled by the user > (who can't then distribute the resulting binary of course).
Are you sure about this? BSD et al all distribute this as binary - what evidence do you have that the CDDL denies this distribution? I thought it was just a standard "incompatible with GPL" problem.
> btrfs is already in Ubuntu 10.04, and might be an installation option
> in two releases.
btrfs is already an installation option in Fedora, but the one test
install
with it for the root fs I did with F13 hung. Could have been
hardware, but
that same install worked fine with ext4.
I've been running a ported zfsstress against btrfs on a F13 test
machine
here. The stock kernel had some issues, but after "yum update" it has
been
running fine for 35 days so far.
That said, it still feels like btrfs has a long way to go to catch up
with
ZFS. It wasn't that long ago that you couldn't delete btrfs snapshots
for
example. File-systems just always seem to take a lot longer than
you'd
like to get to production-ready status.
