Fwd: [zfs-discuss] dedupe is in

[Alex Blewitt]

And this is why we want to continue moving this project forward :-)

Sent from my (new) iPhone

Begin forwarded message:

From: Jeff Bonwick <Jeff.B...@sun.com>
Date: 2 November 2009 13:20:53 GMT
To: "Alex Lam S.L." <alex...@gmail.com>
Cc: zfs-discuss discuss <zfs-d...@opensolaris.org>
Subject: Re: [zfs-discuss] dedupe is in

Terrific! Can't wait to read the man pages / blogs about how to use it...

Just posted one:

http://blogs.sun.com/bonwick/en_US/entry/zfs_dedup

Enjoy, and let me know if you have any questions or suggestions for
follow-on posts.

Jeff
_______________________________________________
zfs-discuss mailing list
zfs-d...@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[James Relph]

> And this is why we want to continue moving this project forward :-)
>
> Sent from my (new) iPhone
>
> Begin forwarded message:

That would be brilliant to have, really neat feature. I've got an
Xserve with 32GB of RAM in that would suit that very nicely...

Is there any way that we could help promote ZFS on OS X development at
all? Does anyone have any experience of whether offering things like
milestone rewards or sponsorship is likely to help or hinder?

James

ACSA 10.5

Website: www.themacplace.co.uk
Blog: www.themacplace.co.uk/blog.html

[Silas Baronda]

On Mon, 2 Nov 2009, James Relph wrote:

>
>> And this is why we want to continue moving this project forward :-)
>>
>> Sent from my (new) iPhone
>>
>> Begin forwarded message:
>
> That would be brilliant to have, really neat feature. I've got an Xserve
> with 32GB of RAM in that would suit that very nicely...
>
> Is there any way that we could help promote ZFS on OS X development at all?
> Does anyone have any experience of whether offering things like milestone
> rewards or sponsorship is likely to help or hinder?

We would setup a donation pool and maybe hire someone? Give them half at
the beginning half at the end? Also we could look into summer of code.

[Alex Blewitt]

On Mon, Nov 2, 2009 at 8:11 PM, James Relph <ja...@themacplace.co.uk> wrote:
>
> Is there any way that we could help promote ZFS on OS X development at all?
> Does anyone have any experience of whether offering things like milestone
> rewards or sponsorship is likely to help or hinder?

I have some things to clear up first, but I hope to be able to tackle
some of the coding going forwards. Right now, the biggest problem is
probably going to be testing any subsequent builds; there are a lot of
QA people at Apple, and I assume that at least some of them were
looking at the builds :-) One of the reasons why we probably didn't
see any intermediary builds (quite apart from the style/NDA angle) was
that intermediate builds had problems.

There's two things we can do from here;

1) Roll forward to late(r|st) builds of the onnv codebase, which will
pick up performance improvements as per
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6859997 et
al

2) Fix bugs in the Apple-specific parts of the drivers

For me, 1) is a quick win, but requires automating to do it properly.
I've got some data, and others have already posted git clones of some
parts, so we might be able to do this in the near future. 2) is more
problematic, and likely will require a greater depth of knowledge on
filesystems than I currently have :-) Having said that, things like
the finder emptying bug are ones that we've lived with for a while and
are probably unlikely to be massive show-stoppers, when combined with
some of the other work that's already gone on.

Specifically for testing: a ZFS install can only be done once per
machine (I don't know of a legal way to run OSX in VMs, at least in
VirtualBox; in any case, it wouldn't surprise me if there's
device-specific things which a VM may not expose). That means test
boxes can't have any sensible ZFS data on them (my working laptop has
my home folder with ZFS), and that the systems need to be capable of
installing, running tests, and potentially crashing. Automated tests
would be good, but if there's a problem with the ZFS box it could
easily lock up a system. There's a slim chance it could format the
disk too ... but probably not.

Anyway, the point is I have access to a second 10.5 box without ZFS
that I can probably use as my own testbed. But I don't have 10.6 yet
(was waiting for ZFS to be released ... oh well) and in any case, in
order to keep up the quality of the build we need a few people willing
to install and hammer the system on an otherwise disposable box to
approve the quality for the general public on this list to be happy
with. There's probably few people who would be willing to help out ...
but we've got to make it easy to help out, too.

Ideally we need some kind of automated build. I can host some builds
here, but I don't have the bandwidth to allow public access via HTTP
to the build server, or to publish binaries. We probably also don't
want to put too many binaries on the google groups file list so that
we don't confuse people (I'm open to suggestions) but it probably
can't be done in an easy automated way; if anyone knows different,
then feel free to correct me.

So: I need to spend some time getting together some updated bits and a
build to start the ball rolling. I need to clear a few things with my
employer before I can release code; once that's done, and we've got an
automated build rolling, we can then talk about the testing aspects,
and once gather steam, start to look for hosting and testers.

Ultimately, though, I think it's going to be something which we have
to work towards. I don't think the target market is large enough to
raise sufficient donations to hire someone to work on it full (or even
part) time; and those of us who are interested in working on it almost
certainly have other commitments (employment, other projects, families
etc.) so it will have to be something done at its own pace.

Alex

[sammy ominsky]

On 02/11/2009, at 22:50, Alex Blewitt wrote:

> in
> order to keep up the quality of the build we need a few people willing
> to install and hammer the system on an otherwise disposable box to
> approve the quality for the general public on this list to be happy
> with. There's probably few people who would be willing to help out ...
> but we've got to make it easy to help out, too.

Count me in for this.

> here, but I don't have the bandwidth to allow public access via HTTP
> to the build server, or to publish binaries.

I do.

> start to look for hosting and testers.

See above.

> Ultimately, though, I think it's going to be something which we have
> to work towards. I don't think the target market is large enough to
> raise sufficient donations to hire someone to work on it full (or even
> part) time; and those of us who are interested in working on it almost
> certainly have other commitments (employment, other projects, families
> etc.) so it will have to be something done at its own pace.

Correct.

--sambo

[lstoll]

On Nov 2, 9:50 pm, Alex Blewitt <alex.blew...@gmail.com> wrote:
>
> Specifically for testing: a ZFS install can only be done once per
> machine (I don't know of a legal way to run OSX in VMs, at least in
> VirtualBox; in any case, it wouldn't surprise me if there's
> device-specific things which a VM may not expose). That means test
> boxes can't have any sensible ZFS data on them (my working laptop has
> my home folder with ZFS), and that the systems need to be capable of
> installing, running tests, and potentially crashing. Automated tests
> would be good, but if there's a problem with the ZFS box it could
> easily lock up a system. There's a slim chance it could format the
> disk too ... but probably not.

FWIW - OSX Server can be run inside VMware Fusion - it's supported on
Apple hardware. Snapshots should work, which would be nice for this.

[James Relph]

For me, 1) is a quick win, but requires automating to do it properly.
I've got some data, and others have already posted git clones of some
parts, so we might be able to do this in the near future. 2) is more
problematic, and likely will require a greater depth of knowledge on
filesystems than I currently have :-) Having said that, things like
the finder emptying bug are ones that we've lived with for a while and
are probably unlikely to be massive show-stoppers, when combined with
some of the other work that's already gone on.

Personally 1) is the big one from my point of view, and I think a lot of people are looking at running this on servers, so things that are almost cosmetic (emptying the Trash) can wait :-)  Big advances would be things like being able to browse snapshots.

Anyway, the point is I have access to a second 10.5 box without ZFS
that I can probably use as my own testbed. But I don't have 10.6 yet
(was waiting for ZFS to be released ... oh well) and in any case, in
order to keep up the quality of the build we need a few people willing
to install and hammer the system on an otherwise disposable box to
approve the quality for the general public on this list to be happy
with. There's probably few people who would be willing to help out ...
but we've got to make it easy to help out, too.

I can certainly help there, used to having a few test machines kicking around.

Ideally we need some kind of automated build. I can host some builds
here, but I don't have the bandwidth to allow public access via HTTP
to the build server, or to publish binaries. 

Bandwidth and servers is something we can do.  We've got about 10u spare in our colo, with a 30mb line.  I could even throw a few Mac minis in there and give people who don't have test machines access to them to do their own testing.  May be able to dedicate an Xserve and SAS chassis to it as well.

Ultimately, though, I think it's going to be something which we have
to work towards. I don't think the target market is large enough to
raise sufficient donations to hire someone to work on it full (or even
part) time; and those of us who are interested in working on it almost
certainly have other commitments (employment, other projects, families
etc.) so it will have to be something done at its own pace.

It's probably best to see how it goes on, but if it ever got to the point where someone vital needed funding in order to keep things going, then I think we could actually get a bit.  We're expanding our hosted DR services at the moment, and I just look at the features that we're missing because of the lack of ZFS (or a comparable filesystem) and it makes me want to scream.  We really, really got close to moving to Solaris, but we've got an investment in Apple hardware and software both in capital and knowledge, so we're a bit stuck.  An actual top-notch ZFS deployment would be worth quite a lot to us.