Status of development Aug 21st

[lun...@lundman.net]

* The upstream-0.6.1 merge has been pulled into Master branch
* SPL has had a license overhaul
* ARC evict fixes
* Reclaim thread implemented
* ACLs support added
* ZVOL fixes
* NFS and AFP export fixes

We have now reached a state where I feel confident enough to run this version myself. It can (will?)
perhaps panic still, but general-usable appears good.

Going into each section in more detail;

* Upstream 0.6.1 merge (now 'master' branch)

The merge was less than 1 hour of work, and future merges should be faster. (There is a pre-0.6.2 branch already started). This
brought us bug fixes, lz4 compression and a few other goodies.

* SPL license

I regret not having removed the license when I first did the porting job, since it has caused some people stress, but I have finally
gone back to rectify it.  Most header files are empty, those were rewritten without license. The headers originating from
Solaris, I had to go back to IllumOS repository, and bring the right license back. Those that I wrote, or brought in from MacZFS
got a new CDDL header, and copyright lines.

Some of the autoconf files (in particular those in config/ directory and the top level include/ directory) are still from ZOL, and GNU. We
will eventually rewrite those as well (our autoconf is already considerably different). But those are only used during build, and
not in the final binary.

* ARC evict

There were issues with ARC's metadata growing without limit, on other ZFS platforms metadata is not limited to the "arc_metadata_limit"
at all, but controlled by other data being evicted. In OSX we had to enforce metadata evicts as well. For systems with less than 4GB of
memory, ARC is further halved, to avoid depleting the zalloc.1024 zone.

* Reclaim thread

It seems silly to have tried to avoid the reclaim thread for so long, when it was quite easy and much cleaner to implement. It did
highlight that very-frequent (10,000+ per second) cv_signal() or cv_broadcast() can/will "kill" the reclaim thread, or rather, the cv_wait()
call will never return. This could be our SPL implementation of condvars. Currently, we do not signal the reclaim thread, but
rather sleep 500ms, and empty the list "statically". List rarely goes over 10,000 elements in 500ms. But clearly a suspect
condvar implementation is less than desirable, since ZFS do use CVs in a few places, albeit with low frequency.

* ACL support

MacZFS's ACL were incomplete, with 'ilovezfs' assistance, we have made ACLs support be complete.


* ZVOL fixes

ZVOL work rather well, partitioning and using Disk Utility work without issue. HFS+ formatting, and use with Timemachine
work as expected. ZFS is supposed to automatically unmount any ZVOL mounted at "export", but there may be
lingering issues here. ZFS also creates a compatibility symlink tree under "/var/run/zfs/zvol/dsk/$POOL/$volume"
which points to the IOKit device node, for example "/dev/disk4".  This is so we can support those users who prefer legacy
naming scheme, rather than the unpredictable, automatic, IOKit naming.

* NFS and AFP fixes.

Automatic NFS export using the "sharenfs" property does not work, but manual export of the ZFS mount-point appears to work
well. The "liberate" AFP is not required, since we do not care what "id" we use in statvfs, I simply made it compatible with AFP.

However, both should be enhanced such that sharenfs/shareafp properties will work directly.



* Future:

snapdir support is lacking
OSX integration is lacking (the automount bundle needs to be implemented)
OSX notifications?

GUI ? I've often been tempted to make a HTML5/WebSockets front-end for ZFS, since I have done these before, but
I detest HTML and well, can't make GUIs, so I am unsure if I want to start this. I looked at ZFS.Guru, but it is a bit of a mess.