The problem shouldn't be so much the kernel as all the binaries you are
using: changing the directory structure will break everything that reads
directory entries - this includes important binaries like 'ls'
(obviously), the shell (for filename globbing, name completion etc),
'make', etc.

I'd go for the compatability hack: creating a new filesystem from
scratch for the new kernel is simply too painful (as well as making
debugging pretty mcuh impossible...  much easier to start off with just
one partition with the extended setup, and test out all the new binaries
on that one first).  I'd also suggest you add a "readdir()" system call
while you are at it, which checks the magic number and returns the
correct type of directory entry.  That way binaries compiled to use it
will be able to function correctly on both the new and old type of
filesystem.

There are a couple of alternatives you could do:

 (a) go for a complete vfs (virtual filesystems) interface.  This would
make it possible to later add totally different types of filesystems.
It's more general, but it's also a lot more coding.  This is the
approach linux uses, and it gives the possibility to mount msdos etc
filesystems transparently.

 (b) do the long filenames by fooling around with several consecutive
minix-type directory entries.  Depending on how you do it, you can make
old binaries see only th first characters of a extended filename, while
new binaries see them all.  Besides, this means you won't waste a full
64-char direntry for short files, but instead use several entries only
when necessary.  The downside is that it's a bit more work in the
kernel.

The directory entries in (b) could be made to work by using a magic
cookie at the end of a filename to mean that the filename continues in
the next entry (which has a inode nr of 0 to make old programs ignore
it).  It could look something like this:

file "really_long_name", use '\000\377' as continuation marker:

        .word inode_number
        .byte 'really_long_\000\377'    /* first 12 chars */
        .word 0x0000                    /* next entry is a continuation */
        .byte 'filename\0'              /* rest of the filename */

This can be extended to any filename length, and old programs will see
only the first 12 characters, but they should work otherwise.  I think
somebody implemented something like this.  Patches, anybody? Naturally,
you'd have to make sure that none of your current filesystems happen to
have the '\000\377' sequence at the end of a directory entry, but that
should be simple enough.

                Linus