My First Experience with ZFS

[MarkZFS]

Sadly, it did not go so well! :(.

I followed the instructions on http://code.google.com/p/maczfs/wiki/GettingStarted

To create a simple pool of two external drives. It was simple and
worked great!

I was super happy to get up an running. So i started moving some files
over, and it copied fast and was working flawlessly.

Was able to stream some movies off the drive, no issues.

Then had to reboot the machine, and then i get a message that the 2
drives are unreadable, eject initiate, cancel message pops up. The
ZFS pool that i created is no where to be found.

My fear is that I lost my data that I put on there as there does not
see to be any way to bring back the pool.

What version of the product are you using? On what operating system?
Using 74.1.0 on 10.7

Anyway I can recover the data/pool?

Also, now when I run diskutil list, i no longer get the ZFS
designation under each of the drives.

Thanks for any help and support!

Mark

[Simon Casady]

The drives unreadable simply means that the kernel did not recognize
the drives file system. ZFS on the mac uses
a utility to try to import the drives and if that fails then you get
the pop up about the drives. The pool should have been exported on
the shutdown but if it wasn't then that may be the problem. Read the
zpool man page and try to do an import from the command line that
forces an import.

[Jason]

Always best to post what you did on the command line. Like after making a pool, zpool status, would show what you have. So we could understand which steps you did or didn't do. Did you format your drives normally and make your pool from s2 partitions or did you try the whole disk? First option is best, this has come up alot here in the forums. I know reading sucks, but.....

Did you make a a filesystem after making the pool? You mention it, but you didn't provide info.

Have you tried sudo zpool import -f yourpoolname?

Are you using USB drives????????

Jason
Sent from my iPad

[Chris Ridd]

On 25 Feb 2012, at 12:38, Jason wrote:

> Always best to post what you did on the command line. Like after making a pool, zpool status, would show what you have. So we could understand which steps you did or didn't do. Did you format your drives normally and make your pool from s2 partitions or did you try the whole disk? First option is best, this has come up alot here in the forums. I know reading sucks, but.....
>
> Did you make a a filesystem after making the pool? You mention it, but you didn't provide info.
>
> Have you tried sudo zpool import -f yourpoolname?

The pool's command history might help, assuming it imports again...

sudo zpool history yourpoolname

Chris

[MarkZFS]

Thanks for you posts!

I followed exactly step by step the GettingStarted command lines. I
am using two external Firewire drives.

# diskutil list

got the ids, disk1 & disk3

ran this command to format the drives
# diskutil partitiondisk /dev/disk1 GPTFormat ZFS %noformat% 100%

then created the pool with the first disk

# zpool create MediaHD /dev/disk1

Then ran
# zpool status MediaHD

all great!

Then i copied files off of disk3 to the newly created pool. All went
super smooth! After copy was complete. I ran:
# diskutil partitiondisk /dev/disk3 GPTFormat ZFS %noformat% 100%

Then i added the new drive to the pool by running:
# zpool add MediaHD disk3
# zpool status MediaHD

size increased to the proper amount! I was super happy.

Then I had to reboot for some other reason and boom, MediaHD was gone!

I ran diskutil list again, and for both of the drives i get the below
kind of partition map:
/dev/disk3
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *999.5 GB disk3
1: 0xEE 999.5 GB disk3s1

Looking around I got the tool gdisk (and hopefully am not doing stupid
things)

i see this when i print out the GPT:


Number Start (sector) End (sector) Size Code Name
1 40 409639 200.0 MiB EF00 EFI System
Partition
2 409640 1951889367 930.5 GiB BF01 %noformat%
121 72057594172145664 576460752303423488 224.0 EiB FFFF
122 7813586405489115136 431266031455 4.6 ZiB FFFF 鎈鯞ꅼ턭
123 16777216 2305843009230471168 1024.0 EiB FFFF 祴数
124 720575940396056576 8316288341928928303 3.3 ZiB FFFF
125 576460752303453286 16777216 7.8 ZiB FFFF

I then decided to delete the crazy 121-125 partitions...(hope i didn't
do something stupid). I am hoping that my data is still secure in the
correct partition 2.

Now what I see when I run diskutil list:
/dev/disk3
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *999.5 GB disk3
1: EFI 209.7 MB disk3s1
2: ZFS 999.2 GB disk3s2

Looks like all is back to normal! But afterward, in my flailing I
didn't know the 'zpool import -f MediaHD' command, and stupidly tried
to recreate the pool again...

I got stuck and had to use a back up to get everything back to HFS.

I now wonder about 2 things:

1. Why were my drives partitions being scrambled?
2. If I did run the import command right after cleaning the
partitions
up, would it have came right back as normal with data intact?

Thanks again for all your support.

[Jason]

In the forums this has occurred before, heck I had it happen, most of us format the disk GPT using HFS+, then use the second partitions for our pools. This way OS X mounts everything in its happy place.

Sounds like you did a good job fixing the error, but then you made a headache for yourself by recreating the pool over top of your pool.

Your first error was in not testing thoroughly understanding the sequence of commands and options. The second was committing data before testing with dummy data, system restart, disconnected disk, etc....

That said if you had posted what you were doing at each step we might have been able to help you progress without the migraine. ;)

My own experience is that your data is probably there with ZFS but you may have made it very hard to get, at least on OS X. If you really don't have proper backups of your data, I recommend downloading FreeBSD and installing in Virtualbox, and seeing if the advanced version of ZFS can work some magic.

However as I'm sure you didn't set the filesystem copies greater than 1 before copying data, it's going to be touch and go.

An example with some links http://serverfault.com/questions/297029/zfs-on-freebsd-recovery-from-data-corruption

Fingers crossed.....

Jason
Sent from my iPad

[MarkZFS]

Yet again thanks for the help!!!

Sadly, i was unable to get the data back off the ZFS, but the good
news is that my backup was up-to-date. Was an old drive so i was a
bit worried...all data has been recovered!

Yes I was being stupid, by jumping in head first before thinking it
through! However, now i tested with just one drive (just dummy data),
and the

zpool import -f MediaHD

command would have brought it back! Shoot! All those worries for
nothing!!

Well i won't forget that one!

As for create two partitions, one for HFS+ and one for ZFS, what size
would you recommend for the HFS+ part, just a token size of a few GB?
so that OS X doesn't puke on it? Going to do some testing on that one
tonight...

Also, is there a reference to the 'filesystem copies greater than 1'
setting, want to learn more about that and what it means...

My main reason for wanting ZFS, is to protect my huge media library
from bit rot, etc. that ZFS i believe does a great job of
preventing...

Thanks again!

Mark

On Feb 29, 2:59 pm, Jason <jasonbe...@belecmartin.com> wrote:
> In the forums this has occurred before, heck I had it happen, most of us format the disk GPT using HFS+, then use the second partitions for our pools. This way OS X mounts everything in its happy place.
>
> Sounds like you did a good job fixing the error, but then you made a headache for yourself by recreating the pool over top of your pool.
>
> Your first error was in not testing thoroughly understanding the sequence of commands and options. The second was committing data before testing with dummy data, system restart, disconnected disk, etc....
>
> That said if you had posted what you were doing at each step we might have been able to help you progress without the migraine. ;)
>
> My own experience is that your data is probably there with ZFS but you may have made it very hard to get, at least on OS X. If you really don't have proper backups of your data, I recommend downloading FreeBSD and installing in Virtualbox, and seeing if the advanced version of ZFS can work some magic.
>
> However as I'm sure you didn't set the filesystem copies greater than 1 before copying data, it's going to be touch and go.
>

> An example with some linkshttp://serverfault.com/questions/297029/zfs-on-freebsd-recovery-from-...

>
> Fingers crossed.....
>
> Jason
> Sent from my iPad
>

[Jason]

Just format the drive GPT HFS+ using DiskUtility. Then you know OS X likes it. Then at the command line

diskutil eraseVolume ZFS %noformat% disk2s2.

The disk needs to be one of yours and partition 2, this leaves the EFI boot partition alone.

After you make a pool and a filesystem run

Zfs get all

Copies will be default 1

zfs set copies=2 yourfilesystem

Now you have considerably more security if something goes wrong. If your doing RaidZ with 4 disks, you could use 3 for the paranoid.

You really should investigate all these options and how your going to backup from the raidz to an archive drive regularly and run scrub regularly......

As most here and on Solaris, OpenSolaris, FreeBSD, etc., will attest, Zfs is really amazing. If you search here, you will find at least 2 stories by me where I've recovered what normally would have been lost data for clients.

Enjoy, ask questions, as you grasp each function and the newbie pain will lessen. ;)

Jason
Sent from my iPad

[MarkZFS]

Thanks again for the help!

This latest tip, got me up and running. Booting with no issues.

Having fun now reading up on ZFS...lots to learn, but this is one of
the reasons why I am doing this!

On Feb 29, 5:06 pm, Jason <jasonbe...@belecmartin.com> wrote:
> Just format the drive GPT HFS+ using DiskUtility. Then you know OS X likes it. Then at the command line
>
> diskutil eraseVolume ZFS %noformat% disk2s2.
>
> The disk needs to be one of yours and partition 2, this leaves the EFI boot partition alone.
>
> After you make a pool and a filesystem run
>
> Zfs get all
>
> Copies will be default 1
>
> zfs set copies=2 yourfilesystem
>
> Now you have considerably more security if something goes wrong. If your doing RaidZ with 4 disks, you could use 3 for the paranoid.
>
> You really should investigate all these options and how your going to backup from the raidz to an archive drive regularly and run scrub regularly......
>
> As most here and on Solaris, OpenSolaris, FreeBSD, etc., will attest, Zfs is really amazing. If you search here, you will find at least 2 stories by me where I've recovered what normally would have been lost data for clients.
>
> Enjoy, ask questions, as you grasp each function and the newbie pain will lessen. ;)
>
> Jason
> Sent from my iPad
>

[Jason]

Excellent news!!!

Jason
Sent from my iPad

[MarkZFS]

One more quirk i noticed in my testing...

Seems that TIme Machine automatically excludes the ZFS pool from
backing up. Every time that I manually change the TIme Machines
options to take the pool off the excluded list...it automatically puts
it right back on!

Is this just happening to me, did i miss something to get this
working? Would love Time Machine to backup my pool as well as my HFS
disks as well.

btw, to be clear, my Time Machine backup is an external Firewire HFS
formatted drive.

Thanks again!

Mark

On Mar 5, 2:17 pm, Jason <jasonbe...@belecmartin.com> wrote:
> Excellent news!!!
>
> Jason
> Sent from my iPad
>

[Raoul Callaghan]

So,

Just thinking out aloud here...

Knowing that the kernel has to be on HFS so EFI can find it etc etc...

How would one go about putting the kernel on a HFS partition (and any other files for that matter) and the rest of the OS on ZFS?

I'm assuming some trickery would be required when the kernel wants to find the rest of the OS on the ZFS partition...

Anyone care to comment?

Cheers,

Raoul.

[Daniel Bethe]

No, Time Machine will only back up a local HFS+ source.  I don't know the reason; I don't know whether it's because Time Machine denies the existence of non-HFS+ volumes, or whether Time Machine is hardcoded to HFS+'s method of notification of filesystem changes in the way that Spotlight is.  I just created a FAQ entry for it, so if anyone wants to contribute an explanation, I'd be glad to document that background information.

So fickle.  >:-/

---

From: MarkZFS <markanth...@med.ge.com>
To: zfs-macos <zfs-...@googlegroups.com>
Sent: Tuesday, 6 March 2012, 15:26
Subject: [zfs-macos] Re: My First Experience with ZFS

[Daniel Bethe]

Here's what I personally know about it, but we could use a new level of information involving proposals for ingenius workarounds.  

http://code.google.com/p/maczfs/wiki/FAQ#Can_I_boot_from_ZFS?__Can_I_convert_totally_from_HFS+_to_ZFS?

We have to logically sort out the whole series of issues, stemming from the boot loader and then probably case-sensitivity.  I don't know if Chameleon (the mac clone boot loader) or rEFIt (the free boot loader intended to augment Boot Camp on Mac hardware and whatnot) can help or what.  I know it's been discussed in the past, so it'd be great if you search the mailing list archives and post links to the relevant threads if you sort out anything authoritative.

In short, I don't know, but I'd like to authoritatively document it for someone who could theoretically solve it.  Go.

---

From: Raoul Callaghan <tan...@mac.com>
To: zfs-...@googlegroups.com
Sent: Tuesday, 6 March 2012, 15:32
Subject: [zfs-macos] booting up using ZFS...

[Alex Blewitt]

The kernel has HFS compiled in so that it can get off the ground. Even if you could get that the kextcache needs to be on the boot volume, assuming that /System etc could be on ZFS. 

As well as the case sensitivity issue, there are an increasing number of functions that wouldn't work unless searchfs is implemented. OSX should be able to run on a case sensitive volume even if some apps don't support it - I've been running on case sensitive HFS for the OS for some time. 

But there are issues like what you do with /tmp and /swap which are a whole different set of problems. MacZFS doesn't support FIFO transfers (which is why you can't pipe a zfs send) and this is sure to break something. 

Whilst OSX on ZFS is a nice idea, it's very unlikely to happen without Apple's involvement because it will need much lower level integration than exists at present. 

Alex

Sent from my iPad

[Alex Blewitt]

TIme Machine works because it fakes snapshots using hard links on directories. Normally Unix filesystems only permit hard links on files, not directories, but one of the HFS++++ extensions over the years was to support hard linked directories. 

When TM writes a snapshot it effectively dupes the same (unchanged) directories with a hard link, thus saving space. It only creates new directories for those with changed contents. 

So the destination needs to be an HFS+ and therefore the source is too. I suspect it relies on the HFS unique identifier for the directory so that it can map to existing backups but the net effect is you end up needing it on both sides. 

I've heard some people having success with an HFS+ DMG on ZFS as a TM destination. You need to do the defaults dance (defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1) and/or touch the .com.apple.timemachine.supported file. 

Alex

Dan, feel free to wikify :-)

Sent from my iPad

[Daniel Bethe]

Yo Alex.  I kinda did wikify, here.

http://code.google.com/p/maczfs/wiki/FAQ#Can_I_host_my_Time_Machine_archives_on_a_ZFS_target?

Check those two consecutive entries to see if there's anything else to add.  I am probably forgetting some factoids from the mailing list history.

---

From: Alex Blewitt <alex.b...@gmail.com>
To: "zfs-...@googlegroups.com" <zfs-...@googlegroups.com>
Cc: "zfs-...@googlegroups.com" <zfs-...@googlegroups.com>
Sent: Tuesday, 6 March 2012, 17:33
Subject: Re: [zfs-macos] Re: My First Experience with ZFS

[splash]

I, too,  followed http://code.google.com/p/maczfs/wiki/GettingStarted and created a pool with a slice of an existing disk.  Basically, the same thing happened to me and my zfs partition is listed as HFS+.

 

 In following GettingStarted I took the following text at face value:

-quote-

If you created a zpool using any method not explicitly listed above, such as ...

• a zpool created using an OS other than Mac OS
• loopback files, as with mkfile 1g file0 ; zpool create filepool /Users/user/file0
• an MBR or APM or any other partition type
• not using Mac OS's ZFS partition type

... then it will work just fine, but you'll need a manual startup and shutdown procedure as described here

-end quote-

I did follow the method "explicitly listed above" and still disaster!    I think that page should direct every new user to install the unmount/export script from the FAQ.  Every user should go through a reboot cycle before considering "getting started" is complete.

I tried the scripts and reboot cycle on a different machine, different ZFS filesystem (a mirror this time) and it works fine.

Now if only I can get over my latent anxiety over ZFS reliability...

 

[Jason]

Not sure I'm clear on the issue, but I think your upset at the fact you didn't test everything is working from a boot on forwards before using ZFS, right? If so, this is an assumed procedure as you should do with any new drive, new file system. 

If I've misunderstood, please correct me. 

Jason

Sent from my iPhone

[splash]

To clarify, I believe that the "getting started" page asserts that no special shutdown/startup process (scripted or manual) is necessary if you follow the posted procedure.  My experience indicates that assertion is wrong.  

 

It would be far better if the page suggests, unconditionally, that the shutdown/startup scripts be installed and then directs the user to write a test file into their new filesystem, reboot and check that the file is successfully preserved.  Having those scripts installed protects you when your spouse/child/SO shuts down the machine or you just plain forget that you have a special filesystem in play.

 I'm still interested in getting back my ZFS partition. One slice of my boot drive contained that partition.  Does anyone know if it is possible to modify a GUID partition map on the boot drive while in single user mode? I tried in normal mode and got a message saying the drive was busy.  

 

[Lucien Pullen]

Also sprach splash at 7/12/12 9:32 AM:

> I'm still interested in getting back my ZFS partition. One slice of
> my boot drive contained that partition. Does anyone know if it is
> possible to modify a GUID partition map on the boot drive while in
> single user mode? I tried in normal mode and got a message saying
> the drive was busy.

You need to boot from the install disk to edit the boot drive.

[Alex Blewitt]

On 11 Jul 2012, at 03:20, splash <richar...@ueagraduate.com> wrote:

> I think that page should direct every new user to install the unmount/export script from the FAQ

No, that is not sane advice. The sane advice is to follow the recommended procedure of creating a ZFS partition. That is the "supported" way and other ways are just workaround that may or may not work.

We should update the wiki to remove the "should work fine" statement and replace with a "not supported but you might find useful". That will bring expectations in line with reality.

Alex

[Daniel Bethe]

> We should update the wiki to remove the "should work fine" statement
> and replace with a "not supported but you might find useful". That
> will bring expectations in line with reality.

Hi Alex.  I just amended it as such.  Also, I had recently amended the FAQ to mention the second set of startup/shutdown scripts that I had posted to the list.  I never got any real feedback on them, and I believe I've denoted them as being experimental and better than nothing.

[Alex Blewitt]

Thanks Dan, that's great.

Alex