Re: [zfs-macos] ZFS in iOS?

[Raoul Callaghan]

I can think of a very good reason where ZFS could be handy on iOS.

Encryption

Also, having the OS take a single snapshot b4 an upgrade, and then destroy the snapshot immediately afterwards opens up the possibility of upgrading without needing iTunes' help.

With multi-core ARM CPUs around the corner, it kind've makes sense to me...

Raoul Callaghan

Apple Senior Support Officer

eSolutions

Monash University

Building 4, Room B25

Wellington Road, Clayton

Monash University VIC 3800

03 9902 0200

Raoul.C...@monash.edu

[Alex Blewitt]

On 12 Feb 2012, at 22:35, Raoul Callaghan wrote:

> I can think of a very good reason where ZFS could be handy on iOS.
>
> Encryption

Note that you don't need ZFS for encryption (and in some cases, the encryption is still not available elsewhere too). For example, my laptop has an encrypted hard drive which uses a randomly generated key which is only accessed at boot by decrypting a password.

Secondly, it's not like ZFS is accessible from outside the phone anyway (unlike a laptop where you can easily rip out the drive) - and on the assumption you're going to come back with "But you can access it if you jailbreak it", the answer is "well, if you've jailbroken it then you have access to the unencrypted data stream anyway".

> Also, having the OS take a single snapshot b4 an upgrade, and then destroy the snapshot immediately afterwards opens up the possibility of upgrading without needing iTunes' help.

Like I said, I really doubt this buys much. In any case, my iPhone has updated itself a couple of times OTA without needing iTunes help. Downloading delta updates isn't exactly difficult; OSX has had it for years.

> With multi-core ARM CPUs around the corner, it kind've makes sense to me...

It kind have makes sense? None of it does. Hefty file system and encryption support is going to drain the battery regardless of whether it's a single or multi-core ARM CPU. Part of the significant advantage of iOS over other mobile devices is the focus on the power budget, and most of the battery length comes from not using CPUs, not throwing them away on pointless tasks.

Alex

[Daniel Bethe]

Since there's interest in discussing it, I'll just toss in my observations.  ^_^

Yeah, the iOS devices after the iPhone 3G have hardware encryption.  For example, if you use iCloud to do a remote wipe of an iPhone 3G, the device will be busy for hours doing its physical erasure, which can be disrupted by powering it off before a certain point in time during the erasure.  An iPhone 3GS and later, however, will be quick because it doesn't erase.  Everything is always encrypted in realtime.  iCloud will simply tell the device to revoke its keys so that the hardware can no longer decrypt it.  I tried the iPhone 3G procedure, but I read about the rest without trying it.

Anyway, I respect all the technical discourse on the subject.  And I apologize for having previously suggested that the only implication was that the reporter's iOS ZFS data was fabricated.  I hadn't considered the obvious possibility that it exists in an internal-only build.  I should have said "it seems to me..."  So that's probably why he hasn't followed up with more evidence or with any tests, because he doesn't have access to the device and cannot do so.

But, we also have to consider the *long* term ramifications.  "Long term" in consumer electronics technology means a year to five years.  Absolutely everything that we can now say about data integrity requirements on embedded systems, will someday be obsolete.  It will someday be as inevitably required as it now is on a server or desktop.  It will be as inevitably required in the future as 32-bit/64-bit multitasking are now on embedded devices.  About ten years ago, people said exactly the same thing about preemptive multitasking and memory protection and TCP/IP and unixlike kernels on embedded systems.  They said it about 3D hardware on phones.  About twenty years ago, people said exactly the same thing about all of those things on desktop systems.  They said that you just don't need them, period, and gave a list of reasons why cooperative multitasking is more efficient.  AmigaOS has memory protection deliberately designed out of it, for that reason.  I used to read countless arguments for DOS over Linux, just because multitasking equals hardware overhead.

Ironically, in the intervening years, people *have* ported unixlike kernels with preemptive multitasking and TCP/IP to completely obsolete 8-bit and 16-bit systems, and they've done it far more efficiently than they could have if they'd tried when that hardware was current.  http://www.contiki-os.org/  Contiki plus GUI apps and services do run in 64kB of RAM on Nintendo Entertainment System and Apple ][ and Commodore 64.  They succeeded, and broke the debacle; they have their cake, and eat it too.

Like Chris said, the iPhones of today are virtually the hardware equivalent to ZFS servers of yesteryear.  These so-called embedded devices will replace desktops at an increasing rate.  ZFS and its underlying hardware will only get more efficient in the future.  We will have phones whose CPUs or GPUs (same thing?) will have a unit for encryption and a unit for checksumming, or something like that, and ZFS will have a module that'll use it directly.

Apple pretty much have an internal port of ZFS, which they can maintain on iOS almost for 'free'.  Two people had already ported it to Mac OS as a skunkworks project in their spare time, so one more can keep building it on iOS today.  It'd be grossly negligent not to have at least a handful of prototypes for at least one person to occasionally test in the face of the oncoming head-on freight train of progress vs. Moore's law.  ;)

But I'm just talkin here.

[Alex Blewitt]

On 12 Feb 2012, at 23:19, Daniel Bethe <d...@smuckola.org> wrote:

About ten years ago, people said exactly the same thing about preemptive multitasking and memory protection and TCP/IP and unixlike kernels on embedded systems.

What? I had a QNX system back in the late '90s that could boot a 386 on an 1.44 floppy and give you multitasking, networking, GUI and a web browser. I also recall early Arm demos in the early '90s of Arm chips powered by a thermocouple attached to an x86 chip (can't recall if it was a 486) - in other words, the Arm CPU was powered from the waste heat of the Intel one. 

Like Chris said, the iPhones of today are virtually the hardware equivalent to ZFS servers of yesteryear.  These so-called embedded devices will replace desktops at an increasing rate.

Both true but missing a fairly important couple of points:

1. Unlike servers, iPhones aren't plugged in 24/7 and thus have to manage their power budget in addition to the number crunching.

2. Servers run hot. No-one wants to carry a portable heating device in their pocket. Minimising speed - even if it can go faster - helps the thermal envelope in ways that the server can exceed. 

3. Servers are always connected and thus caching can be done "In the cloud". For mobile devices, cache content must be stored locally so there may be much more transient data (which incidentally ZFS sucks at if you keep snapshots). 

4. The filesystem on an iDevice is completely abstracted away. There's no way you can see snapshots or restore to a previous one. There's no advantage that you get from ZFS if you aren't able to even consider a filesystem. 

5. Higher end functions like device striping and utilisation if different types of device (spinning magnetic, flash, memory) aren't available. The iDevice has one type of storage (flash) and the memory limitation means that it doesn't make sense to use that - and in any case the memory mapped files behave in that way anyway with page outs automatically. 

No amount of processor advances are going to change 1-4 and 5 doesn't need ZFS anyway. 

ZFS and its underlying hardware will only get more efficient in the future.

Power cables aren't going to get longer though. 

We will have phones whose CPUs or GPUs (same thing?) will have a unit for encryption and a unit for checksumming, or something like that, and ZFS will have a module that'll use it directly.

Such a minute part if ZFS and so generally useless on a phone that it doesn't matter. If my high score table on angry birds becomes corrupt through some cosmic particle, I don't want my phone to panic, nor lock the device into read only mode. Just crash the app and move on. If it keeps doing it I'll delete the app, reinstall and reimage from iCloud. 

Apple pretty much have an internal port of ZFS, which they can maintain on iOS almost for 'free'.  Two people had already ported it to Mac OS as a skunkworks project in their spare time, so one more can keep building it on iOS today.  

Most if not all of the ZFS team left Apple. The port was so far behind that one person found it better to ditch and re-port it from scratch than keep that one going. And it still doesn't change the fact that iDevices don't expose the FS to the user in any meaningful way. 

It'd be grossly negligent not to have at least a handful of prototypes for at least one person to occasionally test in the face of the oncoming head-on freight train of progress vs. Moore's law.  ;)

Moore's law was about transistors on a processor, not on battery performance. Mobile devices have that above all else as their number one priority. Wishes about what might be are way down on the list. 

Alex

[Boyd Waters]

CPU isn't a constraint for ZFS on modern mobile devices.

RAM, however, would be a significant limit for an iOS device running ZFS code.

I have no inside knowledge. But I run the release Solaris 11 on a test box here, 12 TB RAIDZ. This very lightly-loaded system (I'm the only user) shows idle CPU with a 2008-era AMD Athlon II processor. But it is severely starved for RAM with 12 GB in there.

I'm pretty sure that the deduplication feature of ZFS needs as much RAM as you can throw at it. Dedupe is not mandatory.

The Solaris 11 code is onv_151; a 138 version of the Solaris code would correspond to the last OpenSolaris release.

[Daniel Bethe]

>RAM, however, would be a significant limit for an iOS device running ZFS code.

Unless it's not!  http://mail.opensolaris.org/pipermail/osarm-dev/2009-June/000068.html

>I have no inside knowledge. But I run the release Solaris 11 on a test box here, 12 TB RAIDZ. This very lightly-loaded system (I'm the only user) shows idle CPU with a 2008-era AMD Athlon II processor. But it is severely starved for RAM with 12 GB in there.

If that's actually true of your system (as opposed to the typical assumption that a full filesystem cache equals RAM starvation) then correct your configuration!  It's in no way necessary or representative of anything at all.  I have a 16TB RAIDZ2 and 12GB RAM on MacZFS, with its inefficient duplication of Mac OS's own cache, and with no performance tuning, and no such thing happens here.  Not that any anecdotes are necessary.

ZFS is free software, and is built to be totally configurable.  The ZFS working group is designing it to be even more malleable in the future, so that it can verge in any direction and continue to be everything to everyone, without breaking compatibility.  And without requiring drastic forks like NEC's, even as progress in Moore's Law and battery life and other hardware progress obviates the need for forking.  A simple checksum on the minor amount of I/O incurred by a mobile device, is the least of any concerns.  RAM consumption is a variable to be tweaked, and maybe some removal of a bit of source code.

ZFS has been running for a long time on hardware just like today's phones.  People everyday load their phones down with apps which are far more hardware-intensive (GPS, HD moviemaking, 3G), while maintaining a totally acceptable performance profile of battery vs. speed for hours at a time.   If companies had an exclusively primary focus on power consumption instead of features, then we would not have retina displays, 3D hardware, or a plethora of radios.  Most people would have $50 flip phones which could talk for a month on one charge.  But thank goodness, that's not how anything works in the primary driving elements of progress.

This conversation has been an exercise in futile naysaying against absolutely inevitable progress, in total denial of the past and the present!  Whether this aspect of the mobile future will involve ZFS in particular, remains to be seen.  But the need is just as clear as is ZFS's ability to deliver.

And I guess everyone's viewpoints have been accounted for, in the discussion so far.  ;)

[Roddi]

Hi,

if you look at what Apple can gain and what Apple can loose by employing ZFS on an iOS device, I'd say there would be this list:

gains:

- checksums, RAID; don't laugh! If you want to have swap space on a flash drive you have to account for wearing it out sooner than later. Building a raidz pool could help tremedously using unreliable flash mem…

- snapshots: nice to have but this alone would probably not justify the switch

- send/receive snapshots: not imminently useful (for iOS) if you ask me.

losses:

- simplicity: ZFS is (probably) a lot more complicated code base than HFS+. With all those bolted-on HFS+ features this might actually not be true.

- control: Apple fully controlled HFS+ these last decades. I doubt that this control is still important for Apple today. But being able to work on their own schedule is.

So, my opinion: 

Can it be done? Probably.

Could it be useful in a few years? Definitely. Apple is badly in need of a modern FS!

Will it happen soon (i.e. 5.1): I doubt it.

The screenshot: Quite probably fake.

Roddi

[Daniel Bethe]

>gains:
>- checksums, RAID; don't laugh! If you want to have swap space on a flash drive you have to account for wearing it out sooner than later. Building a raidz pool could help tremedously using unreliable flash mem…

Hi there Roddi.  Well, you don't have, or need, swap on a mobile device.  iOS just terminates apps if needed.  On my iPhone 4 with 512MB of RAM, loaded with jailbroken extras, multitasked apps, and graphical themes, I never ever ever came close to running out of RAM.  I always had a few dozen or hundred megabytes free.  I say this in the past tense, because I don't have the iPhone 4 anymore, and I had to go back to my iPhone 3G with 128MB which is a totally different story ;)

>- snapshots: nice to have but this alone would probably not justify the switch

Yeah someone here speculated about simplifying and crashproofing the operating system updates.  Of course it can be an internal-only feature that's invisible to the user, or which is activated with the tap of a button.

>losses:
>- simplicity: ZFS is (probably) a lot more complicated code base than HFS+. With all those bolted-on HFS+ features this might actually not be true.

Well, Don completely gave up on HFS+ ;)  He said he was kinda surprised that they moved forward with it, early on!

>- control: Apple fully controlled HFS+ these last decades. I doubt that this control is still important for Apple today. But being able to work on their own schedule is.

Yeah especially with an open public ZFS working group, with optional corporate secrecy, and a proven fork of ZFS outside of Oracle.  Any remaining patent issues could be resolved in the future, and they certainly don't need critical engineering support from Oracle.  ZFS is a different place now -- less of a gamble.  I'm speculating on the relevancy of that aspect; I don't know what it's like from a megacorporate perspective.

>Will it happen soon (i.e. 5.1): I doubt it.
>The screenshot: Quite probably fake.

I'm sorry to differ, and I don't mean for my verbage to be contrary, but that's a non-sequitur.  The two ideas are not logically connected, and even if you weren't trying to connect them, there is no call for that strong of a conclusion.  :)  Maybe it is a fake.  But, ZFS was already ported to Mac OS, which is substantially the same as iOS.  I don't think there's any substantial reason why at least one person couldn't have an experimental build of it on iOS, especially if they loved on it as much as the original skunkworks people did.  Like Alex said, it could lie in an experimental or personal repository completely outside of the iOS production source code tree.  They do this kind of thing all the time.  It's a big company, full of the best of the best, many of whom are crusaders and idealists with boundless talents.

Amongst the thousands of boundary-spanning Apple employees, maybe there's one *more* photographer who just happens to be a software engineer, who had their precious iPhone photographs corrupted before they could sync them.  Or, they upgraded iOS over the air and it didn't quite make it.  And maybe they finally had enough.

There is no way to even guess how many skunkworks projects, either officially or unofficially, lie within Apple.  Nor their personal motivations.  I have one friend who claimed to have laid eyes upon a Mac OS 10 Developer Preview running on DEC Alpha hardware, in about the year 2000, at Infinite Loop.  The accompanying Apple employee rushed him past it, saying "you didn't see that."  I know a former Apple employee who ported Quicktime to Red Hat Linux (I think it was a personal skunkworks project) in about the same timeframe, and I have another friend who was subcontracted by Apple to port Quicktime to OS/2.  One Apple intern was quoted as recently having ported much of Darwin to the ARM architecture.  In the late 1990s, two Apple employees ported Linux to Mach and to Power Macs, just as an experiment.

Someone could have done it on a bet.  This is how these people are, and this is what they do.  That's the nature of the product that we hold in our hands.  Many of them are just like *us*!  We should watch the "Think Different" video again. ;-)

All of the same naysaying things could have been said about the initial porting of ZFS to Mac OS.  Especially about laptops.

Or, for some inexplicably twisted reason, maybe it's a fake.  ^_^

[Roddi]

Well, you don't have, or need, swap on a mobile device.  iOS just terminates apps if needed.

Well, users call it crashing.

 

- snapshots: nice to have but this alone would probably not justify the switch

Yeah someone here speculated about simplifying and crashproofing the operating system updates.  Of course it can be an internal-only feature that's invisible to the user, or which is activated with the tap of a button.

I know that's why I said: *probably* not justify the switch. If you want snapshots of the root partition, you will want to boot on ZFS too. Noone has laid their eyes on a mac booting on ZFS yet (or claimed to). Not saying it can't be done, though. It's just a lot of work for a small effect.

 

Well, Don completely gave up on HFS+ ;)  He said he was kinda surprised that they moved forward with it, early on!

Going on like nothing happened is always plan C. Happened to all of us! ;-)

 

Will it happen soon (i.e. 5.1): I doubt it.

The screenshot: Quite probably fake.

I'm sorry to differ, and I don't mean for my verbage to be contrary, but that's a non-sequitur.  The two ideas are not logically connected, and even if you weren't trying to connect them, there is no call for that strong of a conclusion.  :)  Maybe it is a fake.  But, ZFS was already ported to Mac OS, which is substantially the same as iOS.  I don't think there's any substantial reason why at least one person couldn't have an experimental build of it on iOS, especially if they loved on it as much as the original skunkworks people did.  Like Alex said, it could lie in an experimental or personal repository completely outside of the iOS production source code tree.  They do this kind of thing all the time.  It's a big company, full of the best of the best, many of whom are crusaders and idealists with boundless talents.

Amongst the thousands of boundary-spanning Apple employees, maybe there's one *more* photographer who just happens to be a software engineer, who had their precious iPhone photographs corrupted before they could sync them.  Or, they upgraded iOS over the air and it didn't quite make it.  And maybe they finally had enough.

There is no way to even guess how many skunkworks projects, either officially or unofficially, lie within Apple.  Nor their personal motivations.  I have one friend who claimed to have laid eyes upon a Mac OS 10 Developer Preview running on DEC Alpha hardware, in about the year 2000, at Infinite Loop.  The accompanying Apple employee rushed him past it, saying "you didn't see that."  I know a former Apple employee who ported Quicktime to Red Hat Linux (I think it was a personal skunkworks project) in about the same timeframe, and I have another friend who was subcontracted by Apple to port Quicktime to OS/2.  One Apple intern was quoted as recently having ported much of Darwin to the ARM architecture.  In the late 1990s, two Apple employees ported Linux to Mach and to Power Macs, just as an experiment.

Someone could have done it on a bet.  This is how these people are, and this is what they do.  That's the nature of the product that we hold in our hands.  Many of them are just like *us*!  We should watch the "Think Different" video again. ;-)

All of the same naysaying things could have been said about the initial porting of ZFS to Mac OS.  Especially about laptops.

Or, for some inexplicably twisted reason, maybe it's a fake.  ^_^

Well, maybe you are right. Maybe there is a ZFS on iOS project inside Apple. Still I believe that a server filesystem like ZFS makes limited sense on a mobile device. Unfortunately Apple won't have anything comparable ready for at least two more years (I think).

We'll have to wait and see...

[Daniel Bethe]

>Well, maybe you are right. Maybe there is a ZFS on iOS project inside Apple. Still I believe that a server filesystem like ZFS makes limited sense on a mobile device. Unfortunately Apple won't have anything comparable ready for at least two more years (I think).

The point is that ZFS is *not* a server filesystem; it's just a filesystem.  And iOS is not iPhone OS anymore; it's just an operating system.  Devices which run iOS are not just mobile or just luxury toys; they're personal computers which smoke the servers of yesteryear.  Given ZFS on Mac OS, then ZFS on iOS is not that difficult and no less necessary.  ^_^

Arbitrary or past-focused assumptions lead to irrelevant and needlessly negative conclusions ;)

A game developer just told me that iOS's OpenGL implementation is somewhat more advanced than Mac OS 10.7's.  :-[  It's a less ancient revision of the OpenGL specification or something like that.  I guess, at least, this discussion has provided me with yet another distraction from the fact that we have just about the most obsolete and slowest low-level graphics subsystem in the world ;-)

Again, I never meant to sound aggressive or confrontational, especially about the unknown.  I'm just trying to promote logical discourse ;)

[Daniel Bethe]

> computers which smoke the servers of yesteryear.  Given ZFS on Mac OS, then ZFS
> on iOS is not that difficult and no less necessary.  ^_^

Just to be clear, I don't mean to sound like it's trivial, because I'm not a software engineer, and I don't actually know the intimate differences between iOS and Mac OS.  I just know that iOS is largely a fork of Mac OS, and that the two cross-pollenate.  I don't know what the kernel is like, but iOS 4 has the same kind of kernel related syntax from the command line perspective.

Too bad it's all BSD licensed, so that Apple isn't required to release huge amounts of source code.  They haven't even released a mostly-buildable Darwin platform since about Mac OS 10.5.  I don't know why.

[Gregg Wonderly]

On Feb 13, 2012, at 1:05 PM, Daniel Bethe wrote:

>
>
>> computers which smoke the servers of yesteryear. Given ZFS on Mac OS, then ZFS
>> on iOS is not that difficult and no less necessary. ^_^
>
>
> Just to be clear, I don't mean to sound like it's trivial, because I'm not a software engineer, and I don't actually know the intimate differences between iOS and Mac OS. I just know that iOS is largely a fork of Mac OS, and that the two cross-pollenate. I don't know what the kernel is like, but iOS 4 has the same kind of kernel related syntax from the command line perspective

The command line interface, is a pretty simple business. The important things about the kernel, is what kinds of low level services there are that higher level kernel software can use, and specifically how the layers interoperate. What's important to understand, is that a filesystem has a lot of things that are specific to how I/O is performed, how I/O errors are dealt with, and how virtualization of parts of the stack can be performed. Remember, that ZFS, like other "Raid-like" system, creates the whole notion of "virtual devices" that become the "logical" file system storage. Given some of the attributes of how storage has changed, and how "persistence of data" has improved, it's probably more likely now that ZFS would work, than it was in iOS 3 or earlier.

> Too bad it's all BSD licensed, so that Apple isn't required to release huge amounts of source code. They haven't even released a mostly-buildable Darwin platform since about Mac OS 10.5. I don't know why.

If you want an open, do what you want with it platform, iOS is not the place to spin your wheels. Apple has always had "customer service" and "maintain our competitive edge" in their actions, or lack thereof. In this day and age, I care a lot less about the OS and its features. I just want to write software, and be able to distribute it. Requiring particular kernel hacks, or specific OS revisions, is something that I don't want to manage as part of my interaction with the customer. The Apple store takes care of those things, and people are motivated to upgrade based on what is possible and what they can afford to make possible for themselves.

Practically, ZFS doesn't "enable" anything that the device needs, really. There are some things that sound interesting, if you are wanting to "torque" with the OS internals. For example, ZFS send/receive could be used for backups I suppose. Dedup would make the outcome of a single block failure much more detrimental to the users data. Adding any amount of flash for mirror, or raidz, would add to the battery load on writes, because of the extra block writes. What specific features were you thinking about being beneficial Daniel?

Gregg Wonderly

[John Stalberg]

>> All of the same naysaying things could have been said about the initial porting of ZFS to Mac OS. Especially about laptops.
>>
>> Or, for some inexplicably twisted reason, maybe it's a fake. ^_^
> Well, maybe you are right. Maybe there is a ZFS on iOS project inside Apple. Still I believe that a server filesystem like ZFS makes limited sense on a mobile device. Unfortunately Apple won't have anything comparable ready for at least two more years (I think).
>
> We'll have to wait and see...

Who would have thought *nix would be the dominant operating system flavour on the (smart)phones just 15 years down the road in the mid 90's?

Something like: Okey, it will probably be advanced comparing to what we have today but why on earth would anyone want UNIX on a phone???

Not meaning a filesystem/LVM is as full of potentiallity as a whole operating system. Just wanted to remind us of how things can turn out when hardware get some Moore performance and software gets shaped through evolution were the fit will prosper and get fat.

Linux have been a system for embeded for some time but at around 95 I would have had a hard time see much of a benefit for a single user on such a well defined piece of hardware as a phone to run *nix :/

// John Stalberg