Allocation size mismatch

[Joel Cretan]

Hello,

Does the information reported on a macFUSE volume by diskutil come from my statfs() implementation?

I have a mounted volume whose Allocation Size appears wrong:

```

$ diskutil info /dev/disk4

   <snip>

   Disk Size:                 5.4 GB (5367775232 Bytes) (exactly 10483936 512-Byte-Units)
   Device Block Size:         512 Bytes

   Volume Total Space:        8.6 GB (8588361728 Bytes) (exactly 16774144 512-Byte-Units)
   Volume Used Space:         5.0 MB (4980736 Bytes) (exactly 9728 512-Byte-Units) (0.1%)
   Volume Free Space:         8.6 GB (8583380992 Bytes) (exactly 16764416 512-Byte-Units) (99.9%)
   Allocation Block Size:     131072 Bytes

```

This results in the wrong total and free size of the volume, which should only be 5.4GB. Assuming this information comes from the struct statvfs *stbuf that I populate in my implementation of statfs(), I dumped the values I put there:

```

FuseHFS_statfs(): stbuf->f_bsize: 81920
FuseHFS_statfs(): stbuf->f_frsize: 81920
FuseHFS_statfs(): stbuf->f_blocks: 65524
FuseHFS_statfs(): stbuf->f_bfree: 65486

```

So I can see that diskutil is calculating the 8.6 GB as the correct number of blocks (65524) times its wrong block size (131072). Somehow my 0x14000 block size became 0x20000 between the time I returned it and when diskutil reported this, unless there's somewhere else I should be looking. AFAICT this only happens for volumes over 2GB. It's entirely possible this is my own bug somewhere, but there is nowhere else that I touch any struct statvfs. Am I thinking about this correctly that the Allocation Size should be the stbuf->f_frsize or stbuf->f_bsize that I return, or is there somewhere else this information may come from?

Thanks,

Joel

[Benjamin Fleischer]

Hello Joel,

The default block size for macFUSE volumes is 4096 bytes. The minimum block size is 128 bytes. In case you don't want to use 4096 bytes as block size you need to set the actual block size at mount-time using the -oblocksize=... mount option.

Best regards,

Benjamin

--
You received this message because you are subscribed to the Google Groups "macFUSE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osxfuse-grou...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/osxfuse-group/CAErWVPfEhPH1BtYHGS6yOgk0uj_RXONf4w7%2B89YzOH1%2B%2B1Et7g%40mail.gmail.com.

[Joel Cretan]

Hi Benjamin,

Thanks for the reply. I don't see that option in the list:

https://github.com/macfuse/macfuse/wiki/Mount-Options

But regardless, I tried changing the -iosize and -oblocksize mount options with no effect. Testing has revealed that indeed the values are supposed to come from my struct statvfs*, as I can set values there that are respected if they're a power of 2. I think we tracked down the problem:

https://github.com/osxfuse/kext/blob/534773bd3135f04b7f6fb801faa2f8991daf4a6b/osxfuse/fuse_vfsops.c#L1131

Of course that's an old version, but does the current macFUSE kext also round the allocation block size up to a power of 2? Our filesystem, HFS standard, does not require this, and in fact depends on it, because the total number of blocks was limited to 65535, necessitating somewhat unusual block sizes to achieve different size volumes, especially large ones. After quite a few years, we only finally noticed this when it was exaggerated on some large disk images. Is there any way to avoid rounding in current macFUSE?

Thanks,

Joel

To view this discussion visit https://groups.google.com/d/msgid/osxfuse-group/ECACE570-7BDE-4FA4-8F1E-CC9A08ED5726%40gmail.com.

[Benjamin Fleischer]

Hi Joel,

It seems the blocksize mount-option has not been documented. You should still set it to the correct value if you are not using the default 4096 bytes. Otherwise the behavior of several file system callbacks might be undefined.

Of course that's an old version, but does the current macFUSE kext also round the allocation block size up to a power of 2?

No, the macFUSE kernel code still expects block sizes to be a power of 2. This has not been an issue in the past, at least I don’t remember it having been brought up. Apparently, macFUSE is more restrictive than it needs to be with regard to supported block sizes. I’m not sure if there are any (weaker) restrictions on block sizes imposed by the macOS kernel, though.

The macFUSE kernel code currently uses the block size (that is passed at mount-time) when performing reads and writes, in vnop_blockmap, vnop_blktooff and uses it as io_devblocksize for the backing (virtual) volume. macFUSE does not make a difference between logical block sizes and physical block sizes. That might be the reason for blocksize being rounded up to the next power of 2.

iosize should always be a multiple of the block size for performance reasons. iosize is currently rounded up to the next power of 2, too.

Best regards,

Benjamin

To view this discussion visit https://groups.google.com/d/msgid/osxfuse-group/CAErWVPeNHZOGe%3D27DK1fCYTwD%2BddixnYeaQNWM0cpGkieg2oqQ%40mail.gmail.com.

[Joel Cretan]

Hi Benjamin,

Thanks, good to know. This is helpful. We're leaning towards basically lying to the OS/macFUSE to get around this rounding, using a smaller power-of-2 block size and a larger number of blocks that result in the correct volume size, neither of which actually matches the real filesystem. When doing this, should the iosize and blocksize options match what we report in the struct statfs? (I decided to do the lying in the statfs_x() callback rather than statfs() to limit it to Apple)

This method seems to work to get the correct volume size but requires more testing for reliability of data. It seems like this should *probably* be safe since we're the only ones actually touching the blocks, the higher level callers just tell us to read part of a file and we go off and do that. I'm not worried about performance, it's not like a 30 year old disk is going to be used in a data center. Other than that, do you see any concerns about playing games with the block size?

Thanks!

Joel

To view this discussion visit https://groups.google.com/d/msgid/osxfuse-group/F9D947A3-1637-4B7C-886C-BE6ABBFD872D%40gmail.com.

[Benjamin Fleischer]

Hi Joel,

macFUSE 4.10.1 should no longer require you to "lie" to macOS with regard to the block size of your file system.

macFUSE tried to keep the device block size and the file system block size in sync. That’s why there was the power of 2 limitation for the file system block size. While there is a power of 2 limitation for the device block in the cluster I/O layer of the macOS kernel, there does not appear to be such a limitation for the file system block size in the kernel.

macFUSE 4.10.1 uses a default device block size of 4096 bytes on Intel Macs and 16384 bytes on Apple Silicon Macs. The device block size can be set at mount-time using the -oblocksize= option. I think the option has not been documented because using a value that exceeds the platform’s page size will result in data corruption for volumes that are not actually backed by a physical device. The kernel needs to set up some private parameters to support device block sizes that exceed the page size of the platform and it does so by querying the physical block device (that does not exist in most cases macFUSE is used)  

For performance reasons the file system block size should be a multiple of the device blocksize. Declaring a file system block size of 81920 bytes should not have any negative impact on performance since its a multiple of the (fake) device blocksize of 16384 bytes.

Please let me know if this works for you.

Best regards,

Benjamin

To view this discussion visit https://groups.google.com/d/msgid/osxfuse-group/CAErWVPe7B3wjnWvZAoqNGU0UCXpZs%2Bf2avpR0aE8UHC5UpAsSA%40mail.gmail.com.

[Joel Cretan]

Hi Benjamin,

Thanks so much for looking into this. It's so nice of you to change this for what appears to be only our obscure archaeological use case. 

If I'm understanding correctly, you default to device block sizes of 4096 or 16384 but now we should be able to override this with more precise values using -oblocksize? The filesystem block sizes I'm handling come in increments of 512, so may not necessarily be multiples of 4096 (I have some with 0xE00, 0x7E00, 0xC200, etc), and the value will in fact only be greater than 0x1000 for volumes larger than 250MB or so, which many old volumes are not. I've been developing against macFUSE 4.9.1 lately, and as of that version allocation blocks smaller than 4096 actually work fine and are reported correctly by diskutil et al. The filesystem does only allocate files in increments of the block size, so it shouldn't be a problem for I/O performance, not that I'm too worried about that anyway.

I see you've just released 4.10.1, so I will try that out and see how it fares with my weird blocks on different volumes. Thanks again for addressing this, I really appreciate it.

Best,

Joel

To view this discussion visit https://groups.google.com/d/msgid/osxfuse-group/3EB71861-610A-4D5C-BE6B-5240921C1320%40gmail.com.

[Benjamin Fleischer]

Hi Joel,

Please find my comments below.

If I'm understanding correctly, you default to device block sizes of 4096 or 16384 but now we should be able to override this with more precise values using -oblocksize?

Yes, but the device block size needs to be a power of 2. The minimum supported device block size is 128 B. I/O is usually performed in chunks that are multiples of the platform’s page size (due cluster I/O layer in the macOS kernel). That’s why the default device block size is the platform’s page size.

The filesystem block sizes I'm handling come in increments of 512, so may not necessarily be multiples of 4096 (I have some with 0xE00, 0x7E00, 0xC200, etc), and the value will in fact only be greater than 0x1000 for volumes larger than 250MB or so, which many old volumes are not.

Using 512 B as device block size should work.

Best regards,

Benjamin

To view this discussion visit https://groups.google.com/d/msgid/osxfuse-group/CAErWVPfaBUUQ98WdDvpiCPBDJ31_RAPubHYJLdH4nq%3DWwhcgAQ%40mail.gmail.com.