Hacking OSXFuse Advice

[arri]

Hi,

We're currently using a solution simiair to this one: https://groups.google.com/forum/#!msg/fuse4x/3Mqcdignv3c/qD-WQScgx5AJ

but instead of using bindfs we're using loopback, as it seemed a much simpeler, and therefore cleaner solution than bindfs.

After solving some issues, also similair to the ones mentioned in the above thread, we've finally got it working pretty stable. With the one exception that we see very frequent kernel-panics on the moment the mount gets unmounted.

We're using this solutuion in a more/less fixed workflow, and we know that at that particular moment, any application using the file system (other than those employed by the OS) have quit and it's our own application that issues the umount command to unmount the volume.

We're currently using Fuse4X, and in my ignorance i thought to just try and switch to using OSXFuse and i built a version of loopback for it. But this only raised more issues, rather than solving them.

Now i'd like some advice which direction to go.

For instance, how to fix the panic? Patching Fuse4x ? Or can it be fixed in the filesystem (loopback)?

Or is it best to switch to OSXFuse and patch that instead? (Or should it simply work there?)

Btw; I'm not exactly a rocket-scientist when it comes to kernel-programming/-debugging, but i'm not afraid to get wet.. So any pointers as to where in the code i'd look are welcome.

arri

[Benjamin Fleischer]

Hi,

After solving some issues, also similair to the ones mentioned in the above thread, we've finally got it working pretty stable. With the one exception that we see very frequent kernel-panics on the moment the mount gets unmounted.

Are you seeing these panics with Fuse4X or OSXFUSE? If you are getting panics using OSXFUSE please send me the crash reports.

We're currently using Fuse4X, and in my ignorance i thought to just try and switch to using OSXFuse and i built a version of loopback for it. But this only raised more issues, rather than solving them.

OSXFUSE 2.5.5 does not support preallocation. It will in the future but I can't give you an exact timeframe.

Now i'd like some advice which direction to go.

For instance, how to fix the panic? Patching Fuse4x ? Or can it be fixed in the filesystem (loopback)?

Or is it best to switch to OSXFuse and patch that instead? (Or should it simply work there?)

Panics need to be fixed in the kernel extension. A user space application should not be able to cause a kernel panic. I'd argue that OSXFUSE would be the logical choice (after it meets your needs) since Fuse4X will not see any updates. 

Btw; I'm not exactly a rocket-scientist when it comes to kernel-programming/-debugging, but i'm not afraid to get wet.. So any pointers as to where in the code i'd look are welcome.

The panic log should tell you most of what you need know to fix the issue. First of all you need the debug symbols for the kernel extension you are using. I don't know if they are publicly available for Fuse4X. For OSXFUSE you can find them in http://sourceforge.net/projects/osxfuse/files/. Then reproduce the panic and symbolicate it. After that you know where in the kernel code the crash occurred.

Regards,

Benjamin

--
You received this message because you are subscribed to the Google Groups "OSXFUSE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osxfuse-grou...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[arritje nof]

Hi Benjamin, thanks for your response!

The panics are with Fuse4x, for which i wont ask support..

But I've abandoned the osxfuse-way for now, due to time-constraints.

I will be coming back to this issue though, and will sure have a lot of additional questions.

When i wrote this last week i wasn't sure where to go, as going back often seems a bad idea, but it's the best thing to do for now.

Thanks for the advice on debugging the issue. I'll report back if anything comes up with osxfuse.

gr

arri

[arritje nof]

Well ok,

So yes, for our production-configuration i've decided to fallback on Fuse4x for now, but i think it's wise to also continue with trying to come to a stable soliution using OSXFuse.

So i actually have a very specific question. I think i could find the answer myself, but currently i simply haven't really got the time get into fuse-development at such a level.

I would like be able to test and compare OSXFuse with our current setup, which is based on Fuse4x 0.9.0 + patch (Fuse4x4Avid) and a modified version of loopback.

In order to do that, i need to apply (at least) one small but important change to the OSXFuse-kext: I need to Implement a stub for 'fuse_vnop_allocate' to satisfy Avid.

Obviously i've already tried to implement it by adding

FUSE_VNOP_EXPORT int fuse_vnop_allocate(struct vnop_allocate_args *ap) {

#pragma unused(ap)

return 0;

}

to fuse_vnops.c, and uncommenting it's declaration in the fuse_vnops.h. But appearantly it is not as easy as that (dûh.. ;) ...

So, looking further i also found

FUSE_VNOP_EXPORT int fuse_biglock_vnop_allocate(struct vnop_allocate_args *ap);

in fuse_biglock_vnops(.c/.h)which i uncommented/implemented in a similair fashion as above.

But then there is also attr->f_capabilities.capabilities in fuse_vfsops.h:876 which had VOL_CAP_INT_ALLOCATE commented-out.

So i un-commented that ...

Anyway, i could continue this way until i drop or maybe i'm lucky, but obviously i haven't got a clue what i should be really doing.. or even what the implications are of what i am doing.

So, my question is:

Can this functionality be implemented (enabled) easily? In such a way that a client application using the filesystem can pre-allocate disk-space (or at least think they can)?

To be clear (as you can also read in linked posts), it's not needed to implement any real code, just an emty function returning 0 (ok).

I'm not sure if you've read related posts, such as the one i linked earlier, but here are some a few for reference:

• implementing ioctl() to support fcntl() (for preallocating diskspace)

https://groups.google.com/forum/#!searchin/fuse4x/preallocate/fuse4x/ep4s-R5KUEM/DvrOnRmc3L0J

• ftruncate() and fuse

http://marc.info/?l=fuse-devel&m=131556528205666&w=1

• Fuse4x4Avid (

https://groups.google.com/forum/#!searchin/fuse4x/preallocate/fuse4x/3Mqcdignv3c/qD-WQScgx5AJ

And perhaps the most informative conversation (between Anatol Pomozov and Florian Bohgner)

> The problems is that we were not able to use Avid's "consolidate" function
> on bindfs "linked" volumes.
>
> Problem description
> After digging into the problem we found out that it's related to a fcntl
> call. This call is used to preallocate storage (F_PREALLOCATE).
> We looked for further information and found out, that fcntl is based onto
> ioctl (that's the reason I contacted you in the first step).
>
> Hours later I found out another
> detail: http://www.opensource.apple.com/source/xnu/xnu-1504.7.4/bsd/kern/kern_descrip.c
> F_PREALLOCATE is mainly related to VNOP_ALLOCATE. This call is currently not
> managed in fuse4x (haven't looked into the original fuse yet).

Ok so the real issue is implementing VNOP_ALLOCATE vnode operation.
The most natural way is
to add FUSE_ALLOCATE response code and send it to client. Then client
should process it. The full workflow
looks like this

1) user calls fcntl() function in libSystem
2) fcntl invokes ioctl() syscall
3) XNU converts F_PREALLOCATE ioctl to VNOP_ALLOCATE as you pointed
4) fuse4x kext handles it and send FUSE_ALLOCATE to fuse filesystem
5) fuse filesystem (e.g. bindfs) processes it
response goes the same path back to user

It is the best way but there are few issues:
 - there is no FUSE_ALLOCATE code in fuse yet, and I am not a big fan
of adding macosx-specific response codes. A good news is that Miklos
(the upstream developer) is fine with adding FUSE_ALLOCATE to
upstream. See this message
http://marc.info/?l=fuse-devel&m=131556528205666&w=1  Once it will be
added to libfuse, adding it to fuse4x-kext should be fairly trivial.
 - fuse filesystems still need to implement action for FUSE_ALLOCATE,
in case of bindfs it will look like:

bindfs_allocate() {
#ifdef _APPLE_
  fcntl(fd); // call preallocate on a real path (most likely it is
HFS+ filesystem)
#else
  // linux has different way
  posix_fallocate(fd); //?
#end
}

> The solution
> For us it's important to fix this problem. Therefore we contacted the
> HyperFS developers which had this problem in the past.
> They reported us, that it's ok (at least in our workflow to just report 0 -
> everything ok) in the VNOP_ALLOCATE handler function.
> Today I implemented this solution and it's working - we will test it over
> the next few days.
>
> I'm not quite sure if it's interesting for you to have a patch as I think
> it's not FUSE compliant - but would be glad to provide it if you like!

If this works for you then adding an empty "return success" handler
for VNOP_ALLOCATE is ok for me as well.
So send me the patch.

But we should cooperate with upstream developers and implement
preallocation in more correct way.

> Further work
> I hope i will be able to get involved into the fuse4x development - ioctl is
> still interesting for me!

It sounds really great! Looking forward to see more patches from you!

I'm guessing that for you it's should be rather straightforward to make a rough estimate 

of what needs to be done and how much time that would take. If it is, could you give 

me an overview of what needs to be done and what these changes implicate?

I'm not asking you to do it for me (i'm not stopping you either..;) but some directions to 

would be very welcome!

Thanks alot for your help!

regards,

arjen

[Benjamin Fleischer]

So, my question is:

Can this functionality be implemented (enabled) easily? In such a way that a client application using the filesystem can pre-allocate disk-space (or at least think they can)?

To be clear (as you can also read in linked posts), it's not needed to implement any real code, just an emty function returning 0 (ok).

Sorry for the late reply. You need to modify fuse_biglock_vnode_operation_entries in fuse_biglock_vnops.c like this

struct vnodeopv_entry_desc fuse_biglock_vnode_operation_entries[] = {
    ...
    { &vnop_allocate_desc,      (fuse_vnode_op_t) nop_allocate      },
   …

}

This should be all you need to do to fake allocate. To be clear, no space is allocated and the user space daemon won't receive a ALLOCATE message.

[arritje nof]

Hi Benjamin,

Thank you for this valuable info! This saved me alot of time. And i'd like to confirm that this is working beautifully!

Also, this helped us getting to where we currently are, which is pretty close to finally switching to the OSXFuse implementation instead of using Fuse4x.

Thanks a lot for your support!

gr

arjen