Java implementation of WinFsp Native API

[Ricardo Fonseca]

Hi.

Apologies if this is not the right place to ask this question.

I am looking for a way to implement the WinFsp Native API calls in Java. Do you know of any open-source library that provides that? There is jnr-fuse but it only seems to implement the POSIX FUSE calls.

If not, I am willing to try implement one myself (should be fun!).

Thanks.

[Bill Zissimopoulos]

Ricardo, apologies for the late response.

 

I am not aware of any native Java implementation of the WinFsp native API. It would be great if you attempted a Java port and I would certainly support such an effort.

 

Bill

--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to winfsp+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/d5b24139-c7ea-4042-a505-67e723e9b83en%40googlegroups.com.

[Ricardo Fonseca]

Hi Bill.

Thanks for the support.

I am trying to mimic what jnr-fuse does while following your tutorial and I got mixed results. I managed to bridge some of the functions in winfsp.h to Java using JNR:

• FspFileSystemPreflight
• FspFileSystemCreate
  

but when I call the FspFileSystemPreflight function with an empty folder path and "WinFsp.Disk" as arguments I get an NTSTATUS C000000E (STATUS_NO_SUCH_DEVICE).

I've looked through your samples but couldn't find any usage of the FspFileSystemPreflight function so maybe I'm missing something that needs to be done before calling that function.

Also, the FUSE API has a point of entry at the fuse_main_real function, which performs many useful operations before mounting the file system. I couldn't find any similar function for the native API.

My questions to you are:

1. Does the STATUS_NO_SUCH_DEVICE error code mean I need to perform some operation before calling the FspFileSystemPreflight function?
2. Are there any plans for adding a function similar to fuse_main_real for the native API?
3. In order to call FspFileSystemCreate do I always need to precede it with the same steps as in samples\passthrough\passthrough.c, such as opening the file handle, retrieving information and closing it?

Cheers,

Ricardo

[Bill Zissimopoulos]

Hello, Ricardo:

 

Re: FspFileSystemPreflight. You do not need to use this API. It is only used to perform preliminary checks, such as whether the file system driver is available, etc. I use it in cases where file system initialization is expensive: if FspFileSystemPreflight does not succeed, it is not necessary to go through the expensive file system initialization.

 

If you use this API either specify an actual drive letter as a mountpoint (e.g. X:) or NULL. The STATUS_NO_SUCH_DEVICE error code likely means that the device driver was not loaded correctly and the primary WinFsp device was not created for some reason. Another likely reason is that you did not pass the correct device name (just pass "WinFsp.Disk" if you want a "disk" file system or "WinFsp.Net" if you want a "network" file system).

 

Re: fuse_main_real. This function performs a lot of steps that are necessary only for FUSE file systems. You do not need to perform all these steps in a native file system.

 

The necessary steps for a native file system are:

 

-      FspFileSystemCreate: Creates the file system in kernel and also prepares necessary data structures for serving file system requests in user mode.

-      FspFileSystemSetMountPoint: Specifies the mountpoint. This is usually a drive letter (e.g. X:) but can also be a directory.

-      FspFileSystemStartDispatcher: This starts a number of threads to service file system requests.

 

When you are done with the file system shut it down as follows:

 

-      FspFileSystemStopDispatcher: This shuts down the file system threads in an orderly manner.

-      FspFileSystemRemoveMountPoint: Removes the file system mountpoint.

-      FspFileSystemDelete: Deletes any user mode data structures and also removes the in-kernel file system.

 

Additionally WinFsp provides optional support for easily creating Windows services (because file systems often work best as Windows services). The passthrough tutorial should explain those steps for you.

 

Re: answers to your questions:

 

1.     Likely you are not passing the correct device name. See my comments above.

2.     No because the steps performed are specific to FUSE.

3.     You do not need to open any files as in passthrough. These steps are specific to passthrough, because it exposes an existing directory as a new file system.

 

Bill

 

 

 

From: 'Ricardo Fonseca' via WinFsp <win...@googlegroups.com>
Sent: Tuesday, July 6, 2021 8:53 PM
To: WinFsp <win...@googlegroups.com>
Subject: Re: [winfsp] Java implementation of WinFsp Native API

 

Hi Bill.

 

Thanks for the support.

 

I am trying to mimic what jnr-fuse does while following your tutorial and I got mixed results. I managed to bridge some of the functions in winfsp.h to Java using JNR:

·       FspFileSystemPreflight

·       FspFileSystemCreate

 

but when I call the FspFileSystemPreflight function with an empty folder path and "WinFsp.Disk" as arguments I get an NTSTATUS C000000E (STATUS_NO_SUCH_DEVICE).

 

I've looked through your samples but couldn't find any usage of the FspFileSystemPreflight function so maybe I'm missing something that needs to be done before calling that function.

 

Also, the FUSE API has a point of entry at the fuse_main_real function, which performs many useful operations before mounting the file system. I couldn't find any similar function for the native API.

 

My questions to you are:

1.       Does the STATUS_NO_SUCH_DEVICE error code mean I need to perform some operation before calling the FspFileSystemPreflight function?

2.       Are there any plans for adding a function similar to fuse_main_real for the native API?

3.       In order to call FspFileSystemCreate do I always need to precede it with the same steps as in samples\passthrough\passthrough.c, such as opening the file handle, retrieving information and closing it?

 

Cheers,

 

Ricardo

 

On Sunday, 4 July 2021 at 11:58:59 UTC+1 bill...@navimatics.com wrote:

Ricardo, apologies for the late response.

 

I am not aware of any native Java implementation of the WinFsp native API. It would be great if you attempted a Java port and I would certainly support such an effort.

 

Bill

 

 

From: 'Ricardo Fonseca' via WinFsp <win...@googlegroups.com>
Sent: Friday, July 2, 2021 3:28 PM
To: WinFsp <win...@googlegroups.com>
Subject: [winfsp] Java implementation of WinFsp Native API

 

Hi.

 

Apologies if this is not the right place to ask this question.

 

I am looking for a way to implement the WinFsp Native API calls in Java. Do you know of any open-source library that provides that? There is jnr-fuse but it only seems to implement the POSIX FUSE calls.

 

If not, I am willing to try implement one myself (should be fun!).

 

Thanks.

--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to winfsp+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/d5b24139-c7ea-4042-a505-67e723e9b83en%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to winfsp+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/b567d520-6c17-489a-be57-937c48a66411n%40googlegroups.com.

[Ricardo Fonseca]

Hi Bill.

Thank you for your help, it looks like I was passing strings from Java to C incorrectly and so WinFsp wasn't recognising the "WinFsp.Disk" string properly. Now that's fixed.

I managed to create the file system, mount it to a folder, and start the dispatcher. However, I can only see the folder appear briefly before my test application aborts with a STATUS_ACCESS_VIOLATION.

Do you know what might be causing it? Do I need to implement a minimal subset of the file interface functions before it can work properly?

I haven't managed to turn debugging on yet because I still don't have a way to pass a valid file handler (stderr in my case) from Java through JNR to your FspDebugLogSetHandle function. Is there an alternative way to turn debugging on so it goes to stderr without me having to pass a file handle?

Thanks again for your help.

Cheers,

Ricardo

[Ricardo Fonseca]

I managed to pass the stderr file handler from Java through JNR to the FspDebugLogSetHandle function, however I don't see any debug logs. Maybe this function doesn't turn debugging on as I thought?

Ricardo

[Ricardo Fonseca]

Hi Bill.

I managed to solve the STATUS_ACCESS_VIOLATION issue! Turns out I had to pass JNR pointers (instead of values) to the FspFileSystemCreate function, so that's a major relief!

I still can't turn debugging on, though. Not sure what I'm doing wrong.

Cheers,

Ricardo

[Bill Zissimopoulos]

Ricardo:

 

Apologies for the late response. I am currently on vacation.

 

Good to hear that you managed to resolve the access violation.

 

Regarding getting debug output: The FspDebugSetLogHandle determines where debug output goes, but does not turn it on. To turn it on you must call FspFileSystemSetDebugLog(FileSystem, DebugLog) and pass it a value of DebugLog==-1 (which will instruct the file system to log all requests/responses).

 

Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/a7620c51-ea60-4dd4-9313-6d552ab23422n%40googlegroups.com.

[Ricardo Fonseca]

Hi Bill,

Thanks again for your help.

I couldn't map the FspDebugSetLogHandle function through JNR for some reason (I noticed it was defined as a static inline function in winfsp.h, so that may have something to do with it).

Fortunately I managed to map the FspDebugSetLogHandleF function (with an extra F at the end of the name) through JNR and call that function instead, which when combined with a previous call to FspDebugSetLogHandle now gives me debug output to stderr.

Cheers and have a nice vacation,

Ricardo

[Bill Zissimopoulos]

You are right. Some of these are inline functions and therefore not directly available to languages other than C/C++. For this reason I have created some "out-of-line" functions with the "F" suffix for use by languages such as Java, Python, etc.

To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/b8422988-30e2-4ae1-9038-7ac13f493156n%40googlegroups.com.

[Ricardo Fonseca]

Hi Bill,

Thanks for confirming my hunch about the inline functions.

I have implemented GetVolumeInfo, Create, Open, GetFileInfo and ReadDirectory, so far.

Looking at my test output, I can see GetVolumeInfo getting called when I view the mount point in explorer.exe. Also, looking at the winfsp debug output, I see GetVolumeInfo getting called with no errors (the total/free disk space values and volume label I assign are correct).

However, I'm having the following issues:

(1) No other calls that I implemented are being called since I cannot see any test output for those.

(2) I can see in the winfsp debug output the Create function being called and returning 0xC0000010 (STATUS_INVALID_DEVICE_REQUEST), despite not being able to capture it via callback yet (there was no test output for it).

(3) When I try opening the drive folder (same thing if the mount point is a device letter) I get an error saying "is not accessible (incorrect function)", and the debug output contains the same Create lines with the 0xC0000010 error.

Do you know what might be causing these issues?

Thanks.

Here are the first few lines of the debug log:

java[TID=32c4]: FFFFC48F9E58C9D0: >>QueryVolumeInformation
java[TID=32c4]: FFFFC48F9E58C9D0: <<QueryVolumeInformation IoStatus=0[0] VolumeInfo={TotalSize=3c:e8a8e000, FreeSize=31:bde08000, VolumeLabel="TESTDRIVE"}
java[TID=32c4]: FFFFC48F9E576100: >>Create [UT----] "\", FILE_OPEN, CreateOptions=21, FileAttributes=10, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000XXX[PID=1c68], DesiredAccess=100080, GrantedAccess=0, ShareAccess=3
java[TID=32c4]: FFFFC48F9E576100: <<Create IoStatus=c0000010[0]
java[TID=32c4]: FFFFC48F9E58EC50: >>Create [UT----] "\", FILE_OPEN, CreateOptions=11, FileAttributes=10, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000XXX[PID=1c68], DesiredAccess=100080, GrantedAccess=0, ShareAccess=3
java[TID=32c4]: FFFFC48F9E58EC50: <<Create IoStatus=c0000010[0]
java[TID=32c4]: FFFFC48F9E58E970: >>Create [UT----] "\", FILE_OPEN, CreateOptions=21, FileAttributes=10, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000XXX[PID=1c68], DesiredAccess=100080, GrantedAccess=0, ShareAccess=3
java[TID=32c4]: FFFFC48F9E58E970: <<Create IoStatus=c0000010[0]
java[TID=32c4]: FFFFC48F9E5821A0: >>Create [UT----] "\", FILE_OPEN, CreateOptions=11, FileAttributes=10, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000XXX[PID=1c68], DesiredAccess=100080, GrantedAccess=0, ShareAccess=3
java[TID=32c4]: FFFFC48F9E5821A0: <<Create IoStatus=c0000010[0]

Cheers,

Ricardo

[Ricardo Fonseca]

Hi Bill,

I have released an open source library called jnr-winfsp (inspired by jnr-fuse) here: https://github.com/jnr-winfsp-team/jnr-winfsp

The plan is to have it available for anyone to use, however at the moment I am still getting the same error I explained before, namely the 0xC0000010 (STATUS_INVALID_DEVICE_REQUEST) for Create operations called immediately when I mount the local disk drive.

To perform the quick test on the command line, we need to install Maven ( https://maven.apache.org/ ) and then run the following commands at the top directory of the repository:

mvn test-compile
mvn -D"exec.mainClass=com.github.jnrwinfspteam.jnrwinfsp.MainTest" -D"exec.classpathScope=test" exec:java

The above will mount a local disk drive at Z: (or the next available device letter downwards).

Cheers,

Ricardo

[Ricardo Fonseca]

Hi Bill,

I finally managed to overcome the problem I was having with the Create operations :)

Turns out I only had to implement one more function (Overwrite) in order to make it work. This line in WinFsp helped me debug the issue: https://github.com/billziss-gh/winfsp/blob/6e3a8f70b2bd958960012447544d492fc6a2f1af/src/dll/fsop.c#L900

Cheers,

Ricardo

[Ricardo Fonseca]

Hi Bill,

I have now a rudimentary memfs in place with all operations implemented, except security functions, reparse point functions and stream functions.

Creating directories and files seems to work with no problems and theoretically so does the file writing/reading part. I say theoretically because it turns out WinFsp is always calling the write operation with the flag "constrainedIO" set to true for some reason.

Is this normal behaviour? Please note that due to that flag always being set to true, I am not allowed to ever change the file size, which makes it impossible to put data into a file.

Cheers,

Ricardo

[Ricardo Fonseca]

Hi Bill,

I solved the issue with the constrainedIO flag. It turns out JNR is not properly converting a windows BOOLEAN to a Java boolean, so my fix was to define the argument type as a byte and perform the conversion to boolean manually. This same issue popped up elsewhere, which was surprisingly difficult to debug, but now that's done and it's working fine.

The memfs implementation in jnr-winfsp now passes all tests from winfsp-tests, except the stream* and reparse* tests (since I have not implemented stream and reparse point operations), and also two other tests.

All executed tests pass with the following command line arguments I am passing to winfsp-tests-x64.exe:

--external --resilient -stream* -reparse* -getfileinfo_name_test -dirnotify_test

As you can see, I am skipping two tests which are failing. I am not sure why dirnotify_test fails, but what's stranger is the fact that getfileinfo_name_test is also failing with winfsp's own memfs mounted as an external drive. To be clear, I have:

1. mounted memfs from winfsp with the arguments -m Z:
   
2. copied winfsp-tests-x64.exe and winfsp-x64.dll to Z:
3. ran the test (in Z:) with the arguments --external getfileinfo_name_test

The above fails with the following assert error:

getfileinfo_name_test.................. KO
    ASSERT(INVALID_HANDLE_VALUE != Handle) failed at info-test.c:342:getfileinfo_name_dotest

Could there be a bug in that test of yours?

Cheers,

Ricardo