Sequential file reads

[Alexandros Paraskevaides]

Hi,

I'm working on a file system using jnr-fuse that needs to respond to read callbacks sequentially, so that the offset is always in line with what's been read so far. Using disk mounts, I've noticed different behaviour using different applications. For example:

- Java Buffered Reader:

java[TID=0bb0]: FFFFA609CB27BDB0: >>Create [UT---C] "\data_50_001.csv", FILE_OPEN, CreateOptions=60, FileAttributes=0, Security=NULL, AllocationSize=0:0, AccessToken=00000000000007D0[PID=16c8], DesiredAccess=120089, GrantedAccess=0, ShareAccess=3
java[TID=0bb0]: FFFFA609CB27BDB0: <<Create IoStatus=0[1] UserContext=0000000000000000:0000018991537FD0, GrantedAccess=120089, FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=0bb0]: FFFFA609CE3D7DB0: >>QueryInformation 0000000000000000:0000018991537FD0
java[TID=0bb0]: FFFFA609CE3D7DB0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=0bb0]: FFFFA609CA3D7AA0: >>QueryInformation 0000000000000000:0000018991537FD0
java[TID=0bb0]: FFFFA609CA3D7AA0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=0bb0]: FFFFA609CCE10DB0: >>QueryInformation 0000000000000000:0000018991537FD0
java[TID=0bb0]: FFFFA609CCE10DB0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=0bb0]: FFFFA609CC90A510: >>QuerySecurity 0000000000000000:0000018991537FD0
java[TID=0bb0]: FFFFA609CC90A510: <<QuerySecurity IoStatus=0[0] Security="O:S-1-0-65534G:S-1-0-65534D:P(A;;0x1f019f;;;S-1-0-65534)(A;;0x12018f;;;S-1-0-65534)(A;;0x12018f;;;WD)"
java[TID=0bb0]: FFFFA609CCE10DB0: >>QueryInformation 0000000000000000:0000018991537FD0
java[TID=0bb0]: FFFFA609CCE10DB0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=1778]: FFFFA609CCE10DB0: >>QuerySecurity 0000000000000000:0000018991537FD0
java[TID=1778]: FFFFA609CCE10DB0: <<QuerySecurity IoStatus=0[0] Security="O:S-1-0-65534G:S-1-0-65534D:P(A;;0x1f019f;;;S-1-0-65534)(A;;0x12018f;;;S-1-0-65534)(A;;0x12018f;;;WD)"
java[TID=1778]: FFFFA609CCE10DB0: >>QueryInformation 0000000000000000:0000018991537FD0
java[TID=1778]: FFFFA609CCE10DB0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=0bb0]: FFFFA609C7CAD530: >>Read 0000000000000000:0000018991537FD0, Address=0000018993FC0000, Offset=0:0, Length=32768, Key=0
java[TID=0bb0]: FFFFA609C7CAD530: <<Read IoStatus=0[32768]
java[TID=0bb0]: FFFFA609C7CB8010: >>Read 0000000000000000:0000018991537FD0, Address=0000018993FC0000, Offset=0:862000, Length=32768, Key=0
java[TID=0bb0]: FFFFA609C7CB8010: <<Read IoStatus=c000000d[0]
java[TID=0bb0]: FFFFA609C7CAD530: >>Read 0000000000000000:0000018991537FD0, Address=0000018993FC0000, Offset=0:f000, Length=32768, Key=0
java[TID=0bb0]: FFFFA609C7CAD530: <<Read IoStatus=c000000d[0]
java[TID=0bb0]: FFFFA609CA3D1DD0: >>Read 0000000000000000:0000018991537FD0, Address=0000018993FC0000, Offset=0:862000, Length=32768, Key=0
java[TID=0bb0]: FFFFA609CA3D1DD0: <<Read IoStatus=c000000d[0]
java[TID=0bb0]: FFFFA609CACDDAA0: >>QueryInformation 0000000000000000:0000018991537FD0
java[TID=0bb0]: FFFFA609CACDDAA0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=0bb0]: FFFFA609CACDDAA0: >>QueryInformation 0000000000000000:0000018991537FD0
java[TID=0bb0]: FFFFA609CACDDAA0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86a000, FileSize=0:869fd0, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:34:47.000Z, LastWriteTime=2023-03-30T13:34:47.000Z, ChangeTime=2023-03-30T13:34:45.000Z, IndexNumber=0:0}
java[TID=0bb0]: FFFFA609CC90A510: >>Read 0000000000000000:0000018991537FD0, Address=0000018993FC0000, Offset=0:0, Length=8192, Key=0
java[TID=0bb0]: FFFFA609CC90A510: <<Read IoStatus=c000000d[0]
java[TID=1778]: FFFFA609CAF48010: >>Close 0000000000000000:0000018991537FD0

Notepad++:

java[TID=0cdc]: FFFFA609CAD20AA0: >>Create [UT---C] "\data_50_10000.csv", FILE_OPEN, CreateOptions=60, FileAttributes=0, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000798[PID=2d00], DesiredAccess=120089, GrantedAccess=0, ShareAccess=3
java[TID=245c]: FFFFA609C9FEF790: <<Close IoStatus=0[0]
java[TID=0cdc]: FFFFA609CAD20AA0: <<Create IoStatus=0[1] UserContext=0000000000000000:000001937C4B1D70, GrantedAccess=120089, FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:28.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=245c]: FFFFA609C9FEF790: >>QueryInformation 0000000000000000:000001937C4B1D70
java[TID=245c]: FFFFA609C9FEF790: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:28.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=0cdc]: FFFFA609C9FEF790: >>QueryInformation 0000000000000000:000001937C4B1D70
java[TID=0cdc]: FFFFA609C9FEF790: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:28.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=245c]: FFFFA609C9FEF790: >>QueryInformation 0000000000000000:000001937C4B1D70
java[TID=245c]: FFFFA609C9FEF790: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:28.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=0cdc]: FFFFA609CA407DE0: >>QuerySecurity 0000000000000000:000001937C4B1D70
java[TID=0cdc]: FFFFA609CA407DE0: <<QuerySecurity IoStatus=0[0] Security="O:S-1-0-65534G:S-1-0-65534D:P(A;;0x1f019f;;;S-1-0-65534)(A;;0x12018f;;;S-1-0-65534)(A;;0x12018f;;;WD)"
java[TID=245c]: FFFFA609CA407DE0: >>QueryInformation 0000000000000000:000001937C4B1D70
java[TID=245c]: FFFFA609CA407DE0: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:28.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=0cdc]: FFFFA609CA407DE0: >>QuerySecurity 0000000000000000:000001937C4B1D70
java[TID=0cdc]: FFFFA609CA407DE0: <<QuerySecurity IoStatus=0[0] Security="O:S-1-0-65534G:S-1-0-65534D:P(A;;0x1f019f;;;S-1-0-65534)(A;;0x12018f;;;S-1-0-65534)(A;;0x12018f;;;WD)"
java[TID=0cdc]: FFFFA609C9FEF790: >>QueryInformation 0000000000000000:000001937C4B1D70
java[TID=0cdc]: FFFFA609C9FEF790: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:28.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=245c]: FFFFA609CA7C7530: >>Read 0000000000000000:000001937C4B1D70, Address=000001937EE90000, Offset=0:0, Length=32768, Key=0
java[TID=245c]: FFFFA609CA7C7530: <<Read IoStatus=0[32768]
java[TID=245c]: FFFFA609CE2B8530: >>Read 0000000000000000:000001937C4B1D70, Address=000001937EE90000, Offset=0:865000, Length=32768, Key=0
java[TID=245c]: FFFFA609CE2B8530: <<Read IoStatus=c000000d[0]
java[TID=245c]: FFFFA609CA5D9320: >>Read 0000000000000000:000001937C4B1D70, Address=000001937EE90000, Offset=0:f000, Length=32768, Key=0
java[TID=245c]: FFFFA609CA5D9320: <<Read IoStatus=c000000d[0]
java[TID=245c]: FFFFA609CA5D9320: >>Read 0000000000000000:000001937C4B1D70, Address=000001937EE90000, Offset=0:865000, Length=32768, Key=0
java[TID=245c]: FFFFA609CA5D9320: <<Read IoStatus=c000000d[0]
java[TID=245c]: FFFFA609CE28C290: >>QueryInformation 0000000000000000:000001937C4B1D70
java[TID=245c]: FFFFA609CE28C290: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:29.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=0cdc]: FFFFA609C9963560: >>QueryInformation 0000000000000000:000001937C4B1D70
java[TID=0cdc]: FFFFA609C9963560: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:29.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=0cdc]: FFFFA609CAD20AA0: >>Read 0000000000000000:000001937C4B1D70, Address=000001937F0A0000, Offset=0:0, Length=131072, Key=0
java[TID=0cdc]: FFFFA609CAD20AA0: <<Read IoStatus=c000000d[0]
java[TID=0cdc]: FFFFA609CAB65050: >>Close 0000000000000000:000001937C4B1D70
java[TID=245c]: FFFFA609CA5D9320: >>Create [UT---C] "\data_50_10000.csv", FILE_OPEN, CreateOptions=200000, FileAttributes=0, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000814[PID=2d00], DesiredAccess=80, GrantedAccess=0, ShareAccess=7
java[TID=0cdc]: FFFFA609CAB65050: <<Close IoStatus=0[0]
java[TID=245c]: FFFFA609CA5D9320: <<Create IoStatus=0[1] UserContext=0000000000000000:000001937D982480, GrantedAccess=80, FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:29.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=245c]: FFFFA609CE28C290: >>QueryInformation 0000000000000000:000001937D982480
java[TID=245c]: FFFFA609CE28C290: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:29.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=0cdc]: FFFFA609CE28C290: >>QueryInformation 0000000000000000:000001937D982480
java[TID=0cdc]: FFFFA609CE28C290: <<QueryInformation IoStatus=0[0] FileInfo={FileAttributes=0, ReparseTag=0, AllocationSize=0:86d000, FileSize=0:86ca06, CreationTime=1970-01-02T00:00:00.000Z, LastAccessTime=2023-03-30T13:38:29.000Z, LastWriteTime=2023-03-25T15:40:53.000Z, ChangeTime=2023-03-25T15:40:51.000Z, IndexNumber=0:0}
java[TID=0cdc]: FFFFA609CCB8BDB0: >>Close 0000000000000000:000001937D982480
java[TID=0cdc]: FFFFA609CCB8BDB0: <<Close IoStatus=0[0]

Notepad: (seems to release the file before reading)

java[TID=28b8]: FFFFA609CCC7F2B0: >>Create [UT---C] "\data_50_10000.csv", FILE_OPEN, CreateOptions=200000, FileAttributes=0, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000798[PID=1410], DesiredAccess=80, GrantedAccess=0, ShareAccess=7
java[TID=12ec]: FFFFA609CCE14530: <<Close IoStatus=0[0]
java[TID=12ec]: FFFFA609CA687AA0: >>Read 0000000000000000:0000020AD94F54B0, Address=0000020ADBF40000, Offset=0:0, Length=32768, Key=0
java[TID=12ec]: FFFFA609CA687AA0: <<Read IoStatus=c0000008[0]

After reading issues from the WinFSP git page and this google group, I thought that perhaps setting the drive as a network drive instead might help. It did help with the constant open release operations, as well as jumping offsets, however, I'm now getting an access violation after the 3rd-4th read:

java[TID=189c]: FFFFA609C9F6DA80: >>Read 0000000000000000:0000027D5B9CDFC0, Address=0000027D5CF90000, Offset=0:0, Length=32768, Key=0
java[TID=189c]: FFFFA609C9F6DA80: <<Read IoStatus=0[32768]
java[TID=0d74]: FFFFA609CE1BEA10: >>Read 0000000000000000:0000027D5B9CDFC0, Address=0000027D5CF90000, Offset=0:8000, Length=32768, Key=0
java[TID=0d74]: FFFFA609CE1BEA10: <<Read IoStatus=0[32768]
java[TID=189c]: FFFFA609C56BF010: >>Read 0000000000000000:0000027D5B9CDFC0, Address=0000027D5CF90000, Offset=0:10000, Length=65536, Key=0
java[TID=189c]: FFFFA609C56BF010: <<Read IoStatus=0[65536]
java[TID=0d74]: FFFFA609C9C66A90: >>Read 0000000000000000:0000027D5B9CDFC0, Address=0000027D5D1A0000, Offset=0:20000, Length=131072, Key=0
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000027d4962bce3, pid=6536, tid=3444
#
# JRE version: OpenJDK Runtime Environment Microsoft-7208460 (11.0.18+10) (build 11.0.18+10-LTS)
# Java VM: OpenJDK 64-Bit Server VM Microsoft-7208460 (11.0.18+10-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# J 3139 c2 jnr.ffi.provider.jffi.DirectMemoryIO.putInt(JI)V (14 bytes) @ 0x0000027d4962bce3 [0x0000027d4962bca0+0x0000000000000043]

if I only read half the size, the access violation doesn't occur, and in the case of Notepad++ it completely resolves all issues (although it's far from ideal), in the case of notepad however, there's some cases where the offset jumps the whole size instead of the half size that is returned. 

Not sure which direction I shouild take to deal with this. Any suggestions?

Many thanks,

Alex

[Bill Zissimopoulos]

> I'm working on a file system using jnr-fuse that needs to respond to

> read callbacks sequentially, so that the offset is always in line

> with what's been read so far. Using disk mounts, I've noticed different

> behaviour using different applications.

 

Unfortunately this is not possible. The file system interface allows for random access and therefore your file system must be ready to accept file offsets that are not sequential.

 

Even if you control the application that issues the I/O and can guarantee that it will only perform I/O sequentially, there are many other components that may issue requests. For example, AntiVirus or other filter software may issue its own I/O requests. Even the operating system kernel may issue its own requests under some scenarios (e.g. if in-kernel data caching is enabled).

 

There is no way around it: your file system must be able to handle random access requests. Now if your storage backend can only handle sequential requests, you may have to introduce an intermediate caching layer that reads files sequentially upon first access and is then able to serve them using random access.

 

Quite a few file systems work with this basic idea. For example, my own file systems objfs and hubfs (you can find them under https://github.com/orgs/winfsp/repositories), rclone, etc. I also suggest reading up on the Andrew file system, which was an early example of this technique.

 

Thanks.

 

Bill

--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to winfsp+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/7d6bdec7-50c1-4a21-bafc-dc3f42f84715n%40googlegroups.com.

[Bill Zissimopoulos]

The in-kernel data caching is only enabled if you supply the option FileInfoTimeout=-1. So it should not be on by default. However you cannot (easily) disable filters and they will make their own I/O requests that will likely not fit your sequential I/O access pattern.

 

Frankly I do not think it is possible to write a file system on Windows that does not do random access. It will fail more often than not.

 

I understand the concern about keeping decrypted data in a cache. So keep them encrypted and only decrypt when you need to service I/O. If you always encrypt/decrypt from the beginning of the file, you may have to come up with a chunking scheme for your files (e.g. maintain your files as 64K blocks and encrypt/decrypt them per block).

 

I do not know why you get an access violation when you enable network drives. I am not aware of any such problems in the base WinFsp-FUSE library. You may have to check with jnr-fuse.

 

Bill

 

 

 

From: win...@googlegroups.com <win...@googlegroups.com> On Behalf Of Alexandros Paraskevaides
Sent: Thursday, March 30, 2023 5:30 PM
To: WinFsp <win...@googlegroups.com>

Subject: Re: [winfsp] Sequential file reads

 

Hi Bill,

 

Thank you for your response.

 

Unfortunately, data caching is not an option as the purpose of this virtual file system is to provide a way to read encrypted data. If the decrypted data is stored somewhere, even in memory, it introduces extra vulnerabilities unnecessarily. The virtual file system therefore checks whether the reads are sequential, and if not, it returns some sort of error. 

 

I understand that some applications might try to access the files randomly, and i'm happy to not support those applications and focus on the ones that do. Notepad++ for example does handle file reads sequentially, so at the very least, that should work fine. Is there a way to disable in-kernel data caching, and overall restrict the amount of times files are accessed non-sequentially?

 

Setting the vfs as a network drive has improved the behaviour, however, for some reason, I am getting an access violation even though I'm only serving as much as what should already be allocated (size parameter).

 

Regards,

Alex

To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/d7fcc05f-6994-4747-ae66-4a3aa7678bc2n%40googlegroups.com.