Overlapped reads of same file not handled by multiple IO threads in userspace FS
Bradley Schatz
Bill Zissimopoulos
SUMMARY
The WinFsp concurrency model allows for concurrent READs to the same file. It is therefore possible to get multiple concurrent READs on different threads. This of course assumes that other application(s) are issuing such READs to the file system and that the Cache Manager is not involved which may influence READ ordering.
Please read about the user mode locking strategy at this link and ensure that you correctly configure your file system.
https://github.com/billziss-gh/winfsp/blob/v1.7B1/inc/winfsp/winfsp.h#L117-L146
EXPLANATION
WinFsp applies locks both in kernel and user mode and for multiple purposes. Many of these locks are needed for the correct operation of WinFsp, but a few are also provided as a convenience to the user mode file system.
The following locks are significant for user mode file system developers. All such locks are exclusive/shared locks. It should be noted that kernel mode locks should be considered an implementation detail and that only the documented user mode locks should be relied upon to always be there.
Kernel mode (managed by the FSD):
- Rename lock: This is a *namespace* lock that is acquired shared during Create/Open and exclusive during Rename.
- I/O lock: This is a *per-file* lock that is acquired shared during read operations (e.g. READ, QUERYINFORMATION, etc.) and exclusive during write operations (e.g. WRITE, SETINFORMATION, etc.)
User mode (managed by the DLL):
- Operation guard lock: This is a *per-operation* lock that is managed according to the strategy specified by FspFileSystemSetOperationGuardStrategy.
Bill
--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
winfsp+un...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/winfsp/b8b69396-9880-4ddb-b0b1-a371ec9b20c7%40googlegroups.com.
Bradley Schatz
This of course assumes that other application(s) are issuing such READs to the file system and that the Cache Manager is not involved which may influence READ ordering.
Bradley
Bill Zissimopoulos
It depends how you are issuing them.
If you open the file handle in overlapped mode and also issue them using overlapped I/O then they will not be serialized.
OTOH, if you open the file handle in non-overlapped mode, then the I/O Manager takes locks on all I/O and therefore the operations will be serialized.
Bill
--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
winfsp+un...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/winfsp/a684f90f-2667-46f9-b8bf-069ae979a67f%40googlegroups.com.
Bradley Schatz
- (a) CreateFile with FILE_FLAG_OVERLAPPED and 16x ReadFile with OVERLAPPED, and
- (b) regular synchronous IO CreateFile with 16x ReadFile (no OVERLAPPED)
On Friday, 6 March 2020 18:39:39 UTC+10, Bill Zissimopoulos wrote:
It depends how you are issuing them.
If you open the file handle in overlapped mode and also issue them using overlapped I/O then they will not be serialized.
OTOH, if you open the file handle in non-overlapped mode, then the I/O Manager takes locks on all I/O and therefore the operations will be serialized.
Bill
On 3/6/20, 1:10 AM, win...@googlegroups.com on behalf of Bradley Schatz wrote:
Hi Bill,
Thanks for your detailed reply.
We are using FSP_FILE_SYSTEM_OPERATION_GUARD_STRATEGY_FINE
for the Per-Operation Guard Lock.
This of course assumes that other application(s) are issuing such READs to the file system and that the Cache Manager is not involved which may influence READ ordering.
The concurrent reads are all from the same thread and using the same file handle. Would that serialise the accesses?
Thanks,
Bradley
--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to winfsp+unsubscribe@googlegroups.com.
Bill Zissimopoulos
My apologies for the late response, but I was traveling.
I think the problem is that you are still doing caching reads. Caching reads work by attempting to copy memory directly from the Cache Manager; if that memory is not paged in, a page fault is incurred which issues a recursive READ to the file system and I believe (but do not know for certain as this is Windows internal code) that this paging READ is synchronous. In other words when using caching READ’s you are at the whims of the Cache Manager algorithms. (Recall that WinFsp uses the Cache Manager.)
You should open files with the FILE_FLAG_NO_BUFFERING flag to disable the cache for them. Running MEMFS with FileInfoTimeout != -1 should also do it.
If you still find that this does not work as expected, please consider opening an issue on GitHub together with your test program so that I can properly look at it. As far as I can remember there is (or should be) nothing in WinFsp that inherently limits shared READ’s so you may have discovered a bug.
Bill
To unsubscribe from this group and stop receiving emails from it, send an email to winfsp+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/winfsp/a684f90f-2667-46f9-b8bf-069ae979a67f%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
winfsp+un...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/winfsp/d85bd694-26bd-41ad-b1f4-929c90b36ca6%40googlegroups.com.