memfs issues (2018.1 B3)

[Stefan Boberg]

   Hi,

   First off thanks for creating WinFSP!

   I've been evaluating WinFSP for an application involving code compilation among other things. To this end I've tried both the passthrough file system and memfs to assess the viability of leveraging WinFSP.

   My memfs test case is this:

* Build memfs with some changes to MaxFileNodes (1024 * 256) and MaxFileSize (2 * 1024 * 1024 * 1024)

* Start memfs with -t -1 -m U:

* in U: -> git clone https://github.com/EpicGames/UnrealEngine UnrealClean

* cd UnrealClean

* .\Setup.bat (click No for the "Register Unreal Engine file types?" question)

* .\GenerateProjectFiles.bat

* .\UE4.sln

* try building the UE4 project

   I'm running into a few unexpected issues. 

* First off it seems like the file system is case sensitive by default, which is a bit weird for Windows. Passing in -i fixes this but perhaps it should default to case insensitive? It would at least have saved this guy some time as I wasn't really expecting a Windows file system to default to case sensitivity.

* After a few build steps the build will stop making forward progress (no CPU usage). I pretty consistently get four stalled link.exe processes and the memfs-x64.exe process sits at 7% CPU usage with 12 threads consuming some CPU each (0.5%)

   The memfs-x64 callstack:

0, ntoskrnl.exe!KiDeliverApc+0x171

1, ntoskrnl.exe!KiSwapThread+0x501

2, ntoskrnl.exe!KiCommitThreadWait+0x13b

3, ntoskrnl.exe!KeRemoveQueueEx+0x262

4, winfsp-x64.sys+0x1aed

5, winfsp-x64.sys+0x192f5

6, winfsp-x64.sys+0x140b1

7, ntoskrnl.exe!IofCallDriver+0x59

8, FLTMGR.SYS!FltpFsControl+0x151

9, ntoskrnl.exe!IofCallDriver+0x59

10, ntoskrnl.exe!IopSynchronousServiceTail+0x1ab

11, ntoskrnl.exe!IopXxxControlFile+0x66f

12, ntoskrnl.exe!NtFsControlFile+0x56

13, ntoskrnl.exe!KiSystemServiceCopyEnd+0x13

14, ntdll.dll!NtFsControlFile+0x14

15, KernelBase.dll!DeviceIoControl+0xc0

16, kernel32.dll!DeviceIoControlImplementation+0x80

17, winfsp-x64.dll!FspFileSystemDispatcherThread+0x142

18, kernel32.dll!BaseThreadInitThunk+0x14

19, ntdll.dll!RtlUserThreadStart+0x21

   The linker callstack:

0, ntoskrnl.exe!KiDeliverApc+0x171

1, ntoskrnl.exe!KiSwapThread+0x501

2, ntoskrnl.exe!KiCommitThreadWait+0x13b

3, ntoskrnl.exe!KeWaitForSingleObject+0x1ff

4, ntoskrnl.exe!FsRtlCancellableWaitForMultipleObjects+0xcb

5, ntoskrnl.exe!FsRtlCancellableWaitForSingleObject+0x27

6, FLTMGR.SYS!FltpLegacyProcessingAfterPreCallbacksCompleted+0x796

7, FLTMGR.SYS!FltpCreate+0x2dd

8, ntoskrnl.exe!IofCallDriver+0x59

9, ntoskrnl.exe!IopParseDevice+0x773

10, ntoskrnl.exe!ObpLookupObjectName+0x73b

11, ntoskrnl.exe!ObOpenObjectByNameEx+0x1df

12, ntoskrnl.exe!NtQueryFullAttributesFile+0x19e

13, ntoskrnl.exe!KiSystemServiceCopyEnd+0x13

14, ntdll.dll!NtQueryFullAttributesFile+0x14

15, KernelBase.dll!GetFileAttributesExW+0x95

16, ucrtbase.dll!waccess_s+0x4e

17, ucrtbase.dll!waccess+0x9

18, link.exe!SzSearchLibEnv+0x9d

19, link.exe!ProcessArgument+0xe4

20, link.exe!CObjFileBase::Release+0x299c4

21, link.exe!ProcessCommandFile+0x11f

22, link.exe!CObjFileBase::Release+0x3ab5

23, link.exe!LinkerMain+0xdc

24, link.exe!wmain+0x341

25, link.exe!MOD::FHasCorMeta+0x231

26, kernel32.dll!BaseThreadInitThunk+0x14

27, ntdll.dll!RtlUserThreadStart+0x21

   Any ideas what might be going wrong here? The build process is pretty parallel and I/O intensive and so might be exposing some race condition.

/Stefan

[Bill Zissimopoulos]

Stefan:

Thanks for the bug report.

Based on the information you give it appears that memfs-x64 is executing DeviceIoControl, which it does in order to fetch the next file system operation to process. That means that this particular memfs-x64 thread is idle, although we do not know what the other threads are doing.

The linker is just trying to execute GetFileAttributesExW, which requires opening the file to get its attributes. I suspect that the linker thread has just sent a CREATE request (IRP_MJ_CREATE) to the WinFsp FSD and for some reason this is not completing.

It is possible to send the stack traces of all memfs threads when this hang happens? BTW, WinFsp includes public symbols in the \Program Files (x86)\WinFsp\sym directory.

Bill

--
You received this message because you are subscribed to the Google Groups "WinFsp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to winfsp+un...@googlegroups.com.
To post to this group, send email to win...@googlegroups.com.
Visit this group at https://groups.google.com/group/winfsp.
For more options, visit https://groups.google.com/d/optout.

[Bill Zissimopoulos]

BTW, it does not appear that https://github.com/EpicGames/UnrealEngine is publicly available.

Bill

On 5/17/18, 2:58 AM, win...@googlegroups.com on behalf of Stefan Boberg wrote:

[Stefan Boberg]

   Yeah it's possible you need to apply to get access, I'm not entirely clear on the process nowadays -- it's free though. I'll investigate both that and the call stacks when I get back to work tomorrow morning (I'm in the EU :)

/Stefan