Hi, In order to answer that question, you first need to know how your target app runs under WinAFL. When you select a target function and fuzz the app this happens:
1. Your target runs normally until your target function is reached.
2. WinAFL starts recording coverage
3. Your target function runs until return
4. WinAFL reports coverage, rewrites the input file and patches EIP so that the execution jumps back to step 2
5. After your target function runs for the specified number of iterations, the target process is killed and restarted. Notice that anything that runs after the target function returns is never reached.
The target function should do these things during its life
1. Open the input file (This needs to happen withing the target function so that you can read a new input file for each iteration as the input file is rewritten between target function runs).
2. Parse it (so that you can measure coverage of file parsing)
3. Close the input file (This is important because if the input file is not closed WinAFL won't be able to rewrite it)
4. Return normally (So that WinAFL can "catch" this return and redirect execution. "returning" via ExitProcess() and such won't work)
Whether the target shows GUI during some of these steps is irrelevant. What matters is whether your target requires user interaction between some steps above. If
a) you have a target function that behaves like the above and
b) your target does not require any user interaction until the end of the target function is reached
you're in luck and you can use WinAFL out of the box. If this is not the case, then unfortunately you need to either patch your target of WinAFL to suite the specific need of your target.
Terminating a process after each fuzzing iteration is possible, but WinAFL gets much of its speed due to the fact that it does *not* terminate the process for each fuzzing iteration and you'd be sacrificing that.